c++opencvimage-processingsignal-processingimage-resizing

CPU bilinear filtering output same results than point sampling


I am adding bilinear filtering to my C++ rendering software. To validate that it works I am trying to downscale a test image.

This is the 1024x1024 test image:

Native image

When I perform point sampling resizing I get this:

Point sampling resized image

When I perform bilinear resizing I unfortunately get almost the same aliased result. Why?:

Bilinear sampling resized image

This is how the straightforward resizing is performed:

const Graphics::Texture::SharedRGBAImagePtr input_image = Texture::Helper::createRGBAImage("D:\\resizing_test\\input_image.png");

const uint32_t destWidth = input_image->getWidth() / 16;
const uint32_t destHeight = input_image->getHeight() / 16;
Texture::RGB32FImage* resizedImage = new Texture::RGB32FImage(destWidth, destHeight);

for (uint32_t j = 0u; j < destHeight; ++j)
{
    for (uint32_t i = 0u; i < destWidth; ++i)
    {
        const float xnorm = (float)i / (float)destWidth;
        const float ynorm = (float)j / (float)destHeight;

#if 1
        // Point sampling.
        const float xpointSrcFloat = xnorm * ((nbFloat32)input_image->getWidth() - 1.0f);
        const float ypointSrcFloat = ynorm * ((nbFloat32)input_image->getHeight() - 1.0f);

        const Graphics::RGBAFColor color = input_image->getNormalizedPixelFromPosition(Math::Uvec2((uint32_t)xpointSrcFloat, (uint32_t)ypointSrcFloat));
        resizedImage->setPixelFromPosition(Graphics::RGBFColor(color.x, color.y, color.z), Math::Uvec2(i, j));

#else
        // Bilinear sampling.
        const Graphics::RGBAFColor color = input_image->getNormalizedPixelFromRatio(Math::Vec2(xnorm, ynorm));
        resizedImage->setPixelFromPosition(Graphics::RGBFColor(color.x, color.y, color.z), Math::Uvec2(i, j));
#endif
    }
}

resizedImage->save("D:\\resizing_test\\ouput_image.png");

The definition of getNormalizedPixelFromPosition.

template <typename T>
inline T TImage<T>::getNormalizedPixelFromPosition(const Math::Uvec2& pos) const
{
    const float div = isFloatingPointImage() ? 1.0f : 255.0f;
    return getPixelFromPosition(pos) / div;
}

The definition of getNormalizedPixelFromRatio. This performs bilinear filtering:

template <typename T>
inline T TImage<T>::getNormalizedPixelFromRatio(const Math::Vec2& ratio) const
{
    const uint32_t widthMinusOne = getWidth() - 1u;
    const uint32_t heightMinusOne = getHeight() - 1u;

    // The sampling weights.
    const Math::Vec2 C00Float = Math::Vec2(ratio.x * (float)getWidth(), ratio.y * (float)getHeight());
    const Math::Vec2 weights = glm::fract(C00Float);

    // The sampling coordinates.
    const Math::Uvec2 C00 = Math::Uvec2(
        Math::clamp((uint32_t)C00Float.x, 0u, widthMinusOne),
        Math::clamp((uint32_t)C00Float.y, 0u, heightMinusOne));

    const Math::Uvec2 C10 = Math::Uvec2(
        Math::clamp(C00.x + 1u, 0u, widthMinusOne),
        Math::clamp(C00.y, 0u, heightMinusOne));

    const Math::Uvec2 C01 = Math::Uvec2(
        Math::clamp(C00.x, 0u, widthMinusOne),
        Math::clamp(C00.y + 1u, 0u, heightMinusOne));

    const Math::Uvec2 C11 = Math::Uvec2(
        Math::clamp(C00.x + 1u, 0u, widthMinusOne),
        Math::clamp(C00.y + 1u, 0u, heightMinusOne));

    // The sampling values.
    const T V00 = getNormalizedPixelFromPosition(C00);
    const T V10 = getNormalizedPixelFromPosition(C10);
    const T V01 = getNormalizedPixelFromPosition(C01);
    const T V11 = getNormalizedPixelFromPosition(C11);

    // Perform the interpolation.
    const auto lerp = [](T t1, T t2, float t3)  { return t1 + (t2 - t1) * t3; };

    const T p0 = lerp(V00, V01, weights.y);
    const T p1 = lerp(V10, V11, weights.y);

    return lerp(p0, p1, weights.x);
}

What am I doing wrong?

I don't think it is related at all but we never know. I am using OpenCV for storing and managing my images. When I use the native resizing it gives same results to point sampling too.

inline void BilinearScaling(const RGB32FImage& src, RGB32FImage& dest)
{
    cv::resize(src.m_image,
        dest.m_image,
        cv::Size(dest.getWidth(), dest.getHeight()),
        0.0,
        0.0,
        cv::InterpolationFlags::INTER_LINEAR);
}

Anyway, this won't help if it works because I need to interpolate manually. My final target is to interpolate a set a 8 spherical harmonics coefficients, not an RGB or RGBA color.


Solution

  • Your getNormalizedPixelFromRatio function correctly implements bilinear interpolation, but using it to sample for each destination pixel during downscaling is the issue.

    You need a downscaling filter that considers the area in the source image corresponding to each destination pixel, rather than just sampling or interpolating at a single point. Implementing a Box filter (averaging the source block) or a more advanced filter will give you the anti-aliased downscaling you expect.
    The same principle will apply when downscaling your spherical harmonics data.