Appropriate Usage of MPSImageGaussianPyramid with Metal

I would like to use MPSImageGaussianPyramid but am very new to Metal's usage and with mipmaps. I would like to use the filter to produce an image pyramid for image processing techniques.

From what I'm able to gather, MPSImageGaussianPyramid creates a mipmapped image, however in my code I'm having a hard time even making sure that I'm seeing the output correctly. Are there any examples where this filter has been used correctly? My questions are:

1. How does one access the mipmapped images after the filter has been applied?

2. Is it possible to copy the mipmapped images to another image for processing?

3. Is this mipmapped image going to be faster than manually creating a pyramid through custom filters?

Thanks, and I will provide some sample code later that I've not been able to get working.

A few pieces of advice for working with MPS kernels in general, and the image pyramid filters in particular:

• If you're going to be using a kernel more than once, cache it and reuse it rather than creating a kernel every time you need to encode.
• Consider setting the edgeMode property of the kernel to .clamp when downsampling, since sampling out-of-bounds (as the Gaussian pyramid does on the first step) will return black by default and introduce artificially dark pixels.
• When encoding a Gaussian pyramid kernel, always use the "in-place" method, without providing a fallback allocator:

kernel.encode(commandBuffer: commandBuffer, inPlaceTexture: &myTexture)

As you note, running an image pyramid kernel puts the result in the available mip levels of the texture being downsampled. This means that the texture you provide should already have as many mip levels allocated as you want filled. Thus, you should ensure that the descriptor you use to create your texture has an appropriate mipmapLevelCount (this is ensured by the texture2DDescriptorWithPixelFormat convenience method, and can be controlled indirectly by using the .allocateMipmaps option with MTKTextureLoader).

Assuming that you now know how to encode a kernel and get the desired results into your texture, here are some answers to your questions:

1. How does one access the mipmapped images after the filter has been applied?

You can implicitly use the mipmaps in a shader when rendering by using a sampler that has has a mip filter, or you can explicitly sample from a particular mip level by passing a lod_option parameter of type level to the sample function:

constexpr sampler mySampler(coord::normalized, filter::linear, mip_filter::linear);
float4 color = myTexture.sample(mySampler, texCoords, level(selectedLod))

This works in compute kernels as well as rendering functions. Use a mip filter of nearest or round the selected LOD if you want to sample from a single mip level rather than using trilinear mip filtering.

2. Is it possible to copy the mipmapped images to another image for processing?

Since a texture that is downsampled by an image pyramid kernel must already have the .pixelFormatView usage flag, you can create a texture view on a mipped texture that selects one or more mip levels. For example, if you want to select the first and higher mip levels (dropping the base level), you could do that like this:

let textureView = myTexture.makeTextureView(pixelFormat: myTexture.pixelFormat,
    textureType: myTexture.textureType,
    levels: Range<Int>(uncheckedBounds: (1, myTexture.mipmapLevelCount)),
    slices: Range<Int>(uncheckedBounds: (0, 1)))

You can also use a blit command encoder to copy from one texture to another, specifying which mip levels to include. This allows you to free the original texture if you want to reclaim the memory used by the lower mip levels.

You can wrap a MTLTexture with an MPSImage if you want to use APIs that work with images rather than textures:

let image = MPSImage(texture: myTexture, featureChannels: 4)

3. Is this mipmapped image going to be faster than manually creating a pyramid through custom filters?

Almost certainly. Metal Performance Shaders are tuned for each generation of devices and have numerous heuristics that optimize execution speed and energy use.