I'd like to export a 16 bit grayscale PNG of my captured sceneDepth data on iOS (for machine learning purposes). I am using the new iPadPro with lidar sensor to capture the datra in an ARSession. I already get the 192x256 depth map, which i scaled up by 7.5 to match the 1440x1920 resolution of my rgb images. This is the code i have so far:
func convertDepthToImg(frame: ARFrame) {
let depthBuffer = frame.sceneDepth?.depthMap
var ciImageDepth:CIImage = CIImage(cvPixelBuffer: depthBuffer!)
// Transform image on pixel level to get the same size as rgb, apply nearest neighbour sampling or linear sampling (depends on performance in network)
let transformation = CGAffineTransform(scaleX: 7.5, y: 7.5)
ciImageDepth = ciImageDepth.samplingLinear()
.transformed(by: combined_transf_matrix)
let contextDepth:CIContext = CIContext(options: nil)
let cgImageDepth:CGImage = contextDepth.createCGImage((ciImageDepth), from: ciImageDepth.extent)!
// convert to required 16 bits gray png img
convertTo16BitGrayPng(image: cgImageDepth)
}
// Function to create vImageBuffer for more functionality on Images
func createVImg(image: CGImage) -> vImage_Buffer? {
guard let vImageBuffer = try? vImage_Buffer(cgImage: image)
else {
return nil
}
return vImageBuffer
}
func convertTo16BitGrayPng(image: CGImage){
let width = 1440
let height = 1920
//create vImageBuffer vor UIImage
var srcBuf = createVImg(image: image)
print("Height: ", String(srcBuf!.height))
print("Width: ", String(srcBuf!.width))
// allocate memory for final size:
let bv = malloc(width * height * 4)!
var db = vImage_Buffer(data: bv,
height: vImagePixelCount(height),
width: vImagePixelCount(width),
rowBytes: width*2)
// create pointer to Buffer that contains the image data
vImageConvert_PlanarFtoPlanar16F(&(srcBuf)!, &db, vImage_Flags(kvImageNoFlags))
let bp = bv.assumingMemoryBound(to: UInt16.self)
let prov = CGDataProvider(data: CFDataCreateWithBytesNoCopy(kCFAllocatorDefault,
bp,
height * width * 4,
kCFAllocatorDefault))!
let cgImage = CGImage(width: width,
height: height,
bitsPerComponent: 5,
bitsPerPixel: 16,
bytesPerRow: 2 * width,
space: CGColorSpace(name: CGColorSpace.linearSRGB)!,
bitmapInfo: CGBitmapInfo(rawValue: CGImageAlphaInfo.noneSkipFirst.rawValue),
provider: prov,
decode: nil,
shouldInterpolate: false,
intent: .defaultIntent)
// save processed image to documents dir
saveDptToDocs(cgImage: cgImage!, type: "dpt")
}
... save Image to documentpath (works fine)
I used this questions answer to convert my images to 16 bit and save them to my documents directory, but i only get 24 bit images. I really can't get the 16 bit export to work. I already exported images in 32, 64, 8 and even 24 bit. However, 16 bit is somewhat tricky? Please help.
If your output image is 16-bit grayscale, I think the colour space you initialise the CGImage should be a grayscale color space (e.g. CGColorSpaceCreateDeviceGray()) and the bitsPerComponent and bitsPerPixel should both be 16. Also, the bitmapInfo should be along the lines of:
CGBitmapInfo.byteOrder16Little.rawValue |
CGBitmapInfo.floatComponents.rawValue |
CGImageAlphaInfo.none.rawValue