objective-cswiftaugmented-realityarkitavdepthdata

What are black pixels in ARKit Depth Map pixel buffer?

When taking a depth pixel buffer from ARKit session like:

CVPixelBufferRef depthDataMap = frame.capturedDepthData.depthDataMap;

many pixels are in black. What do they represent? Why are they black? This doesn't reflect the real depth of the real world. How to interpret this?

enter image description here

Solution

  • It's some sort of "trail" artifact.

    Those black pixels in a Depth channel are digital artifacts. The appearance of this black outline became possible due to the fact that the depth data is captured at a frequency of 15 fps (at every fourth frame), in contrast to the RGB image which is captured at a frequency of 60 fps. Also, a Depth channel image is of a low quality (low-res). Thus, if the subject quickly or abruptly moves its head at the time of AR session running, when every three out of four depth frames contain a completely black background – this leads to a situation when RGBA and Depth data don't match each other, so we get these nasty digital artifacts.

    Look at a picture in a post How to improve People Occlusion in ARKit 3.0 (click on a link below).

    If you need an additional info about a depth map and its trailing effect, please read this post.