I am working on a augmented reality application and need to sort the rendering of three different kinds of geometry.
Level 1
Background geometry. This is a set of triangles that represent basic architecture. Walls etc.
Level 2
Photograph plane. A photograph of the architecture placed as a texture on a plane in space. That plane is geometrically in front of Level 1 (as seen from the virtual camera)
Level 3
Dynamic geometry. These are primitives like boxes etc, to augment the scene. They are geometrically behind Level 2, because they are correctly placed inside the coordinate system of Level 1
Desired effect:
a) I want Level_2 to always be drawn in front of Level_1
This is automatically achieved by placing the image plane closer to the camera.
b) I want Level_3 to be occluded correctly against Level_1, thereby hiding parts of a dynamic box, if it is placed partly behind a wall of Level_1. This automatically happens with normal pipeline too. Depth testing etc.
c) I want all Fragments/Pixels of Level_3 to be drawn ABOVE Level_2, even if they are actually behind it. The effect should make the objects appear, as if they are occluded by parts of the photograph.
I know that this is possible, because i have an application in X3Dom (which renders to webgl) that can achieve this effect via the "sortkey" attribute. But i could not recreate the same effect in pure OpenGL. I thought it should be a simple use of Stencilbuffer or Z-Buffer tricks, but i don't get it to work. How do i give an "always pass" to the Level_3 geometry, so that it is drawn above everything else, but also correctly occluded against Level_1? Or how do i make Level_2 hide only Level_1 and not Level_3 even if it is in front of both?
Basically:
1) Level_2 hides Level_1
2) Level_1 and Level_3 occlude each other
3) Visible parts of Level_3 (against Level_1) are not occluded by Level_2
Any help?
Thanks!
Found out after a while, but forgot to update here. The solution was as simple as expected. DepthTesting stays active all the time. Rendering is called in order 1, 2, 3. Rendering of level 2 is enclosed in
gl.glDepthMask(false);
gl.glDepthMask(true);
Allows for correct depth rendering and occlusion of every level but will not change the depth buffer from rendering level 2 elements. Therefore level 3 elements ignore the depth of level 2 and clip against level 1.