Was trying to get Stencil to work in my app. I use Sprites to render content on to the Device. Content could be movies, pictures or text.
Can I set the stencil buffer using these Sprites, which can be used in later passes to stencil out other Sprites being rendered?
I'm even not able to stencil out any of the Sprites by setting StencilFunction = Compare.Never! Anything wrong that I might be doing?
I've already spent almost an entire day checking out posts related to Stencils in MDX, but just couldn't get it to work.
Bit more searching got me to a technique wherein we draw silhouette of the shapes to be added to the stencil buffer. Is this really required in my case? I just want to use the Sprites, and add them to stencil buffer to stencil out other Sprites.
EDIT: OK.. So I figured out how to get it done. Referred to this post - http://www.gamedev.net/community/forums/viewreply.asp?ID=1999276
But now the sprite is ignoring alpha values and rendering sprites opaque. Similar to the effect we get when alpha blending is set to false. Any ideas?
So, I finally figured it out. It's all related to Sprites and its Begin() method.
Sprite.Begin() will modify the RenderState and disable stencil. We can prevent it by setting SpriteFlags.DoNotModifyRenderState flag, but then the alpha blending is not supported when sprites are drawn.
What I did now was simply call Sprite.Begin() without SpriteFlags.DoNotModifyRenderState flag set. And before the call to Sprite.Draw(), enabled stencil on the device.