I'm trying to write a function for C++ that can examine a union of two polygons and eliminate any leftover faces that didn't get correctly culled because of floating point error or various other small-number glitchy reasons.
Is there a good method by which one can identify a 3D triangle that is completely coplanar and contained within other faces on a 3D polygon and thus is completely redundant?
It's easy enough to detect a triangle within another TRIANGLE, but I'm looking for a situation where I can detect a redundant triangle that might cross and be contained several triangles that form a flat plane.
For example, in the picture above, assuming all vertices are in one plane, the red triangle is completely unnecessary and will only result in z-fighting. And yet I can't seem to come up with a good technique that covers all bases for saying "the red one can be removed."
Any suggestions?
Posting the solution I came up with here in case this question ever can help anyone with a similar problem. What I did was: