I'm trying to write a toy OBJ loader. It seems faces can be more than triangles, and I was wondering how could object loaders simplify it in the minimum number of triangles. Although square and pentagons seems straightforward, how does it works for more?
While thinking about it, I came up with an example I tried in various engine, and which gave me 3 different results. Here's the file:
v 0.0 0.0 0.0
v 0.75 0.0 0.0
v -0.15 0.8 0
v 0.4 0.1 0
v 0 0.45 0
v 0.1 1.1 0
f 1 2 3 4 5 6
Here's a picture of what it should look like:
On a first site, I get:
On the second, I get two different faces:
and
Is it a limit of this file format? Is there a true answer to this? How should it be handled?
The picture you provided of what it should look like is the correct one. However, displaying it correctly requires some amount of work and many software take some shortcuts, which can result in artifacts.
GPUs do not provide a way to display general polygons, so the software will need to triangulate the polygons. The simplest algorithm to triangulate a polygon is to make a triangle fan (which seems to be the algorithm used in the first example) but it only works correctly for convex polygons, which is not the case here. If the polygon is concave, you will see some artifacts.
Another triangulation algorithm is called "ear clipping" but it can be a bit tedious to implement so I would not be surprised if many of them don't implement it if it's not a 3D modeling software.
The second example you provided is a bit hard to explain how it resulted in two different polygons though. But in any case, the result is incorrect.
So in conclusion, this is not a limit of the file format, but an inherent complexity of dealing with concave polygons and GPUs.