I am attempting to load models exported from Blender into OpenGL. My loader only reads in the vertex coordinates and face indices, ignoring the normal and tex.

It then calculates the normal for each face:

float coord1[3] = {Faces_Triangles[triangle_index], Faces_Triangles[triangle_index+1], Faces_Triangles[triangle_index+2]};
float coord2[3] = {Faces_Triangles[triangle_index+3], Faces_Triangles[triangle_index+4], Faces_Triangles[triangle_index+5]};
float coord3[3] = {Faces_Triangles[triangle_index+6], Faces_Triangles[triangle_index+7], Faces_Triangles[triangle_index+8]};

float *norm = this->calculateNormal(coord1, coord2, coord3);

float* Model_OBJ::calculateNormal(float *coord1, float *coord2, float *coord3)
{
   /* calculate Vector1 and Vector2 */
   float va[3], vb[3], vr[3], val;
   va[0] = coord1[0] - coord2[0];
   va[1] = coord1[1] - coord2[1];
   va[2] = coord1[2] - coord2[2];
 
   vb[0] = coord1[0] - coord3[0];
   vb[1] = coord1[1] - coord3[1];
   vb[2] = coord1[2] - coord3[2];
 
   /* cross product */
   vr[0] = va[1] * vb[2] - vb[1] * va[2];
   vr[1] = vb[0] * va[2] - va[0] * vb[2];
   vr[2] = va[0] * vb[1] - vb[0] * va[1];
 
   /* normalization factor */
   val = sqrt( vr[0]*vr[0] + vr[1]*vr[1] + vr[2]*vr[2] );
 
    float norm[3];
    norm[0] = vr[0]/val;
    norm[1] = vr[1]/val;
    norm[2] = vr[2]/val;
 
    return norm;
}

I have 2 questions.

1. How do I know if the normal is facing inwards or outwards? Is there some ordering of the vertices in each row in the .obj file that gives indication how to calculate the normal?
2. In the initialization function, he uses GL_SMOOTH. Is this incorrect, since I need to provide normals for each vertex if using GL_SMOOTH instead of GL_FLAT?

Question 1

glFrontFace determines wind order

Wind order means, what order a set of vertexes should appear for a normal to be considered positive. Consider the triangle below. It's vertexes are defined clockwise. If we told OpenGL glFrontFace(GL_CW) (That clockwise means front face) then the normal would essentially be sticking right out of the screen towards you in order to be considered "outward".

On a side note, counter-clockwise is the default and what you should stick with.

No matter what, you need should really define normals especially if you want to do any lighting in your scene as they are used for lighting calculation. glFrontFace just lets you tell OpenGL which way you want to interpret what the front of a polygon is.

In the above example, and below diagram, if we told OpenGL that we define faces counter-clockwise and also glEnabled glCullFace and set it to GL_BACK then our triangle wouldn't show up because we would be looking at the back of it and we told OpenGL not to show the back of polygons.

You can read more about face culling here: Face Culling.

Wavefront .obj has support for declaring normals in a file if you don't want to create them yourself. Just make sure your exporter adds them.

Additionaly, Wavefront wants each vertex to have a normal defined:

f v1//vn1 v2//vn2 v3//vn3 ...

Where vN is the vertex of the f face and vnN is the normal of the vertex. By providing a normal for each vertex, you achieve a smoother looking surface than you would by defining a normal per face or by setting all of the normals of vertexes of the same face to be the same. Take a look at this question to see the difference you can make on a sphere: OpenGL: why do I have to set a normal with glNormal?

If your .obj file doesn't have normals defined, I would use the face definition order and cross two edges of the defined face. Consider the method used here: Calculating a Surface Normal

Edit

I think I may be a little confusing. The front face of a polygon is only slightly related to its normals. Normals are only really used for lighting calculations. You don't have to have them, but they are one of the big variables used in calculating how lit your object is.

I am explaining the "front-faced-ness" of a polygon at the same time because it sort of makes sense, when talking about convex polygons, that your normal would stick out of the "front" of your triangle with respect to the shape you are making.

If you created a huge cave, or if your camera were to mostly reside inside of some concave shape, then it would make sense to have your normals point inwards since your light sources are probably going to want to bounce off of the inside of your shape.

Question 2

GL_SMOOTH determines which of the shading models you want to use with glShadeModel

GL_SMOOTH means smooth shading, where color is actually interpolated between each vertex, vs GL_FLAT means flat shading, where only one one color will be used. Typically, you'll use the default value, GL_SMOOTH.

You don't have to define normals for each vertex in either case. However, if you want GL_SMOOTH to look "good" you'll probably want to as it will interpolate as it renders between each vertex rather than just picking one vertex for properties.

---

Also, bear in mind that all of this goes out the window whenever you leave the fixed-function pipeline and start using shaders.