Short version: How can a crash (bad memory access exception/nullptr exception) inside glDrawElements/glDrawElementsInstanced
be debugged?
Long version:
You have a path of OpenGL rendering code which uses VAOs
, and commits rendering thru calling glDrawElements
or glDrawElementsInstanced
. That codepath works correctly most of the time. We are talking "editor code", which means: the data may be any geometry and is highly likely to change frequently.
But sometimes after commiting reproducible data changes it simply crashes within the glDrawElements*
driver code (i.e. glDrawElements
is called, the function parameters are OK, the crash happens inside glDrawElements
).
How could you proceed debugging this problem?
P.S.:
At first it should be clear what may cause crashes within the driver. In most cases that's bad memory access.
What may cause access of bad memory inside the driver?
GL_ELEMENT_ARRAY_BUFFER
-binding changed somehow (thus the parameters given to glDrawElements
may cause access beyond the memory of that object)GL_ELEMENT_ARRAY_BUFFER
's content has changed, possibly referencing uninitialized/non-existent vertex data (VBO access out of bounds)GL_ARRAY_BUFFER
objects' data was altered, thus not containing referenced vertex data anymore (VBO access out of bounds)glDrawElements*
mostly mean that any of the objects bound within the VAO state was accessed out of bounds.Without extra debugging code these access violations are very hard to catch. I suggest inserting debug-output right before calling glDrawElements*
. The Debug output should query all the bindings and info available, so you can compare the settings "when it works" with "when it crashes" and figure out what to look for, next.
My debug function looks like this:
void debugVAOState(std::string baseMessage)
{
baseMessage.append( " ... querying VAO state:\n" );
int vab, eabb, eabbs, mva, isOn( 1 ), vaabb;
glGetIntegerv( GL_VERTEX_ARRAY_BINDING, &vab );
glGetIntegerv( GL_ELEMENT_ARRAY_BUFFER_BINDING, &eabb );
glGetBufferParameteriv( GL_ELEMENT_ARRAY_BUFFER, GL_BUFFER_SIZE, &eabbs );
baseMessage.append( " VAO: " + std::to_string( vab ) + "\n" );
baseMessage.append( " IBO: " + std::to_string( eabb ) + ", size=" + std::to_string( eabbs ) + "\n" );
glGetIntegerv( GL_MAX_VERTEX_ATTRIBS, &mva );
for ( unsigned i = 0; i < mva; ++i )
{
glGetVertexAttribiv( i, GL_VERTEX_ATTRIB_ARRAY_ENABLED, &isOn );
if ( isOn )
{
glGetVertexAttribiv( i, GL_VERTEX_ATTRIB_ARRAY_BUFFER_BINDING, &vaabb );
baseMessage.append( " attrib #" + std::to_string( i ) + ": VBO=" + std::to_string( vaabb ) + "\n" );
}
}
OutputDebugString( baseMessage.c_str() );
}
It's still simple and outputs only the most valuable information to be able to see if above mentioned bindings have changed somehow. But this helped me finding numerous crashes that came from aggressive OpenGL optimization.