I have a GLSL shader that's supposed to output NaNs when a condition is met. I'm having trouble actually making that happen.
Basically I want to do this:
float result = condition ? NaN : whatever;
But GLSL doesn't seem to have a constant for NaN, so that doesn't compile. How do I make a NaN?
I tried making the constant myself:
float NaN = 0.0/0.0; // doesn't work
That works on one of the machines I tested, but not on another. Also it causes warnings when compiling the shader.
Given that the obvious computation didn't work on one of the machines I tried, I get the feeling that doing this correctly is quite tricky and involves knowing a lot of real-world facts about the inconsistencies between various types of GPUs.
Don't use NaNs here.
Section 2.3.4.1 from the OpenGL ES 3.2 Spec states that
The special values Inf and −Inf encode values with magnitudes too large to be represented; the special value NaN encodes “Not A Number” values resulting from undefined arithmetic operations such as 0/0. Implementations are permitted, but not required, to support Inf's and NaN's in their floating-point computations.
So it seems to really depend on implementation. You should be outputing another value instead of NaN