How it is possible for C compiler validation suites to use any feature of the C standard library if the C standard library is actually under test and may be buggy (which invalidates the validation suite itself)?
For example, the suite may use itself memset
, fopen
, etc., but memset
, fopen
, etc. are actually under test and may be buggy.
Confused. Clarification / experience is needed.
UPD. Clarified after examining suite's source code. The suite uses two compilers:
Yes, testing implementation X using implementation X is not correct (with exceptions, see below). In my particular case the situation is clarified after examining suite's source code. The suite uses two compilers:
Exceptions. As user Steve Summit mentioned:
A good test suite may consist of many stages or layers. First you test A against something fundamental. Then, once you know A is correct, you can use A as part of your test to validate B. Then, you can use A and B to validate C, etc.
This is viable approach. Hover, since the first (upper) layer cannot be tested (for example the correctness of the ==
operator), we take as an axiom that the first layer is error free.
Also continue quoting user Steve Summit:
Sometimes it's even important to test your tests.
I'd say that it is always important to test the correctness of the tests themselves.