Expected build-failure tests in CMake

Sometimes it's good to check that certain things fail to build, e.g.:

// Next line should fail to compile: can't convert const iterator to iterator.
my_new_container_type::iterator it = my_new_container_type::const_iterator();

Is it possible to incorporate these types of things into CMake/CTest? I'm looking for something like this in CMakeLists.txt:

add_build_failure_executable(
    test_iterator_conversion_build_failure
    iterator_conversion_build_failure.cpp)
add_build_failure_test(
    test_iterator_conversion_build_failure
    test_iterator_conversion_build_failure)

(Of course, these specific CMake directives don't exist, to the best of my knowledge.)

Solution

You can do this more or less as you described. You can add a target which will fail to compile, then add a test which invokes cmake --build to try to build the target. All that remains is to set the test property WILL_FAIL to true.

So, say you have your tests in a file named "will_fail.cpp" which contains:

#if defined TEST1
non-compiling code for test 1
#elif defined TEST2
non-compiling code for test 2
#endif

Then you can have something like the following in your CMakeLists.txt:

cmake_minimum_required(VERSION 3.0)
project(Example)

include(CTest)

# Add a couple of failing-to-compile targets
add_executable(will_fail will_fail.cpp)
add_executable(will_fail_again will_fail.cpp)
# Avoid building these targets normally
set_target_properties(will_fail will_fail_again PROPERTIES
                      EXCLUDE_FROM_ALL TRUE
                      EXCLUDE_FROM_DEFAULT_BUILD TRUE)
# Provide a PP definition to target the appropriate part of
# "will_fail.cpp", or provide separate files per test.
target_compile_definitions(will_fail PRIVATE TEST1)
target_compile_definitions(will_fail_again PRIVATE TEST2)

# Add the tests.  These invoke "cmake --build ..." which is a
# cross-platform way of building the given target.
add_test(NAME Test1
         COMMAND ${CMAKE_COMMAND} --build . --target will_fail --config $<CONFIGURATION>
         WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
add_test(NAME Test2
         COMMAND ${CMAKE_COMMAND} --build . --target will_fail_again --config $<CONFIGURATION>
         WORKING_DIRECTORY ${CMAKE_BINARY_DIR})
# Expect these tests to fail (i.e. cmake --build should return
# a non-zero value)
set_tests_properties(Test1 Test2 PROPERTIES WILL_FAIL TRUE)

You can obviously wrap all of this into a function or macro if you have a lot of these to write.