command-linecmakevalgrindctestmemcheck

How to pass arguments to memcheck with ctest?

I want to use ctest from the command line to run my tests with memcheck and pass in arguments for the memcheck command.

I can run ctest -R my_test to run my test, and I can even run ctest -R my_test -T memcheck to run it through memcheck.

But I can't seem to find a way to pass arguments to that memcheck command, like --leak-check=full or --suppressions=/path/to/file.

After reading ctest's documentation I've tried using the -D option with CTEST_MEMCHECK_COMMAND_OPTIONS and MEMCHECK_COMMAND_OPTIONS. I also tried setting these as environment variables. None of my attempts produced any different test command. It's always:

Memory check command: /path/to/valgrind "--log-file=/path/to/build/Testing/Temporary/MemoryChecker.7.log" "-q" "--tool=memcheck" "--leak-check=yes" "--show-reachable=yes" "--num-callers=50"

How can I control the memcheck command from the ctest command line?

Solution

  • TL;DR

    ctest --overwrite MemoryCheckCommandOptions="--leak-check=full --error-exitcode=100" \
      --overwrite MemoryCheckSuppressionFile=/path/to/valgrind.suppressions \
      -T memcheck

    Explanation

    I finally found the right way to override such variables, but unfortunately it's not easy to understand this from the documentation. So, to help the next poor soul that needs to deal with this, here is my understanding of the various ways to set options for memcheck.

    In a CTestConfig.cmake in you top-level source dir, or in a CMakeLists.txt (before calling include(CTest)), you can set MEMORYCHECK_COMMAND_OPTIONS or MEMORYCHECK_SUPPRESSIONS_FILE. When you include(CTest), CMake will generate a DartConfiguration.tcl in your build directory and setting the aforementioned variables will populate MemoryCheckCommandOptions and MemoryCheckSuppressionFile respectively in this file. This is the file that ctest parses in your build directory to populate its internal variables for running the memcheck step. So, if you'd like to set you project's options for memcheck during cmake configuration, this is the way to got.

    If instead you'd like to modify these options after you already have a properly configured build directory, you can:

    1. Modify the DartConfiguration.tcl directly, but note that this will be overwritten if cmake runs again, since this file is regenerated each time cmake runs.
    2. Use the ctest --overwrite command-line option to set these memcheck options just for that run.

    Notes

    1. I've seen mentions online of a CMAKE_MEMORYCHECK_COMMAND_OPTIONS variable. I have no idea what this variable is and I don't think cmake is aware of it in any way.
    2. Setting CTEST_MEMORYCHECK_COMMAND_OPTIONS (the variable that is actually documented in the cmake docs) in your CTestConfig.cmake or CMakeLists.txt has no effect. It seems this variable only works in "CTest Client Scripts", which I have never used.
    3. Unfortunately, both MEMORYCHECK_COMMAND_OPTIONS and MEMORYCHECK_SUPPRESSIONS_FILE aren't documented explicitly in cmake, only indirectly, in ctest documentation and the Testing With CTest tutorial.

    When ctest is run in the build, it parses the file to populate its internal variables: https://cmake.org/cmake/help/latest/manual/ctest.1.html#dashboard-client-via-ctest-command-line It's not clear to me how this interacts with