rustcode-coverage

How to collect code coverage for the rustc compiler itself?


I am trying to collect code coverage information for the rustc compiler itself to better understand which parts of the code are executed during the compilation process. My goal is to ensure comprehensive testing coverage of critical sections of the compiler’s codebase. However, due to the complexity of the build process, I am unsure how to achieve this.

Any advice or suggestions on how to proceed would be appreciated!

So far, I have tried setting RUSTFLAGS="-C instrument-coverage", but this leads to errors during the compilation of rustc itself.

Do I need to modify any build scripts or configurations specifically for rustc to enable coverage collection? Are there any recommended tools or processes that work specifically for gathering coverage information from the Rust compiler?

Additional Info:

Supplementary Information:

In the config.toml file, I made the following changes:

assertions = true
profiler = true
prefix = "/path/to/install"
sysconfdir = "etc"
llvm-config = "/path/to/self-compile-llvm-config"
llvm-filecheck = "/path/to/self-compile-FileCheck"

Additionally, I set the environment variable:

export RUSTFLAGS="-C instrument-coverage"

Then, I ran the build command:

./x build

However, I encountered the following error messages:

Compiling cc v1.0.99
Compiling std v0.0.0 (/path/to/rust/library/std)
warning: -C instrument-coverage requires symbol mangling version `v0`, but `-C symbol-mangling-version=legacy` was specified

Compiling profiler_builtins v0.0.0 (/path/to/rust/library/profiler_builtins)
error: `profiler_builtins` crate (required by compiler options) is not compatible with crate attribute `#![no_core]`

error[E0463]: can't find crate for `profiler_builtins`
  |
  = note: the compiler may have been built without the profiler runtime

For more information about this error, try `rustc --explain E0463`.

warning: `core` (lib) generated 1 warning
error: could not compile `core` (lib) due to 2 previous errors; 1 warning emitted
warning: build failed, waiting for other jobs to finish...

warning: profiler_builtins@0.0.0: In file included from /path/to/rust/src/llvm-project/compiler-rt/lib/profile/InstrProfiling.h:12,
warning: profiler_builtins@0.0.0:                  from /path/to/rust/src/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:34:
warning: profiler_builtins@0.0.0: /path/to/rust/src/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c: In function ‘doProfileMerging’:
warning: profiler_builtins@0.0.0: /path/to/rust/src/llvm-project/compiler-rt/lib/profile/InstrProfilingPort.h:33:37: warning: ignoring return value of ‘ftruncate’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
warning: profiler_builtins@0.0.0:    33 | #define COMPILER_RT_FTRUNCATE(f, l) ftruncate(fileno(f), l)
warning: profiler_builtins@0.0.0:       |                                     ^~~~~~~~~~~~~~~~~~~~~~~
warning: profiler_builtins@0.0.0: /path/to/rust/src/llvm-project/compiler-rt/lib/profile/InstrProfilingFile.c:445:9: note: in expansion of macro ‘COMPILER_RT_FTRUNCATE’
warning: profiler_builtins@0.0.0:   445 |   (void)COMPILER_RT_FTRUNCATE(ProfileFile,
warning: profiler_builtins@0.0.0:       |         ^~~~~~~~~~~~~~~~~~~~~
Build completed unsuccessfully in 0:00:31

Solution

  • After several attempts, I found the following solution.

    1. Clone the Repository
      Clone the Rust repository:

      git clone --depth 1 https://github.com/rust-lang/rust rust-nightly
      
    2. Edit Configuration
      Copy config.example.toml to config.toml and modify it with the following options (the last two are optional):

      • profiler = true
      • assertions = true
      • debug-assertions = true
    3. Modify RUSTFLAGS in Bootstrap
      Inspired by RustSmith, adjust the RUSTFLAGS parameter in the bootstrap configuration. Since the relevant parameter location may vary, use grep to find it:

      cd /path/to/rust-nightly/src/bootstrap
      grep -Irnsw "rust_new_symbol_mangling"
      

      For instance, in commit 788202a, add the following before line 556 in src/bootstrap/src/core/builder/cargo.rs (apply coverage flags to all modules except std):

      if mode != Mode::Std {
          rustflags.arg("-Cinstrument-coverage");
      }
      
    4. Compile and Install rustc

      ./x build && ./x install
      
    5. Install grcov
      Install grcov (you may also use the binaries available in the release):

      cargo install grcov
      
    6. Run rustc to Generate Coverage Files

      LLVM_PROFILE_FILE="coverage/%p-%m.profraw" /path/to/rustinstall/bin/rustc ...
      
    7. Generate Coverage Information with grcov
      Use grcov to generate coverage data. Specify -s as the source path (pointing to the compiler directory within the Rust source), -b as the binary path (pointing to the bin directory in the installation), --llvm-path for the LLVM suite path (downloaded during Rust compilation), and -t to set the output format (e.g., html or lcov), along with -o for the output file:

      grcov coverage/*.profraw -s /path/to/rust-nightly/compiler -b /path/to/rustinstall --llvm-path /path/to/rust-nightly/build/x86_64-unknown-linux-gnu/ci-llvm/bin -t lcov -o cov.info
      
    8. View Summary with lcov
      Generate a summary with lcov. Note that branch coverage may be missing despite .lcovrc modifications, likely due to an issue with Rust itself:

      lcov --summary cov.info
      

    However, the summary information exported by lcov does not include branch coverage (despite the correct configuration of .lcovrc).