[SOLVED] Bazel - How do I troubleshoot a cmake failure? Foreign Cc - CMake:not all outputs were created or valid

Bazel - How do I troubleshoot a cmake failure? Foreign Cc - CMake:not all outputs were created or valid

Just like the title states. This is a follow-up question to this other one I asked about how to include a cmake project as a py_library dependency.

I added a tinycudann.BUILD.bazel file with the cmake command:

load("@rules_foreign_cc//foreign_cc:defs.bzl", "cmake")

filegroup(
    name = "all_srcs",
    srcs = glob(["**"], exclude = ["bazel-*"]),
    visibility = ["//visibility:public"],
)

cmake(
    name = "built_and_installed_tinycudann",
    install = True,
    lib_source = "@tinycudann_repo//:all_srcs",
    out_static_libs = [
        "libtiny-cuda-nn.a",
    ],
    visibility = ["//visibility:public"],
)

and then referenced this from the BUILD.bazel file in my ./third_party/tinycudann folder:

load("@rules_python//python:defs.bzl", "py_library")
load("@pip_deps_nerfstudio//:requirements.bzl", "requirement")

py_library(
    name = "tinycudann_py_lib",
    #    srcs = ["@tinycudann_repo//:all_srcs"],
    data = ["@tinycudann_repo//:built_and_installed_tinycudann"],
    visibility = ["//visibility:public"],
    deps = [
        requirement("torch"),
    ],
)

and now when I try to bazel build //third_party/tinycudann:tinycudann_py_lib it actually takes the 20 minutes to build the tiny-cuda-nn library.... and then it errors!

ERROR: /home/chuck/.cache/bazel/_bazel_chuck/7ce152f62f7c62dd340375d95f1c9ac8/external/tinycudann_repo/BUILD.bazel:9:6: output 'external/tinycudann_repo/built_and_installed_tinycudann/lib/libtiny-cuda-nn.a' was not created
ERROR: /home/chuck/.cache/bazel/_bazel_chuck/7ce152f62f7c62dd340375d95f1c9ac8/external/tinycudann_repo/BUILD.bazel:9:6: Foreign Cc - CMake: Building built_and_installed_tinycudann failed: not all outputs were created or valid
Target //third_party/tinycudann:tinycudann_py_lib failed to build
INFO: Elapsed time: 1139.767s, Critical Path: 1134.26s
INFO: 3 processes: 2 internal, 1 processwrapper-sandbox.
ERROR: Build did NOT complete successfully

I've tried the troubleshooting steps here to change verbosity, and to check these log files, and I also tried this postfix_script, but I can't get any more information on what's going wrong. Each time I try is another 20 minutes of my day gone.

How can I see where the build files are located? Or is there some other way to access the cmake logs?

When I go to the ./.cache/_bazel_chuck/7ce<...>/external folder I can see a ./tinycudann_repo folder, I can go into that folder and see all of the files that were cloned, I can see where bazel auto-generated my WORKSPACE and BUILD.bazel files in there, but I don't see any ./bin or ./build or any other folders that look like they would be where my files have been built.

I can go to the ./bazel-bin/third_party/tinycudann/built_and_installed_tinycudann folder, and there are subfolders in there for ./bin and ./lib, but they're all empty.

The cmake command is definitely running, it takes the full 20 minutes, and it looks like I've got the name or path wrong on where the output files are located. I got ./libtiny-cuda-nn.a from running cmake on my host machine.

Solution

Use --verbose_failures in combination with --sandbox_debug.

--sandbox_debug leaves the directory sandbox with your build environment around, while --verbose_failures shows you exactly what was run which failed. That will give you the info you need and the files required to dive in and find what's going wrong.