bazeldhall

How can I access the output of a bazel rule from another rule without using a relative path?


I am attempting to use Bazel to compile a dhall program based on dhall-kubernetes to generate a Kubernetes YAML file.

The basic dhall compile without dhall-kubernetes using a simple bazel macro works ok.

I have made an example using dhall's dependency resolution to download dhall-kubernetes - see here. This also works but is very slow (I think because dhall downloads each remote file separately), and introduces a network dependency to the bazel rule execution, which I would prefer to avoid.

My preferred approach is to use Bazel to download an archive release version of dhall-kubernetes, then have the rule access it locally (see here). My solution requires a relative path in Prelude.dhall and package.dhall for the examples/k8s package to reference dhall-kubernetes. While it works, I am concerned that this is subverting the Bazel sandbox by requiring special knowledge of the folder structure used internally by Bazel. Is there a better way?

Prelude.dhall:

../../external/dhall-kubernetes/1.17/Prelude.dhall 

WORKSPACE:

workspace(name = "dhall")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

DHALL_KUBERNETES_VERSION = "4.0.0"

http_archive(
    name = "dhall-kubernetes",
    sha256 = "0bc2b5d2735ca60ae26d388640a4790bd945abf326da52f7f28a66159e56220d",
    url = "https://github.com/dhall-lang/dhall-kubernetes/archive/v%s.zip" % DHALL_KUBERNETES_VERSION,
    strip_prefix = "dhall-kubernetes-4.0.0",
    build_file = "@//:BUILD.dhall-kubernetes",
)

BUILD.dhall-kubernetes:

package(default_visibility=['//visibility:public'])

filegroup(
    name = "dhall-k8s-1.17",
    srcs = glob([
        "1.17/**/*",
    ]),
)

examples/k8s/BUILD:

package(default_visibility = ["//visibility:public"])

genrule(
    name = "special_ingress",
    srcs = ["ingress.dhall",
            "Prelude.dhall",
            "package.dhall",
        "@dhall-kubernetes//:dhall-k8s-1.17"
    ],
    outs = ["ingress.yaml"],
    cmd = "dhall-to-yaml --file $(location ingress.dhall) > $@",
    visibility = [
        "//visibility:public"
    ]
)

Solution

  • There is a way to instrument dhall to do "offline" builds, meaning that the package manager fetches all Dhall dependencies instead of Dhall fetching them.

    In fact, I implemented something exactly this for Nixpkgs, which you may be able to translate to Bazel:

    High-level explanation

    The basic trick is to take advantage of a feature of Dhall's import system, which is that if a package protected by a semantic integrity check (i.e. a "semantic hash") is cached then Dhall will use the cache instead of fetching the package. You can build upon this trick to have the package manager bypass Dhall's remote imports by injecting dependencies in this way.

    You can find the Nix-related logic for this here:

    ... but I will try to explain how it works in a package-manager-independent way.

    Package structure

    First, the final product of a Dhall "package" built using Nix is a directory with the following structure:

    $ nix-build --attr 'dhallPackages.Prelude'         
    …
    
    $ tree -a ./result
    ./result
    ├── .cache
    │   └── dhall
    │       └── 122026b0ef498663d269e4dc6a82b0ee289ec565d683ef4c00d0ebdd25333a5a3c98
    └── binary.dhall
    
    2 directories, 2 files
    

    The contents of this directory are:

    User interface

    The function for building a package takes four arguments:

    Implementation

    The way that the Dhall package builder works is:

    Packaging conventions

    Once you have this function, there are a couple of conventions that can help simplify doing things "in the large"

    Conclusion

    I hope that helps! If you have more questions about how to do this you can either ask them here or you can also discuss more on our Discourse forum, especially on the thread where this idiom first originated: