amazon-web-servicesaws-clibazelamazon-linux-2

How to use bazel to download amzn2-core packages


I need to get tar and gzip for an amazon linux 2 based container that will run in an environment without internet access. Using bazel, I create a container from the latest aws-cli image, and I don't want to install it for all of the instances of this container that I create (in kubernetes), I just want to install them if I need them in specific instances. In order to do this, I need to package the rpms for tar and gzip and include the packages in the containers. I know that I can do it by doing a yum install --downloadonly ... and then upload them to s3 and use bazel to get them from the s3 bucket. However, I'd much rather use bazel to get them directly from the repo rather than have to download and upload to s3 anytime I need a new version. The problem is, when I try the repo base url I get access denied. Any help is greatly appreciated. I'll post some code below of what I would like to be able to mostly because I know people here really want to see code snipets.

# WORKSPACE file

...

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")

http_file(
    name = "tar_rpm",
    downloaded_file_path = "tar.rpm",
    sha256 = "some_sha256_value",
    url = "https://some-repo-url/tar.1.26-35.amzn2.0.4.rpm",
)

...

# BUILD.bazel file

...

load("@rules_pkg//:pkg.bzl", "pkg_tar")
load("@io_bazel_rules_docker//container:container.bzl", "container_image")

container_image(
    name = "latest",
    base = "@aws-cli//image",
    entrypoint = "/usr/bin/entrypoint.sh",
    env = {
        "SRC_DIR": "/src",
        "CONFIG_DIR": "/config",
    },
    experimental_tarball_format = "compressed",
    repository = "my.registry/release",
    tars = [
        ":tar_gzip_tgz",
    ],
    visibility = ["//visibility:public"],
)

pkg_tar(
    name = "tar_gzip_tgz",
    srcs = [
        "@tar//file",
        "@gzip//file",
    ],
    package_dir = "/src/tars",
    strip_prefix = ".",
)

...


Solution

  • I did finally find the appropriate urls via https://amazonlinux.pkgs.org/2/amazonlinux-core-x86_64/.

    i.e. The tar url: https://cdn.amazonlinux.com/2/core/2.0/x86_64/6b0225ccc542f3834c95733dcf321ab9f1e77e6ca6817469771a8af7c49efe6c/../../../../../blobstore/4e36e94ae09a0689d56391d39930157084d0999aec4b80360fc87af7e27209d2/tar-1.26-35.amzn2.x86_64.rpm in the download section on https://amazonlinux.pkgs.org/2/amazonlinux-core-x86_64/tar-1.26-35.amzn2.x86_64.rpm.html

    Similarly gzip url: https://cdn.amazonlinux.com/2/core/2.0/x86_64/6b0225ccc542f3834c95733dcf321ab9f1e77e6ca6817469771a8af7c49efe6c/../../../../../blobstore/91de830723306d2bdac85394f3085ea677f1bb82ccf5d83d07b38b5c6ce92d2f/gzip-1.5-10.amzn2.0.1.x86_64.rpm from the gzip download section on https://amazonlinux.pkgs.org/2/amazonlinux-core-x86_64/gzip-1.5-10.amzn2.0.1.x86_64.rpm.html