unzippants

Pants BinaryNotFoundError


I have a monorepo managed by pants. The build files and application structure are the same for all services in it. But out of all the services, only one does not start: files_service. When calling pants run the files_service starts successfully, but when calling pants package for files_service-bin the following error occurs:

Engine traceback:
  in `package` goal

BinaryNotFoundError: Cannot find `unzip` on `['/bin', '/opt/homebrew/bin', '/usr/bin', '/usr/local/bin']`. Please ensure that it is installed so that Pants can download the tools Pants needs to run.

In addition, it is the only service that implements proto interfaces as a server. All proto are defined in folder with BUILD file through the target in it:

protobuf_sources(
    name="protos",
    sources=[
        "files/v1/chat.proto",
        "files/v1/backups.proto",
    ],
    grpc=True,
)

Service BUILD file

python_sources(
    name="files_service",
    sources=["**/*.py"],
    dependencies=[
        ":files_service-reqs"
    ]
)

python_requirements(
    name="files_service-reqs"
)

pex_binary(
    name="files_service-bin",
    output_path="files_service/package/binary.pex",
    execution_mode="venv",
    entry_point="files_service.main",
    include_sources=True,
    include_tools = True,
    dependencies=[
        ":files_service"
    ]
)

docker_image(
    name = "sherlock-ai-files_service-image",
    dependencies=[
        ":files_service-bin"
    ]
)


python_test_utils(
    name="files_service-test-utils",
    sources=[
        "**/conftest.py",
    ],
    dependencies=[
        ":files_service"
    ]
)

python_requirements(
    name="files_service-test-reqs",
    source="pytest-requirements.txt",
    resolve="pytest"
)

python_tests(
    name="files_service-tests",
    sources=[
        "tests/**/test_*.py",
        "!**/conftest.py",
    ],
    dependencies=[
        ":files_service-test-utils",
        ":files_service"
    ]
)

pants.toml

[GLOBAL]
pants_version = "2.22.0"
colors = true
backend_packages = [
    "pants.core",
    "pants.backend.python",
    "pants.backend.codegen.protobuf.python",
    "pants.backend.build_files.fmt.ruff",
    "pants.backend.experimental.python.lint.ruff.check",
    "pants.backend.experimental.python.typecheck.pyright",
    "pants.backend.docker"
]
concurrent = true

[source]
root_patterns = [
    "src",
    "libs"
]

[environments-preview.names]
local_linux = "//:local_linux"
local_osx = "//:local_osx"
docker = "//:docker"

[python]
interpreter_constraints = [
    ">=3.11,<3.13"
]
enable_resolves = true

[python.resolves]
python-default = "python-default.lock"
pytest = "pytest-default.lock"

[python-protobuf]
mypy_plugin = true

[test]
output = "all"
report = false
use_coverage = true

[coverage-py]
global_report = true

[pytest]
install_from_resolve = "pytest"
args = ["-vv", "-s", "-W ignore::DeprecationWarning", "--no-header"]

BUILD (in root)

 __defaults__({
   pex_binary: dict(
     environment="local_linux",
     tags=["binary"]
   ),
   docker_image: dict(
     build_platform=["linux/amd64"],
     tags=["docker"]
   ),
 })
 
 local_environment(
   name="local_linux",
   compatible_platforms=["linux_x86_64"],
   fallback_environment="docker"
 )
 
 local_environment(
   name="local_osx",
   compatible_platforms=["macos_arm64"],
 )
 
 docker_environment(
   name="docker",
   platform="linux_x86_64",
   image="python:3.12-slim"
 )

System info:

~ % sw_vers
ProductName:        macOS
ProductVersion:     14.0
BuildVersion:       23A344

There is unzip in the environment. The path to it:

~ % which unzip
/usr/bin/unzip

Solution

  • The problem was solved when building under certain conditions (in Linux). To do this, I defined the working and target environments in the root build, then used them when running.

    local_environment(
        name="linux",
        compatible_platforms=["linux_x86_64"],
        fallback_environment="docker"
    )
    
    docker_environment(
      name="docker",
      platform="linux_x86_64",
      image="python:3.12",
    )