I have a Python project which I am building and running with Bazel. The project includes dependencies such as flask, transformers, and numpy. When I attempt to run my application using Bazel, I encounter an ImportError related to NumPy's C-extensions:

Error Message:

INFO: Analyzed target //inference:run_app (1 packages loaded, 4 targets configured).
INFO: Found 1 target...
Target //inference:run_app up-to-date:
bazel-bin/inference/run_app
INFO: Elapsed time: 1.159s, Critical Path: 0.69s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
INFO: Running command line: bazel-bin/inference/run_app
Traceback (most recent call last):
File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/__init__.py", line 24, in <module>
from . import multiarray
File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/multiarray.py", line 10, in <module>
from . import overrides
File "/private/var/tmp/_bazel_modit/96245a706bfbcbe234af177b6d166fc2/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/inference/run_app.runfiles/infaas_deps_numpy/site-packages/numpy/core/overrides.py", line 8, in <module>
from numpy.core._multiarray_umath import (
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'

> ...

ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python interpreter from there.

Steps to Reproduce

1. Clone the repository from GitHub.
2. Run bazel run //inference:run_app.

Observations

• The error suggests that NumPy's C-extensions cannot be found.
• Running the application directly in a virtual environment created from the requirements.txt file in the inference folder works without any issues.

Build and Run Configuration

WORKSPACE File:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")


# Update the SHA and VERSION to the lastest version available here:
# https://github.com/bazelbuild/rules_python/releases.

SHA="f4770504f74b9ef87984224a06df35b931d35a2e695ea898102bd9df30ce4fe6"

VERSION="0.32.0"

http_archive(
    name = "rules_python",
    sha256 = SHA,
    strip_prefix = "rules_python-{}".format(VERSION),
    url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION),
)

load("@rules_python//python:repositories.bzl", "py_repositories")

py_repositories()

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python_3_11",
    # Available versions are listed in @rules_python//python:versions.bzl.
    # We recommend using the same version your team is already standardized on.
    python_version = "3.11",
)

load("@python_3_11//:defs.bzl", interpreter_3_11 = "interpreter")

load("@rules_python//python:pip.bzl", "pip_parse")

# Create a central repo that knows about the dependencies needed from
# requirements_lock.txt.
pip_parse(
   name = "infaas_deps",
   requirements_lock = "//inference:requirements_lock.txt",
   python_interpreter_target = interpreter_3_11,
   download_only = True,
)
# Load the starlark macro, which will define your dependencies.
load("@infaas_deps//:requirements.bzl", "install_deps")
# Call it to define repos for your requirements.
install_deps()

BUILD.bazel

# load("@infaas_deps//:requirements.bzl", "requirement")
load("@infaas_deps//:requirements.bzl", "requirement")


py_library(
    name = "svc",
    deps = [
        requirement("flask"),
        requirement("transformers"),
        requirement("torch"),
    ],
)

py_binary(
    name = "run_app",
    srcs = [
        "app.py",
        ],
    deps = [
        ":svc",
    ],
    main = "app.py",
)

py_binary(
    name = "print_env",
    srcs = ["print_env.py"],
    main = "print_env.py",
    deps = [
        ":svc",
        requirement("setuptools"),  # Ensure setuptools is included for pkg_resources
    ],
)

Additional Information

• Python Version: 3.11
• NumPy Version: 1.26.4
• The error mentions checking the Python and NumPy versions and ensuring they are as expected.

C Libraries and Shared Objects

It appears that the issue may be related to the C libraries and shared objects not being found during runtime. The locations of these libraries and shared objects don't seem to be searched properly.

Question

Why am I encountering this ImportError with NumPy's C-extensions when running the application via Bazel, but not when using a local virtual environment? How can I resolve this issue to successfully run my application using Bazel?

The issue is tied explicitly to enabling bzlmod, which causes workspace toolchains to no longer be registered correctly. This is a known issue tracked in the Bazel rules_python GitHub repository: Issue #1675.

Disabling bzlmod (--noenable_bzlmod) or switching to the bzlmod Python rules resolves the issue.