pythonpython-3.xsetuptoolspython-packagingpyproject.toml

Unable to import function from custom package built using pyproject.toml

This is my project structure

pkg_root
└───venv
└───requirements.txt
└───pyproject.toml
├───src
│   ├───pkg
│   │   ├───__init__.py
│   │   ├───test_main.py
│   │   ├───config
│   │   │   ├───json
│   │   ├───data
│   │   │   ├───input
│   │   │   │   └───data_file
│   │   │   └───output
│   │   │       ├───data_file
│   │   ├───sub_pkg_1
│   │   │   ├───__init__.py
│   │   │   ├───sub_pkg_1_1
│   │   │   │   ├───__init__.py
│   │   │   │   ├───sub_pkg_1_1_a
│   │   │   │   │   └─── __init__.py
│   │   │   │   │   └─── file.py
│   │   │   ├───sub_pkg_1_2
│   │   │   │   └─── __init__.py
│   │   │   │   └─── file.py
│   │   │   ├───sub_pkg_1_3
│   │   │   │   ├───__init__.py
│   │   │   │   ├───sub_pkg_1_3_a
│   │   │   │   │   ├───__init__.py
│   │   │   │   │   ├───file.py

And this is my pyproject.toml

[build-system]
requires = ["setuptools>=67.0.0", "wheel"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
package-dir = {"" = "src"}

[tool.setuptools.packages.find]
where = ["src"]

[project]
name = "pkg"
version = "1.0.0"
dynamic = ["dependencies"]

[tool.setuptools.dynamic]
dependencies = {file = ["requirements.txt"]}

Running pip install . installs the package in my global Python interpreter in spite of being inside a virtual environment.

I opened an idle shell and imported my package and wrote the following code and running it gave me an attribute error.

>> import pkg
>> pkg.test_main.generate_data()
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    pkg.test_main.generate_data()
AttributeError: module 'pkg' has no attribute 'test_main'

Running the command print(pkg) gives the following message

<module 'pkg' from 'path:\\to\\site-packages\\pkg\\__init__.py'>

The desired behaviour is to import subpackages/modules from pkg, but from what I can understand, pkg is being treated as an object. Where am I going wrong?

Solution

  • The namespace for the pkg module only holds what is in the __init__.py file. I.e. if you add a variable or import a module in __init__.py it will be added to the pkg namespace and be available using the dot notation: pkg.<something>.

    In this case, you can either import test_main in your __init__.py file:

    # __init__.py
from . import test_main

    Which now adds test_main to the pkg namespace so you can do:

    >> import pkg
>> pkg.test_main.generate_data()

    Or you can import test_main directly in your code:

    >> import pkg.test_main
>> pkg.test_main.generate_data()