pythonpython-importimporterrorlibraries

Can't import all functions from custom library


I'm trying to make a custom python library that can be shared via github for the first time. I'm able to successfully install it via pip install git+repo_url, but when I try to input it into a python script, only one of the files in the library is recognized as a module. For reference this is the structure of my repo:

mylib/
    utils/
        utils.py
    __init__.py
    plot.py
    colors.py
    codes.py
    fonts.py

For some reason I can only import stuff from colors.py, but none of the other files.

When I do from mylib import colors I can access everything from colors.py, but for example from mylib import codes gives the error:

ImportError: cannot import name 'codes' from 'mylib'

I've seen different suggestions about what to put in __init__.py so I've tried it both empty and with import statements for all the files and neither way worked.

UPDATE

After changing my __init__.py to contain from .module import * for all of my modules I'm now getting the error ModuleNotFoundError: No module named 'mylib.utils' from the from .plot import * in __init__.py. Currently the plot modules imports a function from utils with the from mylib.utils.utils import function. What is the correct way to do this import?


Solution

  • You need the mylib folder itself to be a Python package. Otherwise you will be forever depending on running all your things from a working directory directly above mylib - and then get some luck with its interpretation of a mylib.__init__.py, any.

    Fortunatelly, the default use case for creating a package is pretty straightforward, but very hard to find in the docs: an empty pyproject.toml file in the directory immediatelly above mylib can do it:

    
    mylib/
        pyproject.toml  (0 bytes) (this is the git repo-root!)
        mylib/
            utils/
                utils.py
            __init__.py (0 bytes or containing "from . import plot", (one line for each script) for auto-imports)
            plot.py
            colors.py
            codes.py
            fonts.py
    
    

    After doing a pip install like in the OP, this will install mylib itself as a package, and you will be able to do things like import mylib.codes.

    This works due to pyproject.toml defaulting the project version to 0.0.0 and the build tool to be setuptools in the absense of any declarations. If you ever want to customize anything, search around for minimal contents for the pyproject.toml file. (In particular, it is sensible to auto-finding the code if there are any other folders in the project. You may need a [tool.setuptools] session with a packages = ["mylib"] line inside the file.

    (also, renaming the utils/utils.py file to utils/__init__.py will make its contents available as mylib.utils.myutilfunc instead of mylib.utils.utils.myutilfunc)