pythonimportinit

How to use __init__.py to create a clean API?

Problem Description:

I am trying to create a local API for my team. I think I understand broadly the mechanics of _init_.py. Let's say we have the below package structure:

API/
├── __init__.py         # Top-level package init file
└── core/               
    ├── __init__.py     # Core module init file
    ├── calculator.py   
    └── exceptions.py

Now if I build my API with empty _init_.py files, and then import API in my script, I won't be able to do something like:

import API
API.core
API.core.calculator

Because the submodules have not been imported specifically. What I need to do is to add into the following into my top-level _init_.py:

from . import core

And the following into my core module _init_.py:

from . import calculator
from . import exceptions

Now, when I do this, all imports I am making in my calculator.py or exceptions.py such as numpy or pandas are actually available through my package as follows:

API.core.calculator.numpy

Question 1: What are the best practices to prevent imported libraries to show through my API ?

On the same theme, let's say I want to access my calculator.py functions directly through the core keyword (let's assume _all_ variables are safely set up). Then I can add the following into my core module _init_.py

from .calculator import *
from .exceptions import *

which then allows me to do:

API.core.my_function()

But then again, I can also call API.core.calculator.my_function() at the same time, which might be confusing for users.

2. How to prevent imported functions to be available through both my package name and module name ?

I tried mixing up the approaches, but with no results, please help !

Solution

  • The typical best practice is to define an __all__ list in each sub-module with names you want the sub-module to export so that the parent module can import just those names with a star import.

    Names that you don't want exposed should be named with a leading underscore by convention so that the linters will warn the users if they try to import a "private" name.

    So in your example case, your API/core/calculator.py will look like:

    import numpy as _numpy

__all__ = ['my_function']

def my_function(): ...

    And then API/core/__init__.py will star-import what's exported by calculator (that is, just my_function):

    from .calculator import *

    which then allows you to do:

    API.core.my_function()

    but not:

    API.core._numpy
API.core.calculator._numpy

    Note that the user can always do import API.core.calculator explicitly to access _numpy in that sub-module, but that is just how the module namespace works as there is nothing really private in Python.

    Also note that a star import will not import names with leading underscores so if you do follow the convention of naming your "private" variables as such you don't even need to define an __all__ list.

    Excerpt from the documentation of More on Modules:

    There is even a variant to import all names that a module defines:

    >>>
from fibo import *
fib(500)
0 1 1 2 3 5 8 13 21 34 55 89 144 233 377

    This imports all names except those beginning with an underscore (_).