I'm new to Dagster and I'm struggling importing my module into the Dagster code. This is my project structure.
.
├── pyproject.toml
├── README.md
├── setup.cfg
├── setup.py
├── my-project
│ ├── assets.py
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── assets.cpython-311.pyc
│ │ └── __init__.cpython-311.pyc
│ ├── scaled_dataset.csv
│ └── utilities
│ ├── dataset.py
│ ├── __init__.py
│ ├── nn.py
│ ├── plotting.py
│ ├── testing.py
│ └── training.py
In my __init__.py of my-poject I have set
from dagster import Definitions, load_assets_from_modules
from . import assets, utilities
all_assets = load_assets_from_modules([assets, utilities])
defs = Definitions(
assets=all_assets,
)
And in my code in assets.py I have (for example):
from utilities.plotting import plot_dataset
However, when I run `dagster dev', it returns:
ModuleNotFoundError: No module named 'utilities'
Stack Trace:
File "/home/user/PycharmProjects/Dagster/venv/lib/python3.11/site-packages/dagster/_core/code_pointer.py", line 135, in load_python_module
return importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/home/user/PycharmProjects/Dagster/my-project/my-project/__init__.py", line 3, in <module>
from . import assets, utilities
File "/home/user/PycharmProjects/Dagster/my-project/my-project/assets.py", line 12, in <module>
from utilities.plotting import plot_dataset
What am I missing? Thanks
I would try this:
First, rename my-project to my_project.
Next, install your project as editable from the root folder:
pip install -e .
And then update your imports so they look like this:
from my_project.utilities.plotting import plot_dataset