Packing QGIS standalone application with qgis.gui

I have troubles on packing a simple standalone application based on qgis/pyqgis/pyqt5 with the pyinstaller.

Everything works fine if I do not use a gui-module from qgis. When I use a gui-module in the application (e.g. 'from qgis.gui import QgsMapCanvas') I get this error:

  File "zqgis.py", line 12, in <module>
    from core.main_window import MainWindow
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "core\main_window.py", line 5, in <module>
    from qgis.gui import QgsMapCanvas
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "qgis\gui\__init__.py", line 25, in <module>
RuntimeError: qgis._gui cannot import type '����' from PyQt5.QtCore
[1283260] Failed to execute script 'zqgis' due to unhandled exception!

Here is my setting:

I am working with PyCharm and start it with this batch-file:

@echo off 
set OSGEO4W_ROOT=C:\work\_ANWEN~1\OSGeo4W
set PATH=%OSGEO4W_ROOT%\bin;%WINDIR%\system32;%WINDIR%;%WINDIR%\system32\WBem;%OSGEO4W_ROOT%\apps\Python39;%PATH%

call o4w_env.bat

@echo off
path %OSGEO4W_ROOT%\apps\qgis-ltr\bin;%PATH%
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT:\=/%/apps/qgis-ltr
set GDAL_FILENAME_IS_UTF8=YES
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%OSGEO4W_ROOT%\apps\qgis-ltr\qtplugins;%QT_PLUGIN_PATH%
set PYTHONPATH=%OSGEO4W_ROOT%\apps\qgis-ltr\python;%PYTHONPATH%
set PYTHONPATH=%OSGEO4W_ROOT%\apps\Python39;%PYTHONPATH%
set PYTHONPATH=%OSGEO4W_ROOT%\apps\Python39\Lib;%PYTHONPATH%
set PYTHONPATH=%OSGEO4W_ROOT%\apps\Python39\Lib\site-packages;%PYTHONPATH%
set PYTHONPATH=%OSGEO4W_ROOT%\bin;%PYTHONPATH%

set PYTHONPATH=%OSGEO4W_ROOT%\apps\qgis-ltr\python\plugins;%PYTHONPATH%

set PYCHARM="c:\Program Files\JetBrains\PyCharm Community Edition 2022.3.1\bin\pycharm64.exe"
@echo on
start "PyCharm with QGIS knowledge!" /B %PYCHARM% %*

I have a virtual environment where my pyinstaller-package is installed:

I use a spec-file ('zqgis_folder.spec') to bundle the application. Here I have added the dll-files from the 'qgis-ltr/bin' directory. Because of some errors before, I also added some modules to the list 'hiddenimports'. Here is my spec-file:

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None

a = Analysis(
    ['zqgis.py'],
    pathex=[],
    binaries=[('C:\\work\\_anwendungen\\OSGeo4W\\apps\\qgis-ltr\\bin\\*.dll', 'qgis_bin')],
    datas=[],
    hiddenimports=['pkgutil', 'PyQt5.QtPositioning', 'PyQt5.QtPrintSupport',
    'PyQt5.QtSql', 'PyQt5.QtNetwork', 'PyQt5.QtXml'],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(
    pyz,
    a.scripts,
    [],
    exclude_binaries=True,
    name='zqgis',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)
coll = COLLECT(
    exe,
    a.binaries,
    a.zipfiles,
    a.datas,
    strip=False,
    upx=True,
    upx_exclude=[],
    name='zqgis',
)

But on running the application I get this error:

Traceback (most recent call last):
  File "zqgis.py", line 4, in <module>
    from core.main_window import MainWindow
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "core\main_window.py", line 5, in <module>
    from qgis.gui import QgsMapCanvas
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "PyInstaller\loader\pyimod02_importers.py", line 352, in exec_module
  File "qgis\gui\__init__.py", line 25, in <module>
RuntimeError: qgis._gui cannot import type '����' from PyQt5.QtCore
[1272844] Failed to execute script 'zqgis' due to unhandled exception!

In the log-file in my built-directory ('warn-zqgis_folder.txt') I have among others following messages:

missing module named qgis.gui.QgsMapCanvas - imported by qgis.gui (top-level), core.main_window (top-level)
missing module named qgis.core.QgsProject - imported by qgis.core (top-level), core.main_window (top-level)
missing module named qgis.core.QgsApplication - imported by qgis.core (top-level), C:\work\Projekte\ZQGIS\app\zqgis.py (top-level)
missing module named qgis.core.Qgis - imported by qgis.core (top-level), qgis.core.additions.qgssettingsentry (top-level)
missing module named qgis.core.QgsLogger - imported by qgis.core (top-level), qgis.core.additions.qgssettingsentry (top-level)
missing module named qgis.core.QgsSettingsEntryBase - imported by qgis.core (top-level), qgis.core.additions.qgssettingsentry (top-level)
missing module named qgis.core.QgsSettings - imported by qgis.core (top-level), qgis.core.additions.qgssettings (top-level),

My packed application contains:

the files '_core.pyd' and '_gui.pyd' in the directory 'qgis':

the qgis-dll files:

The file '_gui.pyi' from the directory 'OSGeo4W/apps/qgis-ltr/python/qgis' is not in the bundle, I don't know if it is needed:

I have already tried to add this file to the list 'binaries' in the spec-file, the file was then in the bundle but the error was the same.

Additional here my 'zqgis.py' and 'main_window.py':

zqgis.py:

import sys

from qgis.core import QgsApplication
from core.main_window import MainWindow

if sys.version < '3.0':
    sys.exit("This program requires a python3 runtime")


def run():

    QgsApplication.setPrefixPath("C:/work/_anwendungen/OSGeo4W/apps/qgis-ltr", True)
    app = QgsApplication([], True)
    app.initQgis()

    main_window = MainWindow()

    main_window.show()

    sys.exit(app.exec_())


if __name__ == '__main__':

    run()

main_window.py:

from PyQt5.QtWidgets import QMainWindow
from core import main_window_UI
from qgis.core import QgsProject
from qgis.gui import QgsMapCanvas


class MainWindow(QMainWindow, main_window_UI.Ui_MainWindow):

    def __init__(self, parent=None):
        super(MainWindow, self).__init__(parent)
        self.setupUi(self)

How can I bundle a qgis standalone application including gui-modules? Where are the files in the qgis-installation, which must be added to the bundle (using the spec-file)? What is missing on this error-message:

RuntimeError: qgis._gui cannot import type '����' from PyQt5.QtCore

After fighting with this for a few days on my own project, I believe I've come up with a solution. A similar example is building and a window does open without any errors.

The first part is to run PyInstaller from a batch script that also configures environment variables specific to the QT version provided with QGIS. Here is the batch script I am using (build_exe.bat):

@echo off

set PATH=C:\OSGeo4W\apps\qgis-ltr\bin;%PATH%
set O4W_QT_BINARIES=C:\OSGeo4W\apps\Qt5\bin
set O4W_QT_DOC=C:\OSGeo4W\apps\Qt5\doc
set O4W_QT_HEADERS=C:\OSGeo4W\apps\Qt5\include
set O4W_QT_LIBRARIES=C:\OSGeo4W\apps\Qt5\lib
set O4W_QT_PLUGINS=C:\OSGeo4W\apps\Qt5\plugins
set O4W_QT_PREFIX=C:\OSGeo4W\apps\Qt5
set O4W_QT_TRANSLATIONS=C:\OSGeo4W\apps\Qt5\translations
set QT_PLUGIN_PATH=C:\OSGeo4W\apps\qgis-ltr\qtplugins;C:\OSGeo4W\apps\qt5\plugins
set PYTHONPATH=C:\OSGeo4W\bin;C:\OSGeo4W\apps\qgis-ltr\python;C:\OSGeo4W\apps\qgis-ltr\python\plugins;C:\OSGeo4W\apps\Python39\lib\site-packages;

pyinstaller --clean --noconfirm --log-level=DEBUG zqgis_folder.spec

Note: I have debugging enabled, along with a forced clean of the build directory and no confirmation of an overwrite.

The other part of the issue is a missing hidden import in the SPEC file. Update the list of hidden imports under Analysis to include 'PyQt5.Qsci'. Here is the entire spec file, which includes the hidden import change, as well as an import of the project database:

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None

#('C:\\work\\_anwendungen\\OSGeo4W\\apps\\qgis-ltr\\bin\\*.dll', 'qgis_bin')

a = Analysis(
    ['zqgis.py'],
    pathex=[],
    binaries=[],
    datas=[('C:/OSGeo4W/share/proj/*','proj_db')],
    hiddenimports=['pkgutil', 'PyQt5.QtPositioning', 'PyQt5.QtPrintSupport',
    'PyQt5.QtSql', 'PyQt5.QtNetwork', 'PyQt5.QtXml', 'PyQt5.Qsci'],
    hookspath=[],
    hooksconfig={},
    runtime_hooks=[],
    excludes=[],
    win_no_prefer_redirects=False,
    win_private_assemblies=False,
    cipher=block_cipher,
    noarchive=False,
)
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)

exe = EXE(
    pyz,
    a.scripts,
    a.datas,
    [],
    exclude_binaries=True,
    name='zqgis',
    debug=False,
    bootloader_ignore_signals=False,
    strip=False,
    upx=True,
    console=True,
    disable_windowed_traceback=False,
    argv_emulation=False,
    target_arch=None,
    codesign_identity=None,
    entitlements_file=None,
)
coll = COLLECT(
    exe,
    a.binaries,
    a.zipfiles,
    a.datas,
    strip=False,
    upx=True,
    upx_exclude=[],
    name='zqgis',
)

For reference, I've also changed zqgis.py slightly since I didn't have enough of a code example here to work with. Notice that I added the project directory as an environment variable when the application is running as a bundle to ensure that it can be located and suppress any runtime warning for it not being found.

import os
import sys

from qgis.core import QgsApplication
from qgis.gui import QgsMapCanvas
# from core.main_window import MainWindow

if sys.version < '3.0':
    sys.exit("This program requires a python3 runtime")


def run():
    if getattr(sys, "frozen", False) and hasattr(sys, "_MEIPASS"):
        # Set the environment variable for the project database, which is necessary for CRS mappings
        qgis_proj_dir = sys._MEIPASS + "/proj_db"
        os.environ["PROJ_LIB"] = qgis_proj_dir

    # QgsApplication.setPrefixPath("C:/work/_anwendungen/OSGeo4W/apps/qgis-ltr", True)
    app = QgsApplication([], True)
    app.initQgis()

    canvas = QgsMapCanvas()
    canvas.setWindowTitle("Test Window")
    canvas.show()

    sys.exit(app.exec())


if __name__ == '__main__':
    run()

Once these changes were made, and the application was built with the build_exe.bat script, I was able to run the application from .\dist\zqgis\zqgis.exe. The application started and shows the following window: