I'm trying to create a script that converts a csv file to a feather file, then compile that script to a stand-alone exe which can be invoked from other applications supplying two parameters - path to a csv file and a path to the output feather file. Environment is Windows 10, Python 311. I've created, and successfully run, a script that does what's required when invoked from the command line like:
python arraowreadfeather.py C:\temp\Feather_test\data.csv C:\temp\Feather_test\data.feather
I've compiled the script to an exe using three tools freely available - py2exe, pyinstaller and auto-py-to-exe.
However, when I compile this script to an exe - either stand-alone or in a single folder - then run it from the command line I get this error:
C:\temp\csvtofeather\dist>arrowreadfeather.exe C:\temp\Feather_test\data.feather Calling from_feather with: C:\temp\Feather_test\data.feather Traceback (most recent call last): File "arrowreadfeather.py", line 28, in <module> File "arrowreadfeather.py", line 22, in main File "arrowreadfeather.py", line 7, in from_feather File "pyarrow\feather.pyc", line 226, in read_feather File "pyarrow\array.pxi", line 830, in pyarrow.lib._PandasConvertible.to_pandas File "pyarrow\table.pxi", line 3990, in pyarrow.lib.Table._to_pandas File "pyarrow\pandas_compat.pyc", line 810, in table_to_blockmanager File "pyarrow\pandas_compat.pyc", line 968, in _reconstruct_index File "pyarrow\pandas-shim.pxi", line 126, in pyarrow.lib._PandasAPIShim.pd.get File "pyarrow\pandas-shim.pxi", line 100, in pyarrow.lib._PandasAPIShim._check_import File "pyarrow\pandas-shim.pxi", line 56, in pyarrow.lib._PandasAPIShim._import_pandas ModuleNotFoundError: No module named 'pyarrow.vendored.version'
I simplified the script just to read an existing feather file and that gives the same error. Here is the simplified script and the error output:
`import sys
import pyarrow.feather as ft
def from_feather(_file: str):
table = ft.read_feather(_file)
return table
def main():
if len(sys.argv) < 2:
print("Insufficient args for conversion. Specify a source and dest.")
return None
print("Calling from_feather with: ",sys.argv[1])
feather_file = sys.argv[1]
d_table = from_feather(feather_file)
if __name__ == "__main__":
main()`
C:\Python311\output>arrowreadfeather.exe C:\temp\Feather_test\data.feather Calling from_feather with: C:\temp\Feather_test\data.feather Traceback (most recent call last): File "arrowreadfeather.py", line 23, in File "arrowreadfeather.py", line 19, in main File "arrowreadfeather.py", line 5, in from_feather File "pyarrow\feather.py", line 226, in read_feather File "pyarrow\array.pxi", line 830, in pyarrow.lib._PandasConvertible.to_pandas File "pyarrow\table.pxi", line 3990, in pyarrow.lib.Table._to_pandas File "pyarrow\pandas_compat.py", line 810, in table_to_blockmanager File "pyarrow\pandas_compat.py", line 968, in _reconstruct_index File "pyarrow\pandas-shim.pxi", line 126, in pyarrow.lib._PandasAPIShim.pd.get File "pyarrow\pandas-shim.pxi", line 100, in pyarrow.lib._PandasAPIShim._check_import File "pyarrow\pandas-shim.pxi", line 56, in pyarrow.lib._PandasAPIShim._import_pandas ModuleNotFoundError: No module named 'pyarrow.vendored.version' [20312] Failed to execute script 'arrowreadfeather' due to unhandled exception!
Thanks in advance for any help/pointers.
The information above did solve my problem. Steps taken.
Create a .spec file from the script you want to convert to an exe using this command:
pyi-makespec --onefile yourscriptfile.py
This creates a .spec file that pyinstaller uses to create the stand-alone exe.
Edit the .spec file in a text editor. In my case I added the following details required to solve my issue:
from PyInstaller.utils.hooks import collect_submodules allhiddenimports = collect_submodules('pyarrow.vendored')
Edit the field 'hiddenimports' to read as follows:
hiddenimports=allhiddenimports,
Build the script using the .spec file with the following command: pyinstaller yourspecfile.spec
This did indeed produce an executable that runs without dependency issues.
I don't work on Python that often and needed both those pieces of info. above to get to a working solution. I would like to give joint kudos but...
Thanks.