pythonapple-m1cpythonrosetta

MacOS M1 system is detected as ARM by Python package even though I'm using Rosetta

I'm on a Macbook with M1 (Apple ARM architecture) and I've tried running the following Python code using the layoutparser library, which indirectly uses pycocotools:

import layoutparser as lp
lp.Detectron2LayoutModel()

And I've received the error:

[...]
ImportError: 
dlopen([...]/.venv/lib/python3.9/site-packages/pycocotools/_mask.cpython-39-darwin.so, 0x0002): 
tried: 
'[...]/.venv/lib/python3.9/site-packages/pycocotools/_mask.cpython-39-darwin.so' 
(mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), 
'/usr/local/lib/_mask.cpython-39-darwin.so' (no such file), 
'/usr/lib/_mask.cpython-39-darwin.so' (no such file)

The crucial info for me seems to be [...] is an incompatible architecture (have 'x86_64', need 'arm64e') [...]. Indeed, I am using the Apple ARM architecture, and sometimes it is not supported by some software. This is usually solved by using Rosetta, which simulates an Intel-x64 architecture. So I start a terminal with Rosetta (arch -x86_64 zsh), create a new virtual environment, make a fresh install of the dependencies, and try to run the code again ...

... and I receive the same error that I had also had without Rosetta:

[...] is an incompatible architecture (have 'x86_64', need 'arm64e') [...] 🥲

I've double-checked that Rosetta is really activated:

> uname -m
x86_64

Rosetta seems to be working. And yet, according to the error message, it seems not to be working.

Any ideas what could be the problem with Rosetta, or the library, or whatever, and how I could try fixing it?

Solution

  • Charles Duffy explained the problem in the comments, thank you! 😃

    When I checked the platform in Python, it was indeed ARM:

    > python -c 'import platform; print(platform.platform())'
macOS-12.0.1-arm64-i386-64bit

    So I had been using a Python installation for ARM.

    Now I installed brew and then python3 from the Rosetta terminal and used the newly installed Python to initiate a fresh virtual environment, and this fixed it. (This article helped me a bit with it.)

    Update:

    When creating Python environments with conda, it is possible to specify whether they should use Apple ARM or Intel-x64: