I am using Python3.10 on Ubuntu 22.04.4, and I am trying to run code that I originally wrote on a Windows 11 machine. Whenever I run this script--main.py--it always stops at the import stages, and fails to import beautifulsoup4 and yfinance in particular:
print("Starting imports.")
import pandas
print("Imported pandas...")
from bs4 import BeautifulSoup
print("Imported beautifulsoup4...")
import yfinance as yf
print("Imported yfinance...")
# Rest of code...
When I run this script in the terminal, it only ever prints Starting imports. Imported pandas.... Following that print statement, it spits out Illegal instruction (core dumped):
(myenv) minime@Shaguar:~/Coding/WebScraping$ python main.py
Starting imports.
Imported pandas...
Illegal instruction (core dumped)
I have done some research into this "Illegal instruction" error and have come across failed imports of TensorFlow, numpy, and others because of incompatible cpu architecture, yet I do not know how it would apply to beautifulsoup4 in this situation.
How could I successfully import beautifulsoup4 and yfinance without the Illegal instruction (core dumped) error?
(Architecture output of lscpu):
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 36 bits physical, 48 bits virtual
Byte Order: Little Endian
I have solved it! After further inspection of /var/log/syslog--the Ubuntu error/warning logfile--I found that there was a problem with some etree thing:
kernel: [ 3887.864653] traps: python3[5907] trap invalid opcode ip:74ab4c0807c0 sp:7fffee156560 error:0 in etree.cpython-310-x86_64-linux-gnu.so[74ab4c04e000+329000]
After some research on etree and what it is used for, I found that a library shared by both yfinance and beautifulsoup4 that runs on etree was the cause of the core dumpage: lxml.
I looked at the lxml PyPI page and saw that the latest version (lxml 5.20) did not support CPU architectures that pre-dated 2011; my Ubuntu machine is from 2009. So, I uninstalled and downgraded my lxml version to v5.0 using pip and everything worked smoothly.
Key Takeaway
If you are having a similar issue, it is probably due to incompatible CPU architecture with regards to a library or a library's underlying dependencies. Check your system logfiles after a core dump. Do some research, especially on CPU compatability, on any specific keywords in that logfile--in my case it was etree. Lastly, try to downgrade the suspected problem library--in my case it was lxml--to a version that supports your architecture.
Everything above is what worked for me.