MemoryError: Unable to allocate 10.0 GiB for an array with shape (80213, 16757) and data type float64
How can I avoid this error and allocate more memory? I am on Windows 11 with 16 GB of RAM
For huge matrices, with most values equal to zero, use sparse matrices instead. (e.g., https://docs.scipy.org/doc/scipy/reference/sparse.html) this will not overwhelm your local RAM.