[SOLVED] Dynamic libraries not found when launching a python script using Slurm

Dynamic libraries not found when launching a python script using Slurm

I have difficulties setting my environment variables when launching a python script using Slurm.

The following python script import petsc4py without error when launched from a shell session :

from petsc4py import PETSc
print("petsc4py imported successfully")

However when I try to run it using srun with the following sbatch script (first setting the same environment by sourcing the same python env and loading the same modules) :

#!/bin/bash
#SBATCH --job-name=petsc4py_import_test      
#SBATCH --ntasks=2                 
#SBATCH --cpus-per-task=8        
#SBATCH --time=00:30:00            
#SBATCH --output=output.log        
#SBATCH --error=error.log          
#SBATCH --nodes=1                  

source $HOME/.bash_profile
module purge
module load python/3.10.0 openmpi gcc openblas/latest slurm
source $HOME/my_env/bin/activate

srun --export=ALL --mpi=pmix_v3 python3 petsc4py_test.py

It fails with :

ImportError: liblapack.so.3: cannot open shared object file: No such file or directory
srun: error: node001: tasks 0-1: Exited with exit code 1

Solution

This could be because liblapack is available as a operating system package on the node where the shell session is run and not available on the node on which the job runs. You might have to load the lapack module if one exists

module load python/3.10.0 openmpi gcc openblas/latest lapack/latest slurm