I am installing OpenFOAM with spack. The Spack is configured as follows:
cat << EOF >> $SPACK_ROOT/etc/spack/packages.yaml
packages:
slurm:
externals:
- spec: slurm@22.05.5
prefix: /usr
buildable: False
libevent:
externals:
- spec: libevent@2.1.8
prefix: /usr
buildable: False
pmix:
externals:
- spec: pmix@4.2.3
prefix: /usr
buildable: False
hwloc:
externals:
- spec: hwloc@2.2.0
prefix: /usr
buildable: False
EOF
The available variants for OpenFOAM reported by Spack are:
# spack list openfoam
...
Variants:
Name [Default] When Allowed values Description
====================== ==== ============== ==================================================
build_system [generic] -- generic Build systems supported by the package
int64 [off] -- on, off With 64-bit labels
kahip [off] -- on, off With kahip decomposition
knl [off] -- on, off Use KNL compiler settings
metis [off] -- on, off With metis decomposition
mgridgen [off] -- on, off With mgridgen support
paraview [off] -- on, off Build paraview plugins and runtime post-processing
precision [dp] -- sp, dp, spdp Precision option
scotch [on] -- on, off With scotch/ptscotch decomposition
source [on] -- on, off Install library/application sources and tutorials
vtk [off] -- on, off With VTK runTimePostProcessing
zoltan [off] -- on, off With zoltan renumbering
...
The compilation is performed with the commands:
spack install -j openmpi@4.0.5 +pmi schedulers=slurm ^pmix@4.2.3 ^hwloc@2.2.0
spack install -j openfoam +metis +scotch +zoltan +kahip +mgridgen ^openmpi@4.0.5
After the installation I load the OpenFOAM package,
spack load openfoam
and I follow the instructions in the doc/Build.md file of the source code to test the installation:
# Create the user "run" directory:
mkdir -p "$FOAM_RUN"
# Change to the user "run" directory:
run
# Copy tutorial
cp -r "$FOAM_TUTORIALS"/incompressible/simpleFoam/pitzDaily ./
# Run the tutorial
( cd pitzDaily && blockMesh && simpleFoam )
However, there is no simpleFoam solver installed. A search reveals that the code for the simpleFoam solver is present, but no executable is generated:
# find $(spack location -i openfoam) -name simpleFoam
/mnt/irisgpfs/users/gkafanas/.spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/openfoam-2306-ytaggtrqximxwyjmh6b77mw4j66leaxn/tutorials/basic/simpleFoam
/mnt/irisgpfs/users/gkafanas/.spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/openfoam-2306-ytaggtrqximxwyjmh6b77mw4j66leaxn/tutorials/incompressible/simpleFoam
/mnt/irisgpfs/users/gkafanas/.spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/openfoam-2306-ytaggtrqximxwyjmh6b77mw4j66leaxn/etc/caseDicts/createZeroDirectoryTemplates/solvers/simpleFoam
/mnt/irisgpfs/users/gkafanas/.spack/opt/spack/linux-rhel8-zen/gcc-8.5.0/openfoam-2306-ytaggtrqximxwyjmh6b77mw4j66leaxn/applications/solvers/incompressible/simpleFoam
A more detailed search reveals that multiple utility (applications/utilities/) and solver (applications/solvers/) executables are not installed.
Is it possible to compile and load all the utility and solver executables with Spack? Does it require any extra configuration?
The trick may be to disable METIS and ParaView, see below.
We have several SPACK/OpenFOAM versions on different Linux distributions (with a very similar packages.yaml):
CentOS 7 (manually compiled GCC 10.3.0)
SPACK 0.17.1 openfoam@2112%gcc@10.3.0~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan
SPACK 0.19.0 openfoam@2212%gcc@10.3.0~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan build_system=generic
Ubuntu focal (system GCC 9.3.0):
SPACK 0.19.0 openfoam@2106%gcc@9.3.0~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan build_system=generic
I did not have success compiling OpenFOAM with METIS support (yet) and so far I have trouble compiling ParaView from SPACK, so these is disabled. ParaView is installed from system or upstream binaries.
The following should work:
spack install openfoam@2212 ~float32~int64~kahip~knl~metis~mgridgen~paraview+scotch+source~spdp~vtk~zoltan ^openmpi@4.0.5
SPACK can be a little bit temperamental depending on Linux distribution and compiler (and installed system packages).