[SOLVED] Bamboo - How can I share environment modules between tasks in a job without loading them for every task?

Bamboo - How can I share environment modules between tasks in a job without loading them for every task?

My plan consists of many jobs and many script tasks in each of those jobs.

Most of these script tasks rely on the same 5 or 6 modules that I am loading every time, for example:

module load python/argparse_py2.7/1.2.1
module load python/python/2.7.13

Instead of having to load these environment modules for every task, is there a way to load them once and have all subsequent tasks be able to access them?

Solution

Once you load a module from a script or an interactive shell session, this module stays loaded for the rest of the script or session and it is also still loaded in the context of sub-script launched by the main one or sub-shells launched by the main session.

So in a script context where you launch many sub-tasks/scripts, you should put your module load commands near the beginning of the script to get those modules enables for the rest of the execution or sub-executions.

This is also valid for an interactive shell session.

In addition, new versions of the module command provide a way to save a loaded module context and restore it later on (in script or in other shell sessions).

Here is a small example for module collection usage:

$ module purge
$ module load python/argparse_py2.7/1.2.1 python/python/2.7.13
$ module save pythonjobenv
$ module savelist
Named collection list:
 1) pythonjobenv
$ module purge
$ module restore pythonjobenv
Restoring collection pythonjobenv
  Loading module: python/argparse_py2.7/1.2.1 python/python/2.7.13