supervisord

Supervisord inconsistent ENV expansion

I am attempting to use ENV expansion in my configuration file. This [program:test] chunk works just fine, and the empty files /tmp/test_0_2 and /tmp/test_1_2 are created. I'm using supervisord version=4.2.5.

[program:test]
command=touch /tmp/%(program_name)s_%(process_num)s_%(ENV_NUM_CPUS)s
directory=/tmp/
startretries=0
autorestart=false
environment=NUM_CPUS=2
numprocs=2
process_name=%(program_name)s_%(process_num)s_%(ENV_NUM_CPUS)s

However, this doesn't work:

[program:test]
command=touch /tmp/%(program_name)s_%(process_num)s_%(ENV_NUM_CPUS)s
directory=/tmp/
startretries=0
autorestart=false
environment=NUM_CPUS=2
numprocs=%(ENV_NUM_CPUS)s ; <- note difference here
process_name=%(program_name)s_%(process_num)s_%(ENV_NUM_CPUS)s

I get this error (followed by a list of available ENV_ variables that doesn't include ENV_NUM_CPUS):

Error: Format string '%(ENV_NUM_CPUS)s' for 'program:test.numprocs' contains names ('ENV_NUM_CPUS') which cannot be expanded.

I want to set numprocs with an ENV. Is there any way to do what I'm trying to do here? This seems like a bug? Why does %(ENV_NUM_CPUS)s work in two places but not the third?

Thanks!

Solution

  • This looks like the result of when the particular configuration option is processed. It looks as if supervisord uses the numprocs option before it has processed the environment option.

    The simplest solution would seem to be setting NUM_CPUS external to supervisord, so that your config looks like:

    [program:test]
command=touch /tmp/%(program_name)s_%(process_num)s_%(ENV_NUM_CPUS)s
directory=/tmp/
startretries=0
autorestart=false
numprocs=%(ENV_NUM_CPUS)s ; <- note difference here
process_name=%(program_name)s_%(process_num)s_%(ENV_NUM_CPUS)s

    And you start start supervisord with that environment variable already set:

    NUM_CPUS=2 supervisord -c my_supervisor_config.conf

    If you're curious, you can find the logic here, where some options are read with do_expand=False which inhibits %(...)s expansions; these are presumably expanded later on after config file parsing is complete (at which point they could take advantage of environment variables defined in the configuration file).