Unable to get pillars in salt runner

I'm writing a custom runner in saltstack to do some operations on saltmaster. However I'm unable to invoke the pillars within the runner.

The secrets are stored in the pillars referenced against the saltmaster id in the pillar top.sls for example

prd:
    'my_saltmaster':
        - match: pcre
        - salt_secrets

This is what I've tried

import salt.runner
import logging

log = logging.getLogger(__name__)

runner = salt.runner.Runner(__opts__)
secret = runner.cmd(fun='salt.cmd', arg=['pillars.get', 'my_secret'])
log.info(f"my_secret = {secret}")

Output

my_secret =

I've checked the official source code and couldn't find anything resourceful. It would be great if you can shed some light on this.

If you want to call the module function salt.cmd you should also pass the required kwarg with_pillar to enable rendering the pillars as mentioned on the official doc

You also probably want to correct the arg to pillar instead of pillars

secret = runner.cmd(fun='salt.cmd', arg=['pillar.get', 'my_secret'],  kwargs={"with_pillar": True})

However, I won't prefer this approach for two reasons:

1. it assumes your saltmaster's id matches the minion id you targeted in your top.sls, which is very unlikely as the master's id is suffixed by _master with the hostname of the host it is running on (by default).

2. highly cumbersome and is expensive. Why? because not only it first invokes the runner client which then calls the runner salt module, which later invokes the pillar module finally!

A better approach would be to import and use the pillar module directly as below and save a lot of expensive operations:

import salt.pillar
import salt.runner
import logging

log = logging.getLogger(__name__)

runner = salt.runner.Runner(__opts__)

master_id = "my_saltmaster"
grains = None # specify any grain if required to render your top.sls

# Load required pillars
# deepcopy because we don't want to modify the actual __opts__
opts = copy.deepcopy(__opts__)
opts['pillar'] = salt.pillar.get_pillar(
    opts,
    grains,
    master_id,
    saltenv=opts['saltenv'],
    pillarenv=opts.get('pillarenv')).compile_pillar()

# extract required secret key from the pillar
my_secret = opts['pillar'].get('my_secret', None)
log.info(f"my_secret = {my_secret}")

Read more about the pillar module code here :)