buildbot

How to retrieve a single property from a NestedParameter?


In my buildbot config, I am setting up some util.NestedParameter to improve the UI. The problem is to access the properties. With the config

from buildbot.plugins import util

util.NestedParameter(
    name='artifacts',
    label='Artifacts to store',
    columns=2,
    fields=[
        util.NestedParameter(
            name='tools',
            label='Tools',
            columns=1,
            fields=[
                util.BooleanParameter(
                    name='linux',
                    label='Linux',
                    default=True,
                ),
            ],
        ),
    ],
)

it makes the property artifacts to have the following value:

{
  "tools": {
    "linux": true
  }
}

so it is really nice, BUT I cannot access a single element of it, I can only retrieve the full property as a string when I do:

cmd = './.venv/bin/python ci/buildbot/release.py %(prop:artifacts)s'
util.ShellArg(command=util.Interpolate(cmd))

but I need to access each of those properties individually to provide them downstream like this:

cmd = './.venv/bin/python ci/buildbot/release.py --linux=<HELPME>'
util.ShellArg(command=util.Interpolate(cmd))

I have seen other properties which are also dict and we use them with props.getProperty(property_name) and this gives a dict. But from what I understood in the doc, I cannot use getProperty to build a string.

I tried to ask my AI friend and he explained that I should use util.renderer but I am not familiar with it...

What I have tried so far is to implement a new function

@util.renderer
def nested_property_renderer(props: IProperties, property_name: str, *args: str):
    value = props.etProperty(property_name)
    for key in args:
        value = value[key]
    return value

then using it as

cmd = './.venv/bin/python ci/buildbot/release.py %(nested_property_renderer:artifacts,tools,linux)s'
util.ShellArg(command=util.Interpolate(cmd))

but now it complains it does not understand me:

Configuration Errors: invalid Interpolate selector 'nested_property_renderer'

So basic question is: are there any people out there using NestedParameter on buildbot and accessing each property individually? How do you do this? (And why is it not in https://docs.buildbot.net/current/manual/configuration/properties.html ?)

I have read How to access a property variable of buildbot which is a dictionary? but I could not find (understand?) how to fix my problem.

Note: debugging on-going, I may come back here with updates later. But I really have no clue how to do this, I am doing trial and error. If you know, please share your knowledge :)


Solution

  • So we found a way to achieve our goal, but it is more a workaround. Here is the code in its simplified version (no docstrings or checks on arguments).

    We implement a function to get a renderer:

    def create_nested_property_renderer(property_name: str, *args: str):
        @util.renderer
        def render_nested_property(props: IProperties):
            value = props.getProperty(property_name)
            for arg in args:
                value = value[arg]
            return value
        return render_nested_property
    

    then we set new properties to flatten the nested one (we do not alter the existing nested properties, we create new ones from its values):

    factory.addStep(steps.SetProperty(
        property='artifacts.tools.linux',
        value=create_nested_property_renderer('artifacts', 'tools', 'linux'),
        hideStepIf=True,
    ))
    

    and now we can use it as a regular property. This is of course done early in our factory.