jsonjsonnet

jsonnet top level function and data in the same file not possible?


I'm trying to automatically generate nested templated json structures with jsonnet. Think about a warehouse structure, in which you'd have a physical site, on this site there are n1 warehouses, these warehouses have n2 ailes which in turn have n3 racks and so forth.

The output is supposed to look like this:

{
   "Sites": [
      {
         "SiteId": 1,
         "Warehouses": [
            {
               "SiteId": 1,
               "WarehouseId": "1-1"
            },
            {
               "SiteId": 1,
               "WarehouseId": "1-2"
            },
            {
               "SiteId": 1,
               "WarehouseId": "1-3"
            }
         ]
      },
      {
         "SiteId": 2,
         "Warehouses": [
            {
               "SiteId": 2,
               "WarehouseId": "2-1"
            },
            {
               "SiteId": 2,
               "WarehouseId": "2-2"
            },
            {
               "SiteId": 2,
               "WarehouseId": "2-3"
            }
         ]
      }
   ]
}

The important thing here is that this should be modularized (aka separate templates for the sites, warehouses, aisles, racks) and I want to be able to run each module individually instead of having to generate the entire structure all over.

I actually already got his working with the following code:

//warehouses.libsonnet
{
   warehouses (site, numWarehouses)::
   [
      {
         SiteId: site,
         WarehouseId: site + '-' + y,
      }
   for y in std.range(1, numWarehouses)
   ],
}
//sites.libsonnet
local warehouses = import 'warehouses.libsonnet';

{
   sites (numSites,numLocations)::
   [
      {
         SiteId: y,
         Warehouses: warehouses.warehouses(y,numLocations),
      }
   for y in std.range(1,numSites)
   ],
}
//warehouses.jsonnet
local warehouses = import 'warehouses.libsonnet';

function(s,n)
   {
      Warehouses: warehouses.warehouses(s,n)
   }
//sites.jsonnet
local sites = import 'sites.libsonnet';

function(s,n)
   {
      Sites: sites.sites(s,n)
   }

I can run warehouses.jsonnet and sites.jsonnet individually to generate either just the warehouses or the sites including the warehouses.

I am however struggling with two problems:

  1. getting this into just two files. As you can see my solution requires "launchers" in the form of the jsonnet files which do nothing else except running the libsonnet.
  2. Also there's redundancy, for example the property name "Warehouses" appears in both warehouses.jsonnet and sites.libsonnet.

I tried every permutation I could think of trying to somehow get the top-level function in there with the json template in the respective libsonnet file or to get the redundant property name into the template that generates the property contents where it semantically belongs. I searched all over Stackoverflow, blogs, the docs etc. but I couldn't find anything that helped. I already sank two and a half days into this, so I'd really appreciate the advice of the gurus on this platform.

I tried moving the json template into a function itself, or adding the top-level function from the jsonnet files to the json template files, each time the parser complains it's not expecting a function at that location.

Is there a way to simplify my solution?


Solution

  • Mind that imports can be (mostly) thought as copied-in code inline where the import is used.

    I'm not sure if I understood you correctly (tbh I see your library approach pretty solid), nevertheless below single all.jsonnet slurps in the above four sources into one, and works as expected when invoked as per the documented example in the comments:

    // all.jsonnet
    
    // implementation copied-in from warehouses.libsonnet:
    local warehouses = {
      warehouses(site, numWarehouses)::
        [
          {
            SiteId: site,
            WarehouseId: site + '-' + y,
          }
          for y in std.range(1, numWarehouses)
        ],
    };
    
    // implementation copied-in from sites.libsonnet:
    local sites = {
      sites(numSites, numLocations)::
        [
          {
            SiteId: y,
            Warehouses: warehouses.warehouses(y, numLocations),
          }
          for y in std.range(1, numSites)
        ],
    };
    
    // Below is still a top-level function (all the above are locals) with "main" function
    // keyed by `sites` (default) and `warehouses`.
    // Use it as e.g.:
    //   $ jsonnet all.jsonnet --tla-code s=42 --tla-code n=3
    //   $ jsonnet all.jsonnet --tla-code s=42 --tla-code n=3 --tla-code f="'warehouses'"
    function(s, n, f='sites')
      {
        // implementation copied-in from sites.jsonnet:
        sites:: function(s, n)
          {
            Sites: sites.sites(s, n),
          },
        // implementation copied-in from warehouses.jsonnet:
        warehouses:: function(site, numWarehouses)
          {
            Warehouses: warehouses.warehouses(site, numWarehouses),
          },
      }[f](s, n)