open-policy-agent

How to build OPA bundles when data files can exceed 1gb size?


I'm trying to build an OPA bundle. Some of the included raw data.json files are larger than 1GB (1073741824 bytes) and exceed the preconfigured size_limit_bytes. Build CLI command:

./opa build -b /path/to/bundle

causes the error:

error: load error: bundle /path/to/bundle: bundle file extra/data.json size (1073741825 bytes) exceeds configured size_limit_bytes (1073741824 bytes)

OPA version: 0.62.1

Is there a way to specify custom size_limit_bytes for opa build? For opa run it is possible to configure this parameter via the config file or --set, but there is no such option for opa build.

As a workaround - it is possible to build .tar.gz directly using tar and gzip, but using opa build is a preferred way. Any options to do that?


Solution

  • I'm afraid the bundle size limit is not exposed as a CLI parameter to opa build. It thus falls back to the default of 1G.

    You could

    (Note that I just brought Enterprise OPA up to give you an easy way out. The higher default is not part of its value proposition. And you don't need a license to run it for your purposes, you should be able to go from opa build ... straight to eopa build ...)