I am writing a Jinja2 template to output a JSON DAG where one value is a string containing a list of commands from an external shell script, (containing templated variables. This works if the commands in the script are semicolon-separated on a single line:
echo 'hello'; echo 'world';
But, it fails to render the template when formatted with each command on its own line:
echo 'hello';
echo 'world';
JSONDecodeError: Invalid control character at: line 2 column 29 (char 30)` where character 30 is the line break.
I understand this is because JSON does not support multi-line strings. I am using Jinja2's {% include path %}
to load the file but am not sure how to escape the new lines before rendering them.
I was not able to pipe the include
output through a Python replace successfully:
{% include path | replace("\n", " ") %}
I also tried putting the include
in a macro with the replace
calling inside or outside the macro.
Here is a full example of the template and rendering code.
multi_line.sh
echo 'hello';
echo '{{variable}}';
variables.json
{
"variable": "world!"
}
template.j2
{
{# Render included file into single line #}
"multi": "{% include './multi_line.sh' %}"
}
template_renderer.py
from jinja2 import Environment, FileSystemLoader
import json
with open("./variables.json") as variables_file:
variables_json = json.load(variables_file)
loader = FileSystemLoader("./")
env = Environment(loader=loader)
template = env.get_template("./template.j2")
rendered_template = template.render(variables_json)
json_output = json.loads(rendered_template)
print(json.dumps(json_output, indent=2))
The solution for me was to wrap the include
in a block assignment and then pipe the results into replace
to remove occurrences of \n
.
{% set multi_line_script %}
{% include 'multi_line.sh' %}
{% endset %}
{
"multi": "{{ multi_line_script | replace("\n", " ") }}"
}