javascripttokenizelexernearley

Using nested macros in Nearley nests the data result


The problem

At first sight, macros can't be properly nested without getting some serious bugs.

The main problem is that retrieving a macro's value from the data object nests this value into a list:

a[X] -> $X {% id %}
main -> a["test"] {% id %}

Parse results:
[ [ 'test' ] ]

The expected result would be [ 'test' ].

A quick fix would be to return data[0][0], but it isn't enough, because the result will nest for each layer of macro:

a[X] -> b[$X] {% id %}
b[X] -> $X {% id %}
main -> a["test"] {% id %}

Parse results:
[ [ [ 'x' ] ] ]

To fix the bug we could use data => data[0][0] for each macro. However, this is absolutely ugly.

A real fix would be to use dynamic scoping. Since we can't (to my knowledge) create a macro with no parameter, let's use a useless parameters:

a[X] -> b["_"] {% id %}
b[_] -> $X {% id %}
main -> a["test"] {% id %}

Parse results:
[ [ 'test' ] ]

This stops the nesting hell that happened before - we could pass through 500 sub-macros and still get the same result. But we still need to put data[0][0] for the final sub-macro b, and this prevent us to use the b macro alone by itself - we have to use a for it to work.

We're looking for a solution that: - Allows the use of the last macro by itself - Avoids using data => data[0][0]


Solution

  • The solution

    To avoid the problem, the best solution is the following:

    a[X] -> b[$X] {% id %}
    b[X] -> c[$X] {% id %}
    c[X] -> $X {% data => data[0].join('') %}
    
    main -> a["test"] {% id %}
    

    Parse results:
    [ 'test' ]
    

    Explanation

    The problem was that, when the last sub-macro gets the result, since nearley considers everything as an array by default, the result is nested into an array Then each layer do the same. Using the join method on the array makes it a string - and each macro will stop putting it into an array.