javascriptparsinglexermoonearley

Nearley. How to resolve grammar Nearley use JS syntax

Let's say I have some kind of grammar, but it works by itself, but the JS code does not recognize it. Is there such an option so that you can combine the code of your grammar and syntax in one.

Let's say I have this grammar (Nearley + Moo).

@{%
const lexer = moo.compile({
   start_tag: "<",
   tag: /A-Z.../,
   close_tag: ">"
})
%}

elemcreation -> "<" %tag ">"

That is, my grammar recognizes such this syntax, and works only with it, but how can I make it possible to write and combine the syntax of my grammar and the grammar of the JS itself. That is JSX i want to do it in JS.

That is:

// my grammar syntax

"< tag >"

// js grammar syntax

"const mycon = "Hello, world!";"

How to make it possible to combine them, but not by defining the entire js syntax in nearley

Solution

  • Use a wildcard pattern to capture the characters you don't care about into a token & specify how that token fits into the rest of your grammar.

    Here is an example nearley grammar file that captures characters before, in the middle and outside of the open & close tag.

    @{%
const moo = require('moo');

const lexer = moo.compile({
   open_tag: /\<[a-zA-Z0-9_]+\>/,
   close_tag: /\<\/[a-zA-Z0-9_]+\>/,
   bodyless_tag: /\<[a-zA-Z0-9_]+\/\>/,
   left_arrow : /\</,
   something: { match:/[^\<]+/, lineBreaks:true}
})
%}

@lexer lexer

main        -> something_ elem something_
               {% (d) =>  [ d[0], ...d[1], d[2] ] %}

elem        -> %open_tag something_ %close_tag
               {% (d) => [ d[0], d[1], d[2] ] %}

something_   -> %something:* {% (d) => d[0][0] %}

    On this JSX snippet...

        const jsx = <h1>This is JSX</h1>; 
    const mycon = "Hello, world!

    This is output using nearley-test. Some properties are truncated for brevity.

     [
  [
    { type: 'something', value: 'const jsx = ' },
    { type: 'open_tag', value: '<h1>' },
    { type: 'something', value: 'This is JSX'    },
    { type: 'close_tag', value: '</h1>' },
    { type: 'something', value: '; \nconst mycon = "Hello, world!\n' }
  ]
]