I'm working on a grammar for a language that supports the right shift operator and generic types. For example:
function rectangle(): Pair<Tuple<Float, Float>> {
let x = 0 >> 2;
}
My problem is, during scanning, the right shift operator is correctly tokenized, but the >>
in Pair<Tuple<Float, Float>>
becomes a single >>
token instead of two separate >
tokens (unless I add a space). This is because I have the >>
before the >
in my .jison file:
">>" { return '>>' }
">" { return '>' }
Is there a good way to resolve this in Jison? I feel like this is a common problem as my syntax is similar to every other C-style language, but I haven't found a solution to it yet (besides writing a pre-scan script that manually space-delimits the >
s).
The easiest solution is to just not recognize >>
as a single token in the lexer. Instead, in your parser, recognize two consecutive >
tokens as a right shift, and then check to make sure there's nothing (no whitespace or comments) between them (and give a syntax error if there is).