intellij-ideajflexgrammar-kit

Grammar-Kit: how to handle comment tokens

From the documentation provided for grammar-kit, I cannot figure out how I am supposed to correctly handle something like comments. My lexer currently returns TokenType.WHITE_SPACE for any comment blocks, but then no unique IElementType is generated for me to do syntax highlighting on.

If I create an IElementType and tell flex to return that for comments, I can perform syntax highlighting, but then that token is not a part of my language spec in the BNF, and so it is considered invalid.

What is the correct way to pass comments through as white space, but perform syntax highlighting on them in Intellij/grammar-kit/jflex?

Solution

  • You can use Grammar-Kit implementation as a reference:

    1. Lexer
    2. Grammar
    3. ParserDefinition

    Using TokenType.WHITE_SPACE for comments is a bad idea.

    More details can be found here.