parsinghaskellcompiler-constructionhappyalex

Happy & Alex - Preventing the lookahead from affecting parser-lexer communication


I am currently writing the parser for a compiler of a toy language using Happy & Alex. Since some form of optional layout is required I have to change Alex's state before matching the block non-terminal. Unfortunately it seems that the lookahead token required by Happy is read before I have the chance to change Alex's state.

Here is a small snippet demonstrating the problem:

funcDef : header localDefs block
                          ^ I have to change alex's state 
                            before the underlying lexer
                            starts reading the block tokens.

Is there a common approach to this problem ?


Solution

  • I am assuming you are using a threaded lexer (so Happy and Alex are running in the same monad). The trick I used when faced with a similar problem is to make an empty production rule that you slip into your rule.

    changeAlexState :: { () }
      : {- empty -} {%% \tok -> changeAlexState *> pushTok tok }
    
    funcDef : header localDefs changeAlexState block
    

    Then, you need to add to your monad some state to support pushTok :: Token -> P () (where P is your lexing/parsing monad) and make sure you always pop that token when when you are lexing. What %% does is documented here.

    n : t_1 ... t_n {%% <expr> }

    ... The type of <expr> is the same [still Token -> P a], but in this case the lookahead token is actually discarded and a new token is read from the input. This can be useful when you want to change the next token and continue parsing.

    I mentioned I did something similar not long ago. Here is my "empty" rule, here is an example use of it, here is where my pushing function is defined and here is where I "pop" tokens. Let me know how it goes!