c++quex

Problems porting lexer string accumulator to new version of Quex


While porting my lexer file from Quex 0.64.8 to 0.67.4 I ran into some problems with the string accumulator. The issues I get look like this:

Severity Code Description Project File Line Suppression State
Error C3861 'ecmascript_lexer_Accumulator__clear': identifier not found (compiling source file C:\Users\Patrikj\Work\git\ecmascript_build_vc14_x64\generated\ecmascript_lexer.cpp) ktes C:\Users\Patrikj\Work\git\ecmascript\ecmascript.qx 107

I suppose it's the double underline Accumulator__clear that is the cause of the issue. Maybe I need to supply a new switch to Quex or maybe the API has changed in the newer version. Either way I am at a loss on how to fix the issue.

And example from my lexer (.qx) that generates the issue:

mode StringHelper : EOF
<inheritable: only>
{
  on_exit {
  /// All 3 rows using the accumulator generates an error similiar to the one mentioned above
    if(self.accumulator.text.begin != self.accumulator.text.end)
    self_send(TOK_STRLITPART);

    self_accumulator_flush(TOK_QUOTE);

    self_accumulator_clear(); 
  }
}

Any help fixing this issue would be much appreciated.

Best regards, Patrik J


Solution

  • Version 0.67.3 and later excluded the string accumulator from the main generator. The reason was that for some situations there is no general solution in the construct, include-push, and reset scenarios. Users must specify them as they go along.

    For using the accumulator, no command line option is required. However, in the .qx files the following sections need to be defined (this is an example):

    header {
    #include <quex/code_base/extra/accumulator/Accumulator>
    }
    
    footer {
    #include <quex/code_base/extra/accumulator/Accumulator.i>
    }
    body {
        QUEX_NAME(Accumulator)          accumulator;
    }
    constructor {
        if( ! QUEX_NAME(Accumulator_construct)(&me->accumulator, me) ) {
            return false;
        }
    }
    destructor {
        QUEX_NAME(Accumulator_destruct)(&me->accumulator);
    }
    print {
        QUEX_NAME(Accumulator_print_this)(&me->accumulator);
    }
    

    The case with the PostCategorizer is the same. You find the setup shown below in 'common.qx' files in the demo subdirectories.

    Also, after 'flush()' you do not need to 'clear()'.