[antlr-interest] Is it possible to do rewriting for lexer?

Gokulakannan Somasundaram gokul007 at gmail.com
Tue Jul 26 23:45:38 PDT 2011


Hi Qiao,
    If you feel the GETTEXT() is costly, the easiest way is to write a
parser rule, which receives this as a token and rewrites and returns the
desired. Instead of matching the lexer token in elsewhere in your parser
grammar, try to match the new parser rule.

For example:
Lexer Grammar:
SINGLE_QUOTED_STRING :  SQUOTE (~SQUOTE)* SQUOTE ;

Parser Grammar:
single_quoted_string[std::string& result]  : SINGLE_QUOTED_STRING
{
   pANTLR3_COMMON_TOKEN token = $SINGLE_QUOTED_STRING;
   ANTLR3_MARKER start = token->getStartIndex(token);
   ANTLR3_MARKER end = token->getStopIndex(token);
   result.assign( (char*) start, stop-start+1 ); //create it with your
requirements.

};

Thanks,
Gokul.


More information about the antlr-interest mailing list