[antlr-interest] Q: Matching literals in the parser or Lexer, which is better ????

[Sylvain, Gregory [USA]]

Hello all,

Looking for information about best practice, performance and reasoning with respect to using literals in the parser verses using tokens in the parser.   For example, if you know you will have a phrase of the language you are trying to parse begin with 'CONTENT/', which is better:

Variation A:

r    : 'CONTENT/' <rest> ;


Variation B:

r    : CONTENT_TOK <rest> ;

CONTENT_TOK    : 'CONTENT/' ;

Matching the literal in the parser rule, or creating a TOKEN in the lexer and matching the token in the parser?


Thanks in advance for you feedback.

Greg

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090908/343880e5/attachment.html