[antlr-interest] Gavin: Lexer 2-char seq handling

Graham Wideman gwlist at grahamwideman.com
Mon Oct 12 14:52:27 PDT 2009


Hi Gavin (and Joan and Martin):

In the last couple of days you've advised on how to handle lexer rules that eat a sequence of chars that should stop when it gets to some 2-char sequence:

']]' in Martin's case
'${' in Joan's case

In looking into this, I see the same thing in the generated code that Joan reports: Namely that the rule handling eating the string doesn't look ahead to see the second character during the "select an alternative" phase, so tries to eat it, but then in the "match and consume" phase it does detect that the not-part-of-this-rule character-pair is there, and throws an exception.

You clearly expect this to work -- is there some extra nuance to this? Like a particular option that must be set?

FWIW, using ANTLR 3.2.

Thanks,

Graham



More information about the antlr-interest mailing list