[antlr-interest] Context-sensitive Lexer
Johannes Luber
jaluber at gmx.de
Mon Sep 22 13:15:28 PDT 2008
Arcadio Rubio García schrieb:
> Hi everyone,
>
> I've recently migrated to ANTLR from Yacc, and I don't find the way to
> do context-sensitive scanning (lexing) with ANTLR.
>
> Can anyone point how to define an identifier token so that it's not
> recognized when it's adjacent to an integer for instance, 1a is not
> recognized as an integer + identifier?
>
> In Lex+Yacc, I used to do this with regular expressions, e.g.:
>
> identifier = /\b[a-zA-Z_][a-zA-Z0-9_]*\b/
>
> Thanks in advance.
There is no way to so directly in ANTLR, unless you create a new lexer
class which doesn't consume the entire input at once. The usual method
to deal with this is to rewrite tokens in the parser for the second pass.
Johannes
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
More information about the antlr-interest
mailing list