[antlr-interest] Context-sensitive Lexer

[Johannes Luber]

Arcadio Rubio García schrieb:
> Hi everyone,
> 
> I've recently migrated to ANTLR from Yacc, and I don't find the way to  
> do context-sensitive scanning (lexing) with ANTLR.
> 
> Can anyone point how to define an identifier token so that it's not  
> recognized when it's adjacent to an integer for instance, 1a is not  
> recognized as an integer + identifier?
> 
> In Lex+Yacc, I used to do this with regular expressions, e.g.:
> 
> identifier = /\b[a-zA-Z_][a-zA-Z0-9_]*\b/
> 
> Thanks in advance.

There is no way to so directly in ANTLR, unless you create a new lexer
class which doesn't consume the entire input at once. The usual method
to deal with this is to rewrite tokens in the parser for the second pass.

Johannes
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>