[antlr-interest] Lookahead predicates in the Lexer?

Gerald Rosenberg gerald at certiv.net
Mon Nov 12 15:29:02 PST 2012


In Antlr4, is there a way to do a fixed lookahead in the lexer predicate 
without capturing the lookahead token(s)?  In v3, predicates could be 
used for this purpose.

csvRule : A ( Comma B )* ;

A : P Q R -> pushMode(Alphabet)

mode Alphabet;
B : X Y Z { ~Comma }? -> popMode
    : X Y Z ;

In v4 , the "~Comma" is presumed to be native code.

Basically, looking for a clean, workable way to not require the use of a 
semicolon to explicitly terminate input that matches the csvRule, yet 
have a representation in the lexer that can be used as the popMode trigger.

I do realize that I can write a predicate method to do a stream scan, 
but would prefer a non-native code solution if possible.  Also realize 
that, in the simplest case, csvRule could be pushed down into the 
Lexer.  Where A and B  are not just single terminals in the parser, 
other rules would have to be pushed down also, making for a bit of a mess.

Thanks,
Gerald


More information about the antlr-interest mailing list