[antlr-interest] PEG-style "and" predicates

[Richard Clark]

On 7/2/07, Wincent Colaiuta <win at wincent.com> wrote:

> My understanding of predicates (notes at <http://wincent.com/
> knowledge-base/ANTLR_predicates>) leads me to believe that this is
> failing because in ANTLR, syntactic predicates are used to order a
> rule's alternatives.

Hi Wincent,

It took careful reading of all the available material, but here's what
I know about predicates in v3 (coming from an old hand at v2):

1. Syntactic predicates (of the form (FOO) => foo) are used to remove
the ambiguity among a set of rules. **If a static analysis shows no
ambiguity, the syntactic predicate's code is never generated.** These
may be "hoisted" out of the defining rule to wherever needed.

2. Semantic predicates ( {input.LA(1) == FOO} => foo ) are the same as
syntatic predicates except you get to specify arbitrary code. They too
won't be compiled in if there's no ambiguity in the grammar and may be
hoisted to where needed.

3. Gated semantic predicates ( { myHelper(); }? => foo ) are always
compiled in. They're normally used when outside forces change how the
grammar is handled (e.g. command line options.)

So when you write:
FOO : ('bar' ' '* ('\n' | '\r' | EOF))=> 'bar' ;
the rule isn't actually ambiguous ('bar' matches 'bar' and the first
lexer rule to match wins), so the predicate is unused.

You're right that you need a gated semantic predicate with some custom code:
FOO : { foo_helper(ctx) }?=> 'bar' ;

I was surprised you're doing this in the lexer. It seems that if the
token changes meaning depending on its position in the line, it's the
parser's job to determine such things. (In my opinion, your mileage
may vary, void where prohibited, etc. etc. etc.)

 ...Richard