[antlr-interest] Weird (to me) grammar problem

David-Sarah Hopwood david-sarah at jacaranda.org
Wed Dec 2 17:06:20 PST 2009


Peter Boughton wrote:
>> (It's not the use of string literals vs token rules that is significant
>> here; just whether arrow is a lexer or parser rule. This seems to be
>> one of the most common mistakes made by people new to ANTLR.)
> 
> Just to clarify, you're saying that a parser rule will always take
> precedence over a lexer rule, if both are possible?

The model is that the lexer produces a sequence of tokens that are
input to the parser. So lexer rules don't have precedence relative
to parser rules; they are applied at different stages.

To use ANTLR effectively, you need to have a clear idea of what should
count as a token for your grammar, since that determines the boundary
between the lexer and parser. For most languages, this is constrained
by the fact that tokens can have ignored whitespace between them, but
not within them.

> Does the order of the alternation matter?
> (i.e. "RARROW | COLON" vs "COLON | RARROW")

No, not in this case.
(It can matter, but normally only if you ignore an ambiguity warning.)

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091203/1fce37b0/attachment.bin 


More information about the antlr-interest mailing list