[antlr-interest] Why doesn't this work?

[Thomas Brandon]

I believe that Antlr will automatically give precedence to a longer match.
So while "UW" could be matched as two tokens 'U' and 'W' which are both of a
higher precedence than IDENTIFIER Antlr gives precedence to the match of
single IDENTIFIER token. Hence having a token for 'UW', as in the second
example I gave, causes Antlr to correctly give precedence to this when
seeing "UW".

Tom.

On 6/8/07, Cameron Esfahani <dirty at apple.com> wrote:
>
> I've been looking at the wiki, and in some of the documentation, and it
> claims:
> The earlier a token is defined, the higher is the precedence if a certain
> input can be matched by two or more tokens. This means that using the tokens
> command to define keywords will match those keywords instead a more general
> ID rule
>
> So I'm still not clear why the "UW" is being matched by IDENTIFIER in my
> test grammar.  IDENTIFIER is the last rule in Test.g.
>
> On Jun 6, 2007, at 9:04 PM, Thomas Brandon wrote:
>
> All lexing occurs before parsing. "UW" is being matched by the IDENTIFIER
> rule rather than the 'U' and 'W' rules generated by the parser literals. I
> think because Antlr will prefer a longer match this cannot be fixed with
> precedence (putting the 'U' and 'W' rules above IDENTIFIER, which I think
> they already should be), " 100.W" will correctly parse as then the higher
> precedence of the 'W' rule allows it to win out over the IDENTIFIER rule
> (meaning according to this grammar "U", "W" etc cannot be used as
> identifiers).  Assuming that "UW" is a valid identifier (and so can't be
> excluded from IDENTIFIER) you might need to make your size_qualifier rule
> deal with IDENTIFIERs and use predicates to test the text. Something like:
>
>
>
> Cameron Esfahani
> dirty at apple.com
>
> "Americans are very skilled at creating a custom meaning from something
> that's mass-produced."
>
> Ann Powers
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070608/d921477b/attachment.html