[antlr-interest] Ambiguous lexing task
Cliff Hudson
cliff.s.hudson at gmail.com
Fri Apr 2 13:59:16 PDT 2010
I've played around with it a bit, and I modified NAMECHAR to be:
fragment NAMECHAR
: LETTER
| DIGIT
| '_'
| {input.LA(2) != '>'}?=> '-'
;
This seems to do the trick. However, I'm concerned this is not a best
practice for this kind of situation. Could I get a suggestion as to the
"correct" way to go about this?
On Fri, Apr 2, 2010 at 1:48 PM, Cliff Hudson <cliff.s.hudson at gmail.com>wrote:
> I have a string which I need to parse for IDs and operators. This is
> normally pretty easy, but there is one case where a character in the ID can
> also match one character in the operator. The tokens are:
>
> OP_TRANSFORM : '->'
>
> ID : (LETTER | '_') (options { greedy=true } : NAMECHAR)*
>
> fragment NAMECHAR : LETTER | DIGIT | '_' | '-' ;
>
> LETTER : 'a'..'z' | 'A'..'Z' ;
> NUMBER: '0'..'9' ;
>
>
> The issue is in parsing the following string:
>
> my-identifier->foo
>
> The ID token of course matches 'my-identifier-', and then I am left with an
> extraneous '>'. Is there a way to construct a set of lexing rules, possibly
> with actions, that would correctly separate out the -> from the ID? In this
> case, I want the '-' in OP_TRANSFORM to be the preferred path and to match
> '->' even in the above case.
>
> Thanks.
>
More information about the antlr-interest
mailing list