[antlr-interest] "lexer state" issues and equivalence to nextToken

[Eric Mahurin]

First of all, if you have a lexer with these rules:

X: <rule for X> ;
Y: <rule for Y> ;

should this be equivalent to:

NextToken
: <rule for X> {$setType(X);}
| <rule for Y> {$setType(Y);}
;
protected X: ;
protected Y: ;


I have cases where X and Y look ambiguous, but that is because they
are valid in separate "lexer states" (I may be using a this term in a
different context than the docs talk about).  I qualify them with
semantic predicates correlating to the mode:

X: {cond}? ... ;
Y: {!cond}? ... ;

Unfortunately, this doesn't work and I have to resort to the NextToken
form for the ones it complains about and this works.  Is this a bug?

Is there a better way to handle these lexer modes?  I think I could
spawn off a new lexer and use a selector for different modes, but I
really don't want to deal with that complexity or overhead.

Eric




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/