[antlr-interest] frustrated with lexer

[netminka at netscape.net]

I like the expression parsers and the tree parsers a lot, but this lexer
implementation is bogus, in my opinion. Maybe the wizards out there can
direct me to the simple solution for what I believe should be a simple problem. If not, next time I will using another lexer with antlr, or 
if none is available, use another parser generator totally. That would
be unfortunate, because there is a lot to like here.

BUT THE LEXER!!!

I have spend more hours spinning, trying to find ways around this lexer
to solve what would be simple cases in flex. It's just not worth it!

The latest example:
I sometimes need to scan ahead through the input and once I've
determined the context or whatever, push back what I've scanned onto the input stream. I DON'T NEED to push back everything but the first character!
Which seems to be the consume() default. 

How is this consume default changed? Example please!

Here is the specific situation:
END
    : ("End" LINE_TERMINATOR) => ENDEXIT
    | ("End" (' ' | '\t')+) => ENDCHECK
    ;

In the case that "End" followed by the above stuff is not recognized
(e.g. the string 'EndTest') the lexer consumes the 'E' and I'm left with
'ndTest'. Note my ENDCHECK and ENDEXIT are protected. 

I also don't like
the hoisting of rules in nextToken based on left hand side semantic predicates; the effects are unpredictable and overly complex. The ordering
of matching rules is overly complex and thus unpredictable as well. 

thanks for any help you can give.
megan adams 


__________________________________________________________________
McAfee VirusScan Online from the Netscape Network.
Comprehensive protection for your entire computer. Get your free trial today!
http://channels.netscape.com/ns/computing/mcafee/index.jsp?promo=393397

Get AOL Instant Messenger 5.1 free of charge.  Download Now!
http://aim.aol.com/aimnew/Aim/register.adp?promo=380455

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/