[antlr-interest] second lexical pass

Terence Parr parrt at cs.usfca.edu
Sun Apr 11 10:01:55 PDT 2004


On Apr 11, 2004, at 6:25 AM, idontwantanidwith2000init wrote:

> Is there an elegant way to do a second lexical path instead of
> writing two lexers?
>
> For instance:
> STRING_LITERAL:
> '"' (~('\r'|'n'|'\\') | '\\' .) '"' ;
>
> now STRING_LITERAL will colide with any special string you'd like to
> match, for instance:
> DATE_TIME : '"'YYYY-MM-DD hh:mm:ss'"'
>
> I've managed to merge it with:
> '"'(LETTER)*'"'
> '"'(LETTER)*'.'(LETTER)*'"'
> '"'(LETTER)*'-'(LETTER)*'"'
> Which was a diffucult task but doable.
>
>
> I thought about it and merging these rules is possible, but it will
> take me days to write it and we haven't talked about validation.
> (It is a good to write down my thoughts I'm starting to convince
> myself that it is a good idea)
>
> What do you think is a proper solution?

My first thought is that you have a two-level grammatical issue here.  
First you have to match a string and then determine if it satisfies a 
"nested structure" which is the date/time stuff.  You can't really use 
another rule in the same lexer object as it will consume the input.  
Perhaps a compromise.  Make a new lexer rule that will match DATE_TIME 
as you've done and make it protected.  Then, make a *new* instance of 
your lexer.  Set it up with a copy of the text matched inside the 
string and then directly call DATE_TIME with a try/catch around it to 
catch failure.  Gotta love LL based lexers :)

Ter
--
Professor Comp. Sci., University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again!
Cofounder, http://www.peerscope.com pure link sharing





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list