[antlr-interest] Simple (should be) lexer Q

Terence Parr parrt at cs.usfca.edu
Fri Jun 10 12:08:20 PDT 2005


>> Hi.  What does the -traceLexer option tell you?
>
> At the office at the moment.  Will try it tonight.  (Can this be  
> set as an option in the grammar file?  Also, just discovered that  
> the Eclipse plugins are using 2.7.4.  Will that make any meaningful  
> difference?)

Probably not make a difference.

-traceLexer is a command line option.

>>  I guess you're
>> saying that it never chooses the 2nd alternative.
>
> That is what appears to be happening.  Removing the first  
> alternative results in every NAME token containing a single word.
>
>> What is your k
>> level?
>
> 10
>
>> Is WS a single char or can it be multiple?
>
> Effectively single.  For testing purposes, the source text is known  
> to have only a single space between words.
>
>
> WS:
>     ( ' '
>     | '\t'
>     | ( "\r\n" | '\r' | '\n' )
>       { newline (); }
>     )
>     { $setType(Token.SKIP); }
> ;

Curiouser and curiouser.

Ter
--
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com



More information about the antlr-interest mailing list