[antlr-interest] Re: Troubel With Keywords / Whitespaces

lgcraymer lgc at mail1.jpl.nasa.gov
Tue Mar 9 16:31:05 PST 2004


Jason--

In your EAL_SELECT, "select" is not matched as a literal--it is
matched as 's' 'e' 'l' 'e' 'c' 't'.

To get the behavior that you want, delete the EAL_SELECT rule and add
EAL_SELECT="select" to your "tokens" in the lexer.  Or use "select" in
your parser grammar

--Loring

--- In antlr-interest at yahoogroups.com, Jason <jasonriz at y...> wrote:
> Hello,
> 
> I've run into a problem when I run the lexer generated
> with the grammar which follows on, for instance, this
> input stream:  'select foo bar bar selectbar foo'. 
> Everything works ducky except that rather than
> matching the 'REG_ID' rule, 'selectbar' matches the
> 'EAL_SELECT' rule followed by the "REG_ID' rule.  It's
> obvious why this is happening but I can't figure out
> the best way to fix it.  I know it's unusual to make
> each Keyword a rule, but I make heavy use of semantic
> predicates to determine whether a keyword has been
> matched or not.  The only solution I've come up with
> is to change 'select' to 'select ' but there's got to
> be something better.  If anyone has any thoughts I'd
> greatly appreciate it.
> 
> class TestLexer extends Lexer;
> 
> options
> {
>   k = 8;
>   exportVocab=Test;
> }
> 
> EAL_SELECT: "select"   { };
> 
> protected ALPHA_CHAR: ('a'..'z' | 'A'..'Z') { };
> protected DIGIT: ('0'..'9');
> 
> REG_ID: ALPHA_CHAR ( ALPHA_CHAR | DIGIT )* { };
> 
> // Whitespace -- ignored
> WS	:	
>   (	' ' |	'\t' |	'\f' |	(options
> {generateAmbigWarnings=false;}
> 			:	"\r\n" 
> 			|	'\r'   
> 			|	'\n'   
> 			)
> 			{ newline(); }
> 		)+
> 		{ _ttype = Token.SKIP; }
>   ;
> 
> -jason
> 
> -jason
> 
> __________________________________
> Do you Yahoo!?
> Yahoo! Search - Find what you're looking for faster
> http://search.yahoo.com



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list