[antlr-interest] Re: keywords testLiteral=true
William Lam
xeenman at yahoo.com
Fri Feb 20 12:05:42 PST 2004
I changed my identifier rule to be protected, but my SLASH_IDENTIFIER
rule still absorbs up my keywords
IDENTIFIER options { testLiterals=true; }
: ID
;
protected
ID
: 'a' .. 'z' ( 'a' .. 'z' | '0' .. '9' | '_' | '$' | '#' )*
;
SLASH_IDENTIFIER options { testLiterals=true;}
: ('/' ID)+
;
--- In antlr-interest at yahoogroups.com, mzukowski at y... wrote:
> IDENTIFIER and DIVIDE have the same lookahead for IDENTIFIER. So
the lexer
> is choosing DIVIDE instead of IDENTIFIER. When DIVIDE calls
IDENTIFIER it
> overwrites the returned token type with its own.
>
> Best solution is to make IDENTIFIER protected and test for literals
in
> DIVIDE (and not in IDENTIFIER).
>
> Monty
>
> -----Original Message-----
> From: William Lam [mailto:xeenman at y...]
> Sent: Friday, February 20, 2004 11:29 AM
> To: antlr-interest at yahoogroups.com
> Subject: [antlr-interest] keywords testLiteral=true
>
> Hi all,
>
> In my lexer I have a rule SLASH_IDENTIFIER that calls my IDENTIFIER
> rule. However in the SLASH_IDENTIFIER rule, I get keywords that
come
> out of my IDENTIFIER rule. When I call the IDENTIFIER rule in the
> parser, I do not get keywords.
>
> for example,
>
> SLASH_IDENTIFIER matches this
> /foo/keyword
>
> however IDENTIFIER does not match
> keyword
>
> class SqlLexer extends Lexer;
>
> options {
> exportVocab = Sql;
> testLiterals = false;
> k = 2;
> caseSensitive = false;
> caseSensitiveLiterals = false;
> // charVocabulary = '\3' .. '\177';
> charVocabulary='\u0000'..'\uFFFE';
> }
>
> tokens {
> keyword = "keyword";
> }
>
> IDENTIFIER options { testLiterals=true; }
> : 'a' .. 'z' ( 'a' .. 'z' | '0' .. '9' | '_' | '$' | '#' )*
> ;
>
> DIVIDE
> : '/' { $setType(SLASH_IDENTIFIER ); }
> ( IDENTIFIER ( '/' IDENTIFIER )* )?
> ;
>
>
>
>
> Yahoo! Groups Links
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list