[antlr-interest] Selection between lexer rule and literal

[togol machillan]

Thanks for the prompt reply. But defining tokens section in the lexer does not solve the problem. I would like the parser to match the occurence of COMPONENTS, for example, in the input file with the VALUE rule if it does not find any reference to the p_keyword_list rule in the rule currently being matched. The parser never matches COMPONENTS with the VALUE rule, although it is a valid token of the VALUE type. 

Regards,

Mach  

----- Original Message -----
From: "Alexey Demakov" <demakov at ispras.ru>
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Selection between lexer rule and literal
Date: Tue, 19 Apr 2005 15:06:04 +0400

> 
> ----- Original Message -----
> From: "togol machillan" <togolmach2 at lycos.com>
> To: <antlr-interest at antlr.org>
> Sent: Tuesday, April 19, 2005 2:54 PM
> Subject: [antlr-interest] Selection between lexer rule and literal
> 
> 
> > I have a rule in the lexer which looks like the following.
> >
> > VALUE  options {testLiterals = true; }
> >        :CAPITAL_LETTER (CAPITAL_LETTER|DIGIT|','|':'|'+'|'-')*
> >        ;
> >
> > In the parser, I have defined a list of literals like the 
> > following example rule
> >
> > p_keyword_list returns [ANTLR_USE_NAMESPACE(std)string k]
> >   :
> >    (
> >    {k = LT(1)->getText(); col = LT(1)->getColumn();} "COMPONENTS"
> >    |{k = LT(1)->getText(); col = LT(1)->getColumn();} "DATABANKS"
> >    |{k = LT(1)->getText(); col = LT(1)->getColumn();} "PROP-SOURCES"
> >    )
> >                           ;
> >
> > Now when some parser rule is looking for a VALUE token and it 
> > sees, for example, COMPONENTS, it returns an unexpected token 
> > error.
> It there some way of making the parser intelligent enough so that 
> it considers COMPONENTS (defined as a literal) as a token of type
> VALUE.
> 
> Try to define all tokens in lexer. For keywords there is "tokens" section
> as described in 
> http://marlboro/docs/cc/antlr-2.7.4/metalang.html#TokensSection
> So, your grammar should look like:
> 
> In lexer:
> 
> tokens
> {
>      COMPONENTS = "COMPONENTS";
>      DATABANKS = "DATABANKS";
>      PROP_SOURCES = "PROP-SOURCES";
> }
> 
> In parser:
> 
> p_keyword_list returns [ANTLR_USE_NAMESPACE(std)string k]
> :
>    {k = LT(1)->getText(); col = LT(1)->getColumn();}
>    (  COMPONENTS
>     | DATABANKS
>     | PROP_SOURCES
>    )
> ;
> 
> Regards,
> Alexey
> 
> -----
> Alexey Demakov
> TreeDL: Tree Description Language: http://treedl.sourceforge.net
> RedVerst Group: http://www.unitesk.com

-- 
_______________________________________________
NEW! Lycos Dating Search. The only place to search multiple dating sites at once.
http://datingsearch.lycos.com