[antlr-interest] Controlling Lexer from Parser

[Gokulakannan Somasundaram]

Thanks, I actually realised that it is not the best way to achieve the
desired one.

On Thu, Dec 3, 2009 at 11:42 PM, David-Sarah Hopwood <
david-sarah at jacaranda.org> wrote:

> Gokulakannan Somasundaram wrote:
> > Hi,
> >    I have seen some relevant articles in the FAQ. But i want to know,
> > whether the following approach will always work for me.
> >
> > I am trying to parse a SQL grammar, in which the SQL Keywords are
> sometime
> > allowed as table names / column names.
> > a)  Say when i am expecting a table_name /column name from parser, i set
> a
> > global variable called x.
> > b) i check this x to set the token type of that particular token.
> >
> > This will succeed only if the parser completes executing the parsing
> actions
> > before trying to make tokens out of the inputstream. Is it always the
> case
> > with ANTLR?
>
> No; unless you use a different TokenStream class, the whole stream will be
> lexed before any token is parsed.
>
> In any case, lexer->parser feedback is almost certainly not the best way to
> solve your problem above. (Lexer->parser feedback is horribly complicated,
> in any parser generator but especially in ANTLR, and should be used only as
> a last resort in my experience.)
>
> To solve the problem that keywords can sometimes be used as identifiers,
> define a parser rule 'keyword' whose alternatives are all of the keywords,
> and then define
>
>  id : ID | keyword ;
>
> This might introduce some ambiguities depending on the language you're
> parsing, but you can usually solve those using predicates.
>
> If you want to change the token type of a keyword to ID when it is used
> as an identifier, then define
>
>  id : ID
>     | k=keyword { if ($k.tree != null) $k.tree.getToken().setType(ID); }
>     ;
>
> --
> David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com
>
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091204/1f600177/attachment.html