[antlr-interest] Controlling Lexer from Parser
David-Sarah Hopwood
david-sarah at jacaranda.org
Thu Dec 3 10:12:06 PST 2009
Gokulakannan Somasundaram wrote:
> Hi,
> I have seen some relevant articles in the FAQ. But i want to know,
> whether the following approach will always work for me.
>
> I am trying to parse a SQL grammar, in which the SQL Keywords are sometime
> allowed as table names / column names.
> a) Say when i am expecting a table_name /column name from parser, i set a
> global variable called x.
> b) i check this x to set the token type of that particular token.
>
> This will succeed only if the parser completes executing the parsing actions
> before trying to make tokens out of the inputstream. Is it always the case
> with ANTLR?
No; unless you use a different TokenStream class, the whole stream will be
lexed before any token is parsed.
In any case, lexer->parser feedback is almost certainly not the best way to
solve your problem above. (Lexer->parser feedback is horribly complicated,
in any parser generator but especially in ANTLR, and should be used only as
a last resort in my experience.)
To solve the problem that keywords can sometimes be used as identifiers,
define a parser rule 'keyword' whose alternatives are all of the keywords,
and then define
id : ID | keyword ;
This might introduce some ambiguities depending on the language you're
parsing, but you can usually solve those using predicates.
If you want to change the token type of a keyword to ID when it is used
as an identifier, then define
id : ID
| k=keyword { if ($k.tree != null) $k.tree.getToken().setType(ID); }
;
--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 292 bytes
Desc: OpenPGP digital signature
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091203/cc2112ba/attachment.bin
More information about the antlr-interest
mailing list