[antlr-interest] Generating tree walkers.

Bharath Sundararaman bharath at starthis.com
Tue Jun 22 09:14:49 PDT 2004


Hi Grp,

I have the lexer, parser and an AST for a PASCAL-like grammar and I would
like to perform semantic analysis/symbol table checks/constant folding etc.
Is it a cumbersome task to write a tree walker for the abstract syntax tree
to perform these checks? When you perform semantic analysis on your input,
would you prefer to walk the tree or would you just embed it in your grammar
as actions?

Please let me know.

Bharath.

-----Original Message-----
From: Terence Parr [mailto:parrt at cs.usfca.edu] 
Sent: Sunday, June 20, 2004 2:50 PM
To: antlr-interest at yahoogroups.com
Subject: Re: [antlr-interest] unicode 16bit versus new 21bit stuff



On Jun 20, 2004, at 12:15 PM, John D. Mitchell wrote:
> Ah, I see.  So rather than having a passive lexer eating input and 
> providing it to the parser, the lexer is actually directed by the 
> parser.

Yep.  This makes it handle nasty things like the C++ template vs ">>" 
token problem simply disappear.  I.e., when lexing

List<List<int>> a;

you'll see that the nested template has ">>" in it.  The lexer, without 
context, cannot know which to pick.  Only the parser knows that it 
expects ">" followed by ">" not ">>" token. :)

The beauty of this is that the new lexer mechanism will deal with this 
easily.  Just create a new lexer rule for each parser context that is a 
set of alternatives.  The parser calls 
lexer.mySpecificContextTokenSet() rather than lexer.nextToken().  There 
are some issues such as ID vs literals and any other set of 
ambiguous/subset tokens, but I won't go into that at the moment.  
Whitespace can be tricky too.

This also seems to be a reasonable solution for keywords that can be 
variables.  When the parser must see it as a keyword, it asks for the 
keyword not an ID. :)

Woohoo!

This will be attacked after the core system is working as it is a 
direct extension and doesn't affect the underlying engine.

Ter
--
CS Professor & Grad Director, University of San Francisco Creator, ANTLR
Parser Generator, http://www.antlr.org Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again! Cofounder,
http://www.peerscope.com pure link sharing





 
Yahoo! Groups Links



 




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list