[antlr-interest] Writing a two pass compiler.

Dean Tribble tribble at deantribble.com
Sat Jan 22 12:57:54 PST 2005

Depending on the grammar, you may be able to use Antlr's support for 
switching parsers and/or grammars in mid parse.  I made use of that in the 
parser for the E grammar.  You can find a version (not complete) in 
http://www.erights.org/e-impls/e-on-e/egrammar/index.html.  In particular, 
the various constructs beginning with "quasi" in the e.g file connect to 
the quasi.g file, which is a subgrammar.  The QUASIOPEN item in the lexer 
changes the active lexer, and the corresponding token in the QuasiString 
construct in the parser changes the parser.  Note that this is all based 
on the standard approach within ANTLR for combining multiple grammars, 
with the primary exception that antlr does not provide sufficient support 
for *popping* out of a nested grammar.  The result was that I had to 
add parser behavior to pop teh nested grammar.

I have a newer, not yet posted, version that ended up simplifying this 
such that I no longer use a separate parser for the sub-language.  I also 
made subclasses of the lexer and parser types to make managing the lexer 
selection support a little easier.  It should be done enough in a week or 
so, but email if it owuld be useful sooner than that.

On Sat, 22 Jan 2005, Monty Zukowski wrote:
> 	From: 	  monty at codetransform.com
> 	Subject: 	Re: [antlr-interest] Writing a two pass compiler.
> 	Date: 	January 21, 2005 8:21:28 AM PST
> 	To: 	  olczyk2002 at yahoo.com
> \On Jan 20, 2005, at 8:47 PM, Thaddeus Olczyk wrote:
> > I have a grammar for a language which I discover is
> > written quite poorly. Certain types of strings can
> > have different meanings depending on context, so
> > I cannot unambiguously tokenise the strings.
> > Mainly this is do to the fact that there is
> > no way to distinguish text data from "source".
> >
> > Having examined the language closely, I believe it
> > can be parsed using antlr to generate a two pass
> > compiler. Pass2 being a context sensitive parse.
> >
> > A difficuly for me is that I've only written a few
> > very trivial parsers ( at the calcualtor level ). I
> > was wondering if there were any simple samples of
> > antlr being used to implement a two pass compiler to
> > guide
> > me along the way.
> >
> I'm not aware of any full blown examples of two pass parsing.  I do 
> have a small example of something similar, where I embed another parser 
> in a TokenFilter in order to disambiguate strings, numbers and labels 
> based on things like whether they follow a GOTO or not.  Read my 
> article at http://www.codetransform.com/filterexample.html and then 
> post any questions here.  Perhaps you will have some tricky examples 
> you will need help with.  If so please post some sample code to be 
> parsed as well as your best guess for the rules involved.
> Monty Zukowski
> ANTLR & Java Consultant -- http://www.codetransform.com
> ANSI C/GCC transformation toolkit -- 
> http://www.codetransform.com/gcc.html
> Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html

More information about the antlr-interest mailing list