[antlr-interest] Re: A simple prepocessor with a lexer?

maaxxxcal maaxxxcal at yahoo.com
Tue Oct 7 17:36:27 PDT 2003


Hummm ok, I promise this is the last post on that topic. I realize 
that maybe I should have been more clear about what I was trying to 
achieve from the beginning.
So the class you're talking about is LexerSharedInputState, but I'm 
not sure it would do for me because I don't know when to stop the 
inner Lexer. Let me explain.
One of the effect of my pragma is to define what the statement 
separator is going to be for the next portion of code. I already have 
a lexer and parser that know how to parse a single statement, with no 
separator. Now what I'm trying to do is to wite a lexer that will 
process my script, understand the pragmas and cut statements to feed 
them to my single-statement lexer/parser.

Here's the structure that the scripts I have to parse can have:
-- PRAGMA separator=@
-- PRAGMA error=on
<statement>@
<statement>@
-- PRAGMA error=off
-- PRAGMA separator=%
<statement>%
<statement>%

What's interesting in this problem is that an important part of the 
syntax (the statement separator) is only defined at runtime. I 
believe there is no way to do that with only 1 lexer/parser with 
ANTLR because it has to be able to predict statically what the 
alternatives are.

So what strategy would you use?
Thanks again for your help!!

> Oh that's different.  Simplest thing to do is create another lexer 
that 
> handles that text and then, using a SharedLexerInputState (or 
whatever 
> it's called), create an instance of the new lexer in the outer 
lexer 
> and then make a while loop around it's nextToken().
> 
> You can grep for that class name in the examples probably.
> 
> Ter
> --
> Professor Comp. Sci., University of San Francisco
> Creator, ANTLR Parser Generator, http://www.antlr.org
> Co-founder, http://www.jguru.com
> Co-founder, http://www.knowspam.net enjoy email again!
> Co-founder, http://www.peerscope.com pure link sharing


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 




More information about the antlr-interest mailing list