[antlr-interest] ANTLR 3: Problem with static DFA class generation

[Terence Parr]

On Sep 9, 2005, at 9:12 AM, Oliver Zeigermann wrote:

> Folks,
>
> I have a problem with code looking like this:
>
> lexer grammar XMLLexer;
> {
>     boolean tagMode = false;
> }
>
> TAG_START_OPEN : { !tagMode }? '<' { tagMode = true; } ;
> TAG_END_OPEN : { !tagMode }? "</" { tagMode = true; } ;

Yes, this is intended behavior though not desirable!  I spent hours  
trying (twice!) to come up with a convenient and efficient means to  
passing the "this" pointer around in the DFAs so that they would not  
be static.  I can't remember the issues now.  Originally, it was  
because I had separated the cyclic DFAs into bytecodes rather than  
java source code.

No matter what, if you have a ref to a parameter in a predicate that  
will end up in a cyclic DFA, it will not be visible!  There is  
absolutely no way around this.  You can generate an arbitrary DFA in  
java code without using ptrs to objects.  Many people have try to  
show me some try-finally stuff that will do arbitrary gotos but I  
always show a backwards jump that isn't possible.

I'm very open to suggestions, but at the moment I'll be forced to  
define the semantics of predicates to not allow parameters (even  
though they will work for non-cyclic DFAs).  This is highly  
undesirable!  Heh, wait!  What if I computed the value first and then  
passed to the DFA!!!!???  THe value cannot change during the DFA  
execution so the value would be the same!  So I would compute the  
predicate(s) first and then pass a vector of results to the DFA.   
This means that for cyclic DFAs, all predicates are evaluated (mostly  
unnecessarily) but they are not allowed to have side-effects so this  
is ok.

Interesting!  The following rule needs a cyclic DFA to see past the  
DOT*:

a[int x] : {x!=0}? DOT* ID | {x>=0}? DOT* SEMI ;

I would generate a DFA that looked like this:

boolean[] pred34 = {x!=0, x>=0};
int alt34 = DFA34.predict(input, pred34);

Holy crap!  That might work. :)

Actually, don't forget though that hoisting makes parameters unusable  
also.

a[int x] : {x!=0}? ID ;

If {x!=0}? gets hoisted into another decision it won't compile.   
Officially then only values visible to the class are valid, though in  
most cases even parameters would work if I can figure out the pre- 
evaluation thing.

Adding to the "blog":

http://www.antlr.org/blog/antlr3/lookahead.tml

[yes, I still need to convert my TML "blog" to the antlr wiki]

Ter
--
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com