[antlr-interest] Serious Bug when using BitSetgeneration

[Geir Ove Skjaervik]

Hello,

Just one more comment on this Bug. It is ** clearly ** a bug, since
ANTLR does work correctly when BitSets are NOT used. I find it strange
that this Bug  seems to be passed in almost silence: I would think it to
be rather serious to have a "sleeping" bug that may come and bite you
any time you encounter the right conditions.

Terrence asked me to try to produce a small grammar example to try to
reproduce the problem. I will as soon as time on my project permits it. 

BUT I would think it is equally IMPORTANT for the ANTLR team to try to
hunt down this ** serious ** bug. It is nice that ANTLR 3.0 is under
development, but we sure need a strong ** core ** technology that won't
fail. 

I used several months to hunt down errors in my grammar, when it all
turned out to work OK as soon as I disabled the use of BitSets. However,
this clearly slows down the parsing!



Geir Ove

-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Geir Ove
Skjaervik
Sent: 7. november 2005 14:20
To: 'Terence Parr'
Cc: 'ANTLR Interest'
Subject: RE: [antlr-interest] Serious Bug when using
BitSetgeneration-Forgot


Hello,

I saw your reply first today. Maybe I do not understand your example,
but what give me problems when using BitSets are grammars like this:



class T extends Parser;

x : A|C|D|E|F|G|H;

s : a b ;

// here is the rule of interest
a : B
   | {p1}? ( )
   | {p2}? ( )
   |			Emtpty Production: Generates problems using
BitSets, no Problem when NOT using Bitsets
   ;

b : A|D|F|G|H;



Geir Ove

-----Original Message-----
From: Terence Parr [mailto:parrt at cs.usfca.edu] 
Sent: 6. november 2005 02:02
To: ANTLR Interest
Cc: Geir Ove Skjaervik
Subject: Re: [antlr-interest] Serious Bug when using
BitSetgeneration-Forgot


> Hopefully we can find a solution to this "Empty Pruction & BitSet 
> wrong code generation".
>
> It has kept my puzzling about errors in my grammar for a long long 
> time, but I have put off investigating it because there had been other

> more important things to look at. However, I know had to solve it, and

> it turned out to be a Bug !

Hi.  Here is my test case:

class T extends Parser;

x : A|C|D|E|F|G|H;

s : a b ;

// here is the rule of interest
a : B
   | {p1}? ( )
   | {p2}? ( )
   ;

b : A|D|F|G|H;

All is well.  It generates the following for rule a:

             if ((LA(1)==B)) {
                 match(B);
             }
             else if (((_tokenSet_1.member(LA(1))))&&(p1)) {
                 {
                 }
             }
             else if (((_tokenSet_1.member(LA(1))))&&(p2)) {
                 {
                 }
             }

clearly the bitsets are there.  Can you nail your bug down to a  
simple grammar like this that fails?

I realized that ANTLR will not (as I said in last email) delete the  
lookahead; it's smart enough to know that the predicates are only  
valid in the right context and so it keeps the lookahead the same.

Thanks,
Ter