[antlr-interest] NullPointerException in parser bug

Mike Gadsdon m.gadsdon at rheagroup.com
Thu Mar 24 01:51:58 PST 2005


Hi Ter,

>> Yeah, LA(i) should return -1...hmm...perhaps 2.7.5 has a fix for
>>this...can't remember.

I just re-ran tests with 2.7.5 and I see the same problem. As far as I can
see the LA method is just not built to handle nulls in the queue.

With further thought, the problem should go away if we ensure nulls never
get in the queue. I modified the fill method in TokenBuffer as below to
insert a token type 1 (I presume this represents EOF) into the circular
buffer if a null is pulled in from the input stream. This solves the problem
I'm having, and I can't see that such a change should have a negative effect
anywhere. It only makes a difference when we are beyond the end of file
anyway.

    private final void fill(int amount) throws TokenStreamException {
        syncConsume();
        // Fill the buffer sufficiently to hold needed tokens
        while (queue.nbrEntries < amount + markerOffset) {
            // Append the next token, ensuring null does not end up in the
queue
        	Token tok = input.nextToken();
        	if (tok == null) {tok = new Token(1);}
            queue.append(tok);
        }
    }

Cheers

Mike
==============================================================
Mike Gadsdon                   mailto:m.gadsdon at rheagroup.com
Rhea System S.A                Tel +44 (0)20 8891 0702
                               Fax +44 (0)20 8891 6702
==============================================================



More information about the antlr-interest mailing list