[antlr-interest] TreeNodeStream has broken index() method
Philippe Faes
philippe at faes.net
Wed Dec 23 06:22:34 PST 2009
Dear all,
I've noticed that a CommonTreeNodeStream does not increment its index.
In other words, if I consume a token from the stream, then input.index()
remains 0.
This is bad for tree walkers, because the recover() method will think that
is encounters the same error twice, and skip over the token. See
BaseRecognizer.recover()
I think the problem is in LookaheadStream.remove().
int p is the index, but it gets reset.
public T remove() {
T o = get(0);
p++;
// have we hit end of buffer and not backtracking?
if ( p == data.size() && markDepth==0 ) {
// if so, it's an opportunity to start filling at index 0 again
clear(); // size goes to 0, but retains memory
}
return o;
}
Things work for me if I skip the clear(); instruction, but perhaps there is
a better solution that can be patched in the next official release.
kind regards
Philippe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20091223/4a7cb44b/attachment.html
More information about the antlr-interest
mailing list