[antlr-interest] Lazy load of CommonTokenStream??

[Jim Idle]

On Sun, 2008-08-17 at 21:44 +0200, Kay Röpke wrote:

> On Aug 17, 2008, at 9:13 PM, Jim Idle wrote:
> 
> >>
> >> CommonTokenStream buffers all tokens up when it is first used.
> >> Unfortunately, size() does not trigger filling the buffer which seems
> >> like a bug. I see no reason why it shouldn't fill the buffer, that  
> >> is.
> >
> > I agree. In fact this might well be more intuitive, however we should
> > think about any performance impact as this might be the reason it does
> > not do this - in the C runtime I can take care of this by flipping
> > functions after the first call, but you can't hack that in Java.
> 
> 
> Not sure I understand what you are getting at.
> fillBuffer() will set p = 0 when it is first called, thus getting the  
> size of the tokens calls fillBuffer() only once.
> Actually all other methods that access tokens (with the notable  
> exception of get()) perform this check, too, so they all "suffer" from  
> that if.


Right - so, if size is called a lot then you will add the if to that
too. I think that internally size won't be called before something else
has already called fillBuffer(). So my point is that we should look at
all the paths and cover all the runtime possibilities, then see if it
makes sense to have the if in size() - as in, can it be added there
without increasing the overall checking done for if (p==0). I found the
call to size() to be significant enough that I wanted to optimize it,
and while a check for an integer == 0 is quick, it is still overhead.



JIm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080817/882e033d/attachment.html