[antlr-interest] Lazy load of CommonTokenStream??

[Jim Idle]

On Sun, 2008-08-17 at 21:01 +0200, Kay Röpke wrote:

> On Aug 17, 2008, at 9:50 PM, Vitaliy wrote:
> 
> > I've encountered some really interesting behavior of the  
> > CommonTokenStream:
> >
> > Right after the constructor of the parser, I'm trying to use the  
> > method Size() of CommonTokenStream, to retrieve the number of tokens  
> > in the input.
> > It would return 0, no matter how many tokens actually exist in the  
> > input.
> > BUT funny enough, if I call before that to the method LA(1), then   
> > everything works fine.
> 
> CommonTokenStream buffers all tokens up when it is first used.  
> Unfortunately, size() does not trigger filling the buffer which seems  
> like a bug. I see no reason why it shouldn't fill the buffer, that is.


I agree. In fact this might well be more intuitive, however we should
think about any performance impact as this might be the reason it does
not do this - in the C runtime I can take care of this by flipping
functions after the first call, but you can't hack that in Java.

Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080817/d250c2a1/attachment.html