[antlr-interest] Aggressive vs. Lazy Buffering in ANTLR

David Holroyd dave at badgers-in-foil.co.uk
Tue Dec 19 09:19:29 PST 2006


On Tue, Dec 19, 2006 at 03:59:47PM +0000, Foolish Ewe wrote:
> Hi All:
> 
> Thanks for the help (especiallly Ter), in a previous e-mail, I suggested 
> that aggressive buffering in
> in ANTLRInputStream made ANTLR problematic for parsing interactive input, 
> since the
> ANTRLInputStream constructor blocks on reading all input until EOF into a 
> buffer.
> 
> However, I tried making a simple input method that uses Lazy buffering 
> (i.e. reads characters
> on demand extending the buffer).  This lazy input stream no longer blocks 
> on the constructor,
> however, now I've noticed that the CommonTokenStream constructor appears to 
> block.
> It appears to be blocking in fillBuffer, but I don't see an explicit call 
> in the constructor.
> The impact of this is that it negates the interactive support that the lazy 
> buffering of my
> home-grown input stream class, so I'd like to fix it.
> 
> Is there a lazy buffering (buffer as input becomes available/on demand) 
> variant of TokenStream
> support or have I made some error?

I've got this one you can use,

  http://svn.badgers-in-foil.co.uk/metaas/trunk/src/main/java/uk/co/badgersinfoil/metaas/impl/antlr/LinkedListTokenStream.java

It happens to also create a doubly-linked list of Tokens, but you can
just strip that out, I expect.

ta,
dave

-- 
http://david.holroyd.me.uk/


More information about the antlr-interest mailing list