[antlr-interest] Aggressive vs. Lazy Buffering in ANTLR
Foolish Ewe
foolishewe at hotmail.com
Tue Dec 19 10:05:58 PST 2006
Hi Kay:
Thanks to you for the hints. I'm somewhat restricted in which code I can
publish without
managerial/legal approval. I have modified the Expr example from the wiki
to have some
additional debugging output and created a "MyLazyInputStream" class. The
constructor
for this new class is pretty simple, it does an assignment of the input
stream and allocates
storage to buffer input and does little else (the class' stream position
pointers, etc.
all use compile time initialization).
Since my last post, I've taken a closer look at my code and discovered that
I
had statements of the form
CommonTokenStream tokens = new CommonTokenStream(lexer);
System.out.println("After TokenStream Constructor, tokens = " +
tokens + ">");
Instead of dumping the reference address, this appaeared to somehow invoke
fillbuffer,
when I changed System.out.println to no longer try to print the tokens
object/reference,
then controll passed into the parser which I think then blocks on
tokens.fillbuffer();
I removed the dumps of the tokens and parser objects, now the software hangs
so that
the Eclipse debugger can't break it in the parser.prog (unusual because I
was able to do this
with all previous examples).
I still suspect that the tokens.fillbuffer() method aggressively buffers
tokens, but I'll
need to check it more carefully to confirm this, since I'm not sure of why I
get no output
when I enter
a = 3 + 4
In the Eclipse debugger (I don't know of anyway to get Eclipse to pass an
EOF on input, it
appears to filter out control D).
Regards:
Bill M.
>From: Kay Roepke <kroepke at classdump.org>
>To: Foolish Ewe <foolishewe at hotmail.com>
>CC: antlr-interest at antlr.org
>Subject: Re: [antlr-interest] Aggressive vs. Lazy Buffering in ANTLR
>Date: Tue, 19 Dec 2006 17:32:28 +0100
>
>
>On 19. Dec 2006, at 16:59 , Foolish Ewe wrote:
>
>>However, I tried making a simple input method that uses Lazy buffering
>>(i.e. reads characters
>>on demand extending the buffer). This lazy input stream no longer blocks
>>on the constructor,
>>however, now I've noticed that the CommonTokenStream constructor appears
>>to block.
>
>What do you do in your input stream's constructor? The CommonTokenStream
>constructors shouldn't block,
>they just set the members.
>
>>It appears to be blocking in fillBuffer, but I don't see an explicit call
>>in the constructor.
>
>fillBuffer is invoked on the first call to LT. Maybe you're seeing that?
>Though it doesn't sound
>likely based on what you described...
>
>>The impact of this is that it negates the interactive support that the
>>lazy buffering of my
>>home-grown input stream class, so I'd like to fix it.
>>
>>Is there a lazy buffering (buffer as input becomes available/on demand)
>>variant of TokenStream
>>support or have I made some error?
>
>No there's nothing like that in the standard distribution. Could you post
>some code?
>
>cheers,
>
>-k
>
>--
>Kay Röpke
>http://classdump.org/
>
>
>
>
_________________________________________________________________
Get free, personalized online radio with MSN Radio powered by Pandora
http://radio.msn.com/?icid=T002MSN03A07001
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Test.java
Type: application/octet-stream
Size: 1035 bytes
Desc: not available
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20061219/7e64b509/attachment.obj
More information about the antlr-interest
mailing list