[antlr-interest] Simple Grammar breaks ANTLRWorks Interpreter & Debugger?

David-Sarah Hopwood david-sarah at jacaranda.org
Fri Aug 14 15:49:58 PDT 2009


Jim Idle wrote:
> David-Sarah Hopwood wrote:
>> Gavin Lambert wrote:
>>> At 10:12 14/08/2009, consiliens at gmail.com wrote:
>>>  >QuizLexer lexer = new QuizLexer(new ANTLRFileStream(input));
>>>  >CommonTokenStream tokens = new CommonTokenStream(lexer);
>>>  >// prints 0
>>>  >System.out.println(tokens.size());
>>>  >
>>>  >So according to this there are no tokens, which would explain 
>>>  >the blank debugger Input window. If the ANTLRWorks Interpreter runs
>>>  >the code correctly, why not the Debugger or unit tests?
>>>
>>> Actually that's a bit deceptive :)
>>>
>>> The stream starts out empty, and thus .size() returns 0.
>>
>> You're right, and that seems like a bug. In CommonTokenStream:
>>   
> No - you need to calls something like LT(1) first - otherwise all methods have 
> to check for init which is an overhead that isn't really needed.

Except that all public methods except .size() and .consume() already do
(directly or indirectly) perform this check. That is, almost all of the
overhead is being paid already -- and is needed, since the generated
parser code doesn't ensure that it calls LT, or any other single method,
first.

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com



More information about the antlr-interest mailing list