[antlr-interest] Understanding priorities in lexing (newbie)

Kay Roepke kroepke at classdump.org
Fri Jul 13 00:37:05 PDT 2007


On Jul 13, 2007, at 2:05 AM, Wincent Colaiuta wrote:

> One thing to bear in mind is that lexing and parsing are completely  
> separate phases in ANTLR; sure the parser and lexer run at the same  
> time because the parser is just saying "give me a token, give me  
> another token" etc until all tokens are produced, but conceptually  
> because there is no communication from the parser to the lexer you  
> can think of them as two completely separate phases.

Actually, once you use a TokenStream (at least the ones that come  
with the Java and Objective-C targets, dunno about the others), they  
are separate phases even at runtime. That's because a TokenStream  
buffers up all tokens before the parser even gets a chance to say  
nextToken().
Sure, that's an implementation detail but it may matter to some (i.e.  
reading the textual input from a stream, memory considerations, etc.)

cheers,
-k
-- 
Kay Röpke
http://classdump.org/






More information about the antlr-interest mailing list