[antlr-interest] Understanding priorities in lexing (newbie)
Kay Roepke
kroepke at classdump.org
Fri Jul 13 00:37:05 PDT 2007
On Jul 13, 2007, at 2:05 AM, Wincent Colaiuta wrote:
> One thing to bear in mind is that lexing and parsing are completely
> separate phases in ANTLR; sure the parser and lexer run at the same
> time because the parser is just saying "give me a token, give me
> another token" etc until all tokens are produced, but conceptually
> because there is no communication from the parser to the lexer you
> can think of them as two completely separate phases.
Actually, once you use a TokenStream (at least the ones that come
with the Java and Objective-C targets, dunno about the others), they
are separate phases even at runtime. That's because a TokenStream
buffers up all tokens before the parser even gets a chance to say
nextToken().
Sure, that's an implementation detail but it may matter to some (i.e.
reading the textual input from a stream, memory considerations, etc.)
cheers,
-k
--
Kay Röpke
http://classdump.org/
More information about the antlr-interest
mailing list