[antlr-interest] Understanding priorities in lexing (newbie)

Gavin Lambert antlr at mirality.co.nz
Fri Jul 13 01:21:55 PDT 2007


At 19:59 13/07/2007, Kay Roepke wrote:
 >> In other words, the separation of lexer and parser is just
 >> something that you have to accept and is unlikely to ever
 >> change.
 >
 >Also, were the lexer dependent upon the parser you couldn't 
store
 >the tokens in an archive nor could you use a totally different
 >lexer (e.g. a handwritten one, output from another lexer 
generator).
 >
 >It's sometimes hard to draw the line between the two, especially 

 >for "weird" languages, where you sometimes feel the urge to 
somehow
 >direct the lexer from the parser, unfortunately. But keeping 
them
 >separate is a Good Thing(tm), IMHO.

Yep.  Definitely.

(I kinda prefer V2's style a little in that regard, since it made 
it more explicit that they were separate by not permitting 
combined grammars, thereby removing some of the potential for 
confusion.  But once I figured out the whole 
first-letter-uppercase bit I was happy with the V3 style 
too.  Though I tend to avoid using character literals in the 
parser because I prefer getting nice token names ;))



More information about the antlr-interest mailing list