[antlr-interest] lexer vs. grammer

Pavel Grinfeld pg at freeboundaries.com
Tue Jun 22 07:18:04 PDT 2010


Is it always clear cut whether to call something a token or make it part 
of grammar.

For example, suppose we are parsing TeX and a command is something that 
looks like this:


Should "command" be in the lexer section or grammar section?

Sorry for such a vague question, but I am just starting out...



More information about the antlr-interest mailing list