[antlr-interest] Lexer Strangeness

[Gavin Lambert]

At 23:30 14/02/2007, Shmuel Siegel wrote:
 >The way I look at this is:
 >In the lexer, '~' is a set operator. It says to use all of the
 >other symbols from the vocabulary. NEWLINE is neither a 
character
 >nor a set of characters; hence, set complement doesn't have any
 >meaning.

I can see the logic behind that position, but I can distinctly 
remember when writing a 2.7 grammar that I heartily wished that 
was not the case.  Ok, for complex lexer rules that's fair enough 
(but it should probably generate an error if it doesn't know how 
to treat it as a set), but for simple set-of-alternatives lexer 
rules it should be able to treat the two the same way.

This hasn't come up yet when using ANTLR 3, but then I've only 
been playing with a very simple grammar in v3 so far :)