[antlr-interest] skipping tokens

[Terence Parr]

So, should lexer rules do like v2 and set "token = Token.SKIP;" in an  
action so that a token is ignored and another is attempted?  What  
about a method call "skip();" or something?  The method call is more  
flexible, but what if some targets can only skip by setting a local  
var or something in the lexer rule?  The skip() method would not be  
able to alter them.

Perhaps we use skip() as the default action and languages that must  
set locals can force v2 style assignments.  Thoughts?

Ter