[antlr-interest] Lexer Predicates?

[Gavin Lambert]

At 06:34 4/08/2008, Foust wrote:
 >Yes... it started out that way. But to allow spaces to be part
 >of a config value (read up to EOL), the Lexer needs to honor
 >state. (Place spaces in the HIDDEN channel for all other cases
 >- outside of a special config/preprocessor rule).

Are you hiding the EOLs as well?  (Usually they're lumped in with 
whitespace.)

If so, then you'll have to match everything in the lexer anyway, 
since the parser won't be able to see the EOL.  As long as you 
have something fairly distinctive to start matching on, this 
shouldn't be hard, and you shouldn't need to do any parser->lexer 
contortions.  See how line and block comments are implemented in 
the examples.

(And if you can modify the language you're parsing, now would be a 
good time to make it use a quoted string or similar instead of 
simply reading to EOL.)