[antlr-interest] Dealing with issues in non-greedy (.)* rules

[Jim Idle]

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Marc-André Laverdière
> Sent: Wednesday, June 13, 2007 11:08 AM
> To: Gavin Lambert
> Cc: ANTLR Interest
> Subject: Re: [antlr-interest] Dealing with issues in non-greedy (.)*
> rules
> 
> I'm not sure I understand what to do... what's a final lexer rule?

When you have specified all other tokens, if you still have input sets that are not completely covered by the lexer tokens you have defined, you can add, as the last rule for your lexer:

UNKNOWN : . ;  // Your token name of course

You can then skip those characters, or you can have your own error message and so on. Otherwise you get a lexer "Mismatched character", which isn’t much use. You can override the lexer error processor too but it seems more natural (to me at least) to have a lexer that covers all possible character input, so you can see what happens to it by inspecting the grammar).

Jim