[antlr-interest] unquoted strings in small line-based language
Gavin Lambert
antlr at mirality.co.nz
Wed Oct 17 23:46:20 PDT 2007
At 16:42 18/10/2007, John Pye wrote:
>How can I make the lexer *not* peer into this free-text and
attempt
>to identify tokens in it? How would I then catch the resulting
>string in my parser?
The only way to do this is to do all the work in the
lexer. Remember that lexing is a completely independent step that
occurs before parsing, so the parser cannot "inform" the lexer
about any context.
Something like this would probably do the trick:
CASE
: 'CASE' WSChar+ INTEGER WSChar+ (~'\n')*
;
fragment WSChar : ' ' | '\t';
WS : WSChar { $channel = HIDDEN; };
This will match the whole CASE line as a single token, which isn't
ideal. You could probably improve this by using the technique
discussed on the Wiki for making a lexer rule output more than one
token.
More information about the antlr-interest
mailing list