[antlr-interest] Basic predicate question re: lexer
Dukie Banderjee
dukie_banderjee at hotmail.com
Sun Jun 14 18:09:56 PDT 2009
Hi,
I'm working on a parser for a file format that can contain text and delimiters. One of the delimiters is a ':', and you can escape the delimiter by following it with a '?' such as ':?'.
I'd like to have the lexer consider the ':?' as part of the TEXT token, and ':' match the SEPARATOR token. So I tried this:
file : contents+ EOF ;
contents : TEXT
| CSEP
;
CSEP: ':';
TEXT: ('a'..'z'|'A'..'Z'|'0'..'9'|' '|'-'|','|'.'|'/'|COLON)+ ;
fragment COLON: ':?' ;
This didn't work, as the input "hello:" caused an error. I guess it was expecting to continue a TEXT token (next char would be '?'), and met with an EOF instead.
I imagine there's a way to do this, perhaps with predicates, but I'm not experienced enough to see the obvious solution. Can anyone help?
Thanks,
Rob
_________________________________________________________________
Create a cool, new character for your Windows Live™ Messenger.
http://go.microsoft.com/?linkid=9656621
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090614/40345f16/attachment.html
More information about the antlr-interest
mailing list