[antlr-interest] Special filtering in a lexer
Corno Schraverus
corno.schraverus at isd.nl
Wed Jan 17 09:48:17 PST 2007
Hi all,
I want to make a lexer that splits a tokenstream up into known tokens and all the text that is in between 2 know tokens.
Say 'Foo' and 'Bar' are know tokens, then the following string;
bla bla bla Foo bla Bar xxx
should result in 5 tokens with the respective types and values;
OTHER "bla bla bla "
FOO "Foo"
OTHER " bla "
BAR "Bar"
OTHER " xxx"
Should I use filter=RULE and create a token manually in that rule or is there a better way to realize this?
TIA,
Corno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070117/ea941ab8/attachment.html
More information about the antlr-interest
mailing list