[antlr-interest] Special filtering in a lexer

Corno Schraverus corno.schraverus at isd.nl
Wed Jan 17 09:48:17 PST 2007


Hi all,
 
I want to make a lexer that splits a tokenstream up into known tokens and all the text that is in between 2 know tokens.
Say 'Foo' and 'Bar' are know tokens, then the following string;
 
bla bla bla Foo bla Bar xxx
 
should result in 5 tokens with the respective types and values;
 
OTHER "bla bla bla "
FOO "Foo"
OTHER " bla "
BAR "Bar"
OTHER " xxx"
 
Should I use filter=RULE and create a token manually in that rule or is there a better way to realize this?
 
TIA,
 
Corno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070117/ea941ab8/attachment.html 


More information about the antlr-interest mailing list