[antlr-interest] Patch for filter mode (+1 Vote!)

Mark Venbrux mark.venbrux at gmail.com
Thu Jun 14 13:00:37 PDT 2007


Hi,
A while ago, the following message was posted:
http://www.antlr.org/pipermail/antlr-interest/2007-June/021243.html
It was very valuable to me and enables me to manipulate source files
in an elegant way.
Thanks a lot Wincent!
But it seems that it was not picked up at all.
So I just add a vote here for adding the "filter patch" to AntLR, so
that I can remove my local patch.

Most important part of the well elaborated message:
""
I'm aware that the filter mode was intended to enable the creation of
"fuzzy" lexers but I've also found it very useful for parsing things
like wikitext or templating languages (PHP or any like it) where you
have a large amount of free-form text (no special markup) studded
with meaningful chunks of a more formal language (wikitext
directives, PHP code sections etc). In this case you don't want to
filter out and throw away the dross; you want to keep it.

Without filtering mode it is very hard to write a lexer for this kind
of input, yet with filtering mode as it currently is implemented you
can't really use a parser either. For one thing, backtracking gets
turned on in the parser whether you want it or not, and much more
crucially any actions which you define or @after blocks which you set-
up will never be executed (although @init blocks will be); there may
be other issues as well but those are the ones I'm aware of.

Conversely, trying to write the lexer without filtering mode turned
on is fiendishly difficult.

""

-- 
Mvg,
    Mark


More information about the antlr-interest mailing list