[antlr-interest] Processing filter-ed lexer tokens with ANTLR parser?

Hendy Irawan gauldong at gmail.com
Wed Mar 30 05:42:17 PST 2005


Hi!

I'm currently writing a template engine, which transforms HTML
templates into C# classes.

I currently uses both the ANTLR parser and lexer for this task. I
define several tokens and the LITERAL token, which is "." (which
matches anything). This generates warnings, but it actually "works"
because I prefer other tokens, LITERAL last.

The problem lies when a non-LITERAL matches, then it doesn't match in
the middle (usually a template syntax error, but not always). I want
the template grammar to be as loose as possible (syntax errors later,
or even ignore these syntax errors), and treat any syntax errors as
LITERALs.

I tried using the "filter" option, which seems to be exactly what I
want, but it doesn't allow me to return the token (it seems to always
discard the token, although it allows me to have an action, basically
anything but returning a token). What I want is when something went
wrong (i.e. "no match"), whatever it is, it will get labeled as a
LITERAL then returned to the parser as a "normal" token.

Is this possible?

P.S.: For Terence, thanks a lot for answering my questions... It felt good. :-)
-- 
Hendy Irawan - http://dev.gauldong.net - GaulDong Developer Center


More information about the antlr-interest mailing list