[antlr-interest] Antlr too eager? Writing a template engine.

Oliver Christensen olivercsa at gmail.com
Mon May 18 14:37:08 PDT 2009


Hi,

I've run into a problem that despite my best attempts over the last couple
of days, i can't solve myself. Thus, i'm asking for your help. I'm gonna
give you a bit of context first, and then a clearly stated problem
definition in the end (you can skip there if you want).

I'm writing a template language/engine like the one in the popular Django
framework for C#. I'm doing a few things differently; number one is that i
want other developers to be able to extend the language with new tags (for
instance {print 2+2}) simply by writing classes in c#, and not touching the
antlr grammer. Their code to parse a tag could then look like this:

bool isPrint = tag.readIdentifier() == 'print';
Expression expr = tag.readExpression();

This works by taking giving the string value of the tag ("print 2+2" in the
example) to antlr, and then calling one method at a time, to "chop off" the
pieces as needed.
:
I ran into problems when i wanted to implement the {for ...} tag which i
want to have syntax like this: {for item in [expression] [options]};
example: {for user in users.all reversed=true}. Which is where i ran into my
problem. The code to chop up those parts looks almost like this:

tag.readIdentifier(); // gets "for"
tag.readIdentifier(); // gets "user"
tag.readIdentifier(); // gets "in"
tag.readExpression(); // *should* get: "users.all"
tag.readAssign(); // *should* get: "reversed=true"

However, antlr dosen't give me the expression when an assignment follows it.
It simply fails to parse the expression. I think that's wierd because given
an input such as (psudo code) "2 2".readExpression() it'll happly ignore the
second "2" and just return the first one. Given "2+2 10".readExpression(),
it'll return the tree for 2+2". It even works as wanted with "2+2
reversed=true".readExpression().

The problem appears when the first token is an ID followed by an expression,
like "users reversed=true". I have what i belive to be a very simple tree
grammer, which i'd be happy to share, but here i'll just present a
simplified version which highlights my problem:

PROBLEM: Given the following grammer, how do i make it so that when given
the input "users reversed=true" the expression rule will only match the
first "users", so that i can use the assignment rule after to pull out the
assignment tree.

------------ test.g ----------
grammar test;
options{output = AST;ASTLabelType = CommonTree;language  =  CSharp;}

expression : value | assignment;
assignment : ID '=' expression;
value    : ID | INT;

ID      : ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'_'|'0'..'9')* ;
INT     : ('0'..'9')+;
WS    : ('\r'|'\n'|' '|'\t')+ {Skip();};
--------------------------------

I've been stuck on this for days, and i'm hoping somebody in the know will
help me.

Thank you!
   Oliver
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090518/101275b7/attachment.html 


More information about the antlr-interest mailing list