[antlr-interest] Embedding expressions in plain text -- how to?

[Austin Hastings]

Create an "outer" lexer that tokenizes your input as STRING PROGRAM 
STRING ...

fragment OpenBrace : '${' ;
fragment CloseBrace: '}';

STRING:  ( ~('$') | ('$' ~('{')))+ ;
PROGRAM: OpenBrace .* CloseBrace ;


Then parse the "PROGRAM" text elements as you like.

Note that if you want nested curly braces, you'll have to implement that 
via a nested fragment that recurses on itself.

=Austin


Rob Greene wrote:
> How should I go about embedding an expression language in plain text.  
> The expression is wrapped in '${' and '}' but may occur zero to many 
> times within plain text.  When in plain text, I want to keep the 
> whitespace, but within the expression, it should be ignored.
>
> I've currently placed a boolean into a ThreadLocal and check that 
> before calling skip() in my whitespace rule.  But, that causes issues 
> with the actual expressions.
>
> For instance, "${true}" returns Boolean.TRUE while "${ true }" returns 
> a null.  If I take out the test in WS, I can have those spaces within 
> an expression, but the plain text loses all white space.
>
> I'm certain this is a simple fix, but I haven't a clue what to do!
>
> I'm using ANTLR 3.0.1.
>
> Thanks!
> -Rob
>
> /**
>  * Language is the simple expression language used throughout the XML 
> Template Engine.
>  * General language constructs:
>  * - functionName(argument, argument, ...)
>  * - object.method.method
>  * - 'string constant'
>  * - "string constant"
>  * - 1234.56
>  * - true
>  * - false
>  * - null
>  * - [ array, values, ... ]
>  */
> grammar Language;
>
> /**
>  * Use the embeddedExpression entry for expressions that are embedded 
> within other
>  * forms of text.
>  * <p>
>  * Note that unknown magic occurs because the plainText is somehow
>  * glommed together into one string instead of character
>  * by character.
>  */
> embeddedExpression returns [List<Object> list]
> @init { list = new ArrayList<Object>(); }
>     :    (    '${' expression '}'    { list.add($expression.value); }
>         |    plainText            { list.add($plainText.value); }
>         )*
>     ;
>
> /**
>  * Handle plain text that wraps around the expression.
>  */
> plainText returns [String value]
> @init { ThreadState.current().ignoreWhitespace = false; }
> @after { ThreadState.current().ignoreWhitespace = true; }
>     :    '\\' '$'                { $value = "$"; }
>     |    '\\' b=~('$')            { $value = $b.text; }
>     |    a=~('$')                { $value = $a.text; }
>     ;
>
> /**
>  * This is the bare expression evaluation.  Evaluates one expression 
> and returns
>  * an Object.
>  */
> expression returns [Object value]
>     :    function            { $value = $function.value; }
>     |    constant            { $value = $constant.value; }
>     |    reference            { $value = $reference.value; }
>     ;
>
> /**
>  * Evaluate a function invocation, along with parameters and return 
> the Object result.
>  */
> function returns [Object value]
>     :    ID '(' arguments ')'        { $value = 
> Helper.invoke($ID.text,$arguments.value); }
>     ;
> catch [LanguageException le] {
>     emitErrorMessage(le.getMessage());
>     throw new RuntimeException(le);
> }
>
> /**
>  * Evaluate a constant value and return that value.
>  */
> constant returns [Object value]
>     :    STRING                { $value = 
> $STRING.text.substring(1,$STRING.text.length()-1); }
>     |    NUMBER                { $value = Double.valueOf($NUMBER.text); }
>     |    'true'                { $value = Boolean.TRUE; }
>     |    'false'                { $value = Boolean.FALSE; }
>     |    'null'                { $value = null; }
>     |    '[' arguments ']'        { $value = $arguments.value; }
>     ;
>
> /**
>  * Evaluate an object reference and any nested evaluations and return 
> that value.
>  */
> reference returns [Object value]
>     :    a=ID                { $value = Helper.find($a.text); }
>         ('.' b=ID            { $value = Helper.reflect($value,$b.text); }
>          )*
>     ;
> catch [LanguageException le] {
>     emitErrorMessage(le.getMessage());
>     throw new RuntimeException(le);
> }
>
> /**
>  * A comma-separated argument list.  Use for the function invocation 
> and array list.
>  */
> arguments returns [List<Object> value]
> @init { value = new ArrayList<Object>(); }
>     :    (a=expression            { $value.add($a.value); }
>         (',' b=expression        { $value.add($b.value); }
>          )* )?
>     ;
>
> ID    :    Char (Char|Digit)*
>     ;
> STRING    :    '"' ~('"')* '"'
>     |    '\'' ~('\'')* '\''
>     ;
> NUMBER    :    ('-'|'+')? Digit+ ('.' Digit+)?
>     ;
> WS
>     :    (' '|'\t'|'\r'|'\n')+    { if 
> (!ThreadState.current().ignoreWhitespace) skip(); }
>     ;
>
> fragment Char
>     :    'a'..'z' | 'A'..'Z' | '_'
>     ;
> fragment Digit
>     :    '0'..'9'
>     ;
>