[antlr-interest] Newbie question on lexing '{' and '{action(); }'

[Richard Clark]

On 7/13/07, Stephen Parker <Stephen.Parker at pi-shurlok.com> wrote:
> This seems to be some kind of FAQ, but I haven't found an answer.
> I want to parse a language like this:
>
>         machine G {
>          /* xxx. */
>          id1 {action1();}
>          id2 {action2();}
>         }
>
> And I have a grammar like this:
>
>         grammar Test;
>
>         prog : 'machine' ID OPEN state+ CLOSE;
>         state : case+ ;
>         case : ID ACTION ;
>
>         OPEN :  '%{';  // Would like '{'.
>         CLOSE : '%}';  // Would like '}'.
>
>         ACTION :   '{' ( ACTION | ~('}'|'{') )* '}' ;
>         ID  :   ('a'..'z'|'A'..'Z')+ ;
>         COMMENT : '/*' ( options {greedy=false;} : . )* '*/' {skip();};

I would push some of the work up into the parser (which makes more
sense than having the lexer do context-sensitive work). Basically, do
a little more work to define what can happen in an action:

grammar Test;

tokens {
  OPEN = '{';
  CLOSE = '}';
  SEMI = ';';
  LPAREN = '(';
  RPAREN = ')'
}

prog : 'machine' ID OPEN state+ CLOSE;
state : case+ ;
case : ID actionBlock ;
// Define an action block
actionBlock : OPEN (statement | actionBlock)* CLOSE ;
//  Only handling function calls for now
statement: ID LPAREN RPAREN SEMI ;
// Not much left for the lexer to do...
ID  :   ('a'..'z'|'A'..'Z')+ ;
COMMENT : '/*' ( options {greedy=false;} : . )* '*/' {skip();};

Most people run into the problem you did when the grammar is a little
under-defined.

Hope this helps,

 ...Richard