[antlr-interest] Tree construction with island grammar

[Jean-Christophe Bach]

Hi list,

I finally succeeded in posting to the list. I paste here mails I wanted to send
last week :

First mail/question :

« Hi all,

I'm trying to use antlr but I have a problem.  I have several grammars which
seem to work well independently. Now, I am trying to make them working together
: I have a host language (let's say Java, but it could be any other language)
which I parse with my HostLanguageParser and when I find specific constructions,
I would like to call the appropriate parser.

I have looked at the island-grammar example and it seems to be what I am looking
for. I tried to implement it, and it is almost ok : the parsers are called, but
I have a problem when I build the tree. I found a temporary solution with a
global variable, but I do not like it. I paste a part of my prototype with the
"solution" :

grammar HostLanguage;

options {
  backtrack=true;
  output=AST;
  ASTLabelType=Tree;
  tokenVocab=HostTokens;
}
...
@parser::member{
  public static Tree intermediateResult; // :\
}
...
<few rules>
...
sublanguageRule : BACKQUOTE -> ^({intermediateResult})

BACKQUOTE : '`('
{
  SubLanguage2Lexer lexer = new SubLanguage2Lexer(input);
  CommonTokenStream tokens = new CommonTokenStream(lexer);
  SubLanguage2Parser parser = new SubLanguage2Parser(tokens);
  SubLanguage2Parser.subLanguage2Construct_return res =
parser.subLanguage2Construct();
  HostLanguageParser.intermediateResult = (Tree)res.getTree();
}
...
(I also wrote other grammar files for my "sub-languages")

It seemed to work with a very simple example but after few tests, I saw that not
only this solution does not satisfy me, but it does not work very well : indeed,
if there are many subLanguage2 constructs, the produced tree is totally wrong.
For instance, with an example containing something like that :
...
`(foo)
`(bar)
...
my tree contains two backquote nodes with "bar" instead of a node with "foo" and
a node with "bar".

My question is : is there any simple method to build a tree with island grammars
? What would you do to obtain a tree like that :

(Program (HostBlock ...) ... (SubLanguage2Block <tree produced by my 2nd parser>) ... )

Thanks in advance,

JC »

And my second mail :

« Hi,

> My question is : is there any simple method to build a tree with
> island grammars
> ? What would you do to obtain a tree like that :
> (Program (HostBlock ...) ... (SubLanguage2Block <tree produced by my 2nd  parser>) ... )

I have an idea, but I am not sure : I could use a custom token
(http://www.antlr.org/wiki/pages/viewpage.action?pageId=1844) with a custom
field which would be a tree. And then, I could construct my tree like that :
b=BACKQUOTE -> ^(b.myTree)
Do you think that this solution could work/is correct ?

JC »

I used this solution, and my prototype seem to work well. But is it a good
practice ?  Is there any other better solution ?

Thanks in advance,

JC