[antlr-interest] Bug (difference) in ANTLR 3.2 tree matching.

[Michael Matera]

Hi Jim,

I am not constructing tree nodes myself.  The test case I provided
contains a complete grammar that shows the problem.  I'll reattach it to
this message.  Also here's the relevant parts of my grammar:

Lexer/Parser:

> tokens { 
>   BLOCK; 
> } 
> 
> file : trycatch+ EOF ;
> 
> trycatch : 'try' '{' statement+ '}' cblock ablock 
>   -> ^('try' ^(BLOCK["body"] statement+) cblock ablock)
>   ;
>   
> cblock : 'catch' '{' statement+ '}' 
>   -> ^(BLOCK["catch"] statement+)
>   ;
> 
> ablock : 'always' '{' statement+ '}' 
>   -> ^(BLOCK["always"] statement+)
>   ;
> 
> statement : STUFF ; 
> 
> STUFF: (('a' .. 'z') | ('A' .. 'Z') | ('0' .. '9'))+ ; 
> 
> WS : (' '|'\r'|'\n'|'\t')+ {$channel = HIDDEN;} ;

Tree Parser:

> @members {
> private TokenStream m_TokenStream = null;
> 
> public void setTokenStream(TokenStream ts) {
>   m_TokenStream = ts;
> }
> 
> public void run(Tree t) throws RecognitionException {
>   CommonTreeNodeStream cns = new CommonTreeNodeStream(t);
>   cns.setTokenStream(m_TokenStream);
>   TestTree test = new TestTree(cns);
>   test.exec();
> }
> 
> }
> 
> exec : (trycatch | block)+  ;
> 
> trycatch : ^('try' t=. c=. a=.)
>   {
>     System.out.println("Try/Catch: ");
>     System.out.println("  exec TRY: " + $t.toStringTree());
>     run($t);
>     System.out.println("  exec CATCH: " + $c.toStringTree());
>     run($c);
>     System.out.println("  exec ALWAYS: "  + $a.toStringTree());
>     run($a);
>   }
>   ;
> 
> block @init {
>   int sno = 0;
>       }
>       : ^(b=BLOCK {
>                       System.out.println("    Executing " + $b + " block.");
>          } (st=STUFF {
>                       System.out.println("      " + sno++ + ": " + $st);
>          })+) 
>       ;

Thanks!
./m


Jim Idle wrote:
> Are you by any chance constructing those nodes/trees yourself? I have seen behavior differences because people did not call setParent on nodes they constructed their selves and the tree walker then ends up following siblings/children that are not really there. Perhaps you can show the code/grammar you are using to generate the AST? There might be a bug I suppose.
> 
> Jim
> 
>> -----Original Message-----
>> From: Michael Matera [mailto:mike.matera at xilinx.com]
>> Sent: Tuesday, December 15, 2009 4:01 PM
>> To: Gavin Lambert
>> Cc: Jim Idle; antlr-interest at antlr.org
>> Subject: Re: [antlr-interest] Bug (difference) in ANTLR 3.2 tree
>> matching.
>>
>> Hi,
>>
>> Sorry to nag, but my test case shows that what you have suggested is
>> not
>> the problem.  During the test case execution I print out the AST that
>> resulted from matching the wildcard and, in both 3.1.1 and 3.2, the
>> resulting AST is exactly the same.  The difference in behavior is
>> caused
>> by running my TreeParser on the resulting AST.  In 3.2 it seems that
>> more nodes magically appear when they shouldn't.
>>
>> Cheers
>> ./m
>>
>>
>> Gavin Lambert wrote:
>>> At 10:44 10/12/2009, Michael Matera wrote:
>>>> It seems that the '.' operator has become more greedy in
>>>> the 3.2 release.
>>> I haven't tried 3.2 myself yet, but from what I recall of prior
>>> discussion on the list:
>>>  - in 3.0 and 3.1 the . operator was "match a single token" and did
>> not
>>> consume an entire subtree.
>>>  - in 3.2, the . operator is supposed to be "match a single token or
>>> subtree".
>>>
>>> So 3.2's . should be the equivalent of 3.0 or 3.1's "(. | ^(. .+))".
>>> (Whether that's actually the case or not, I don't know.)
>>>
>>>
>>> ie. given a rule:
>>>   rule : ^(ROOT a=. b=. c=.) ;
>>>
>>> under 3.0/3.1 people were reporting:
>>>   input = ^(ROOT ID ^(BLOCK ONE TWO) THREE)
>>>   a = ID
>>>   b = BLOCK
>>>   c = DOWN
>>> (I think.  c might have been ONE or THREE instead [I don't remember],
>>> but b wasn't the whole subtree.)
>>>
>>> Under 3.2 it's supposed to do what you'd expect.  (Again, whether it
>>> does or not...)
>>>
>>>
>> This email and any attachments are intended for the sole use of the
>> named recipient(s) and contain(s) confidential information that may be
>> proprietary, privileged or copyrighted under applicable law. If you are
>> not the intended recipient, do not read, copy, or forward this email
>> message or any attachments. Delete this email message and any
>> attachments immediately.
>>
> 
> 
> 
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 



This email and any attachments are intended for the sole use of the named recipient(s) and contain(s) confidential information that may be proprietary, privileged or copyrighted under applicable law. If you are not the intended recipient, do not read, copy, or forward this email message or any attachments. Delete this email message and any attachments immediately.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: AntlrTest.tar.gz
Type: application/x-gzip
Size: 1639 bytes
Desc: not available
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20091216/b084cadb/attachment.gz