[antlr-interest] Tree parser changes Token type?

[Anakreon Mejdi]

There is only one DO in the test file.
The System.out should be for that "DO" only.
In the tree printed, DO has the original type (unmodified) and
the tree structure is the expected one.
If the lexer had a problem with DO how could the parser have matched the
rule do_statement.
In the lexer the DO is defined anyway.

I solved the problem by modifying the tree parser
not #(DO_WHILE... but (#DO ...

If you want the grammar for further investigation I can post it.

Anakreon

Terence Parr wrote:
> Howdy.  It looks like you have a problem in the lexer not the parser.  
> You probably reference a "Do" somewhere or don't define DO in the lexer.
> 
> Terence
> 
> On Friday, November 29, 2002, at 04:09  AM, Anakreon Mejdi wrote:
> 
> 
>>I observed somthing really strange.
>>The parser changes the type of the Token (which I verified with a
>>System.out) but when the tree is printed (with ASTVisitor) the Token
>>apears to have the original type.
>>
>>The mail has two attachments.
>>The file used for testing and the output of the program.
>>Here is the rule for the do statement.
>>do_statement
>>   { boolean whileFound = false;}
>>   :
>>   d:DO^
>>   (
>>   /*
>>     The problem is when
>>     do while a < 0
>>         while z > 3
>>           .....
>>         wend
>>         ...
>>     loop
>>     can not distinguish betwen do while .. while .. wend loop
>>     It should be ok
>>    */
>>     options {
>>         warnWhenFollowAmbig = false;
>>     }
>>     :
>>     STATEMENT_END!
>>     | do_while {whileFound = true; d.setType(DO_WHILE);} 
>>(STATEMENT_END!)?
>>   )?
>>   (statements)*
>>   LOOP!
>>   (
>>     {!whileFound}? do_while {d.setType(WHILE_DO);}
>>     |
>>   )
>>   {
>>       System.out.println("DO type is:" + d.getType());
>>   }
>>   ;
>>
>>If interested in investigating the phainomenon I can send all the 
>>gramar.
>>
>>By.
>>Anakreon
>>
>>
>>
>>Your use of Yahoo! Groups is subject to 
>>http://docs.yahoo.com/info/terms/
>>
>>Vbs Parser
>>DO type is:67
>>found obj_member:eof
>>found obj_name: rs
>>found not
>>found obj_member:bof
>>found obj_name: rs
>>found not
>>found and
>>found obj_member:Fields
>>found obj_member:Count
>>found obj_name: rs
>>found minus
>>found obj_member:Fields
>>found obj_member:Name
>>found obj_name: rs
>>found if and expr and statements
>>ANTLR Tree Parsing RecognitionException Error:
>>File Name:<AST>
>>[-1:-1]
>><AST>: unexpected AST node: Do
>>	at gr.omadak.leviathan.asp.VbsTree.if_statement(VbsTree.java:841)
>>	at gr.omadak.leviathan.asp.VbsTree.statement(VbsTree.java:354)
>>	at gr.omadak.leviathan.asp.VbsTree.statements(VbsTree.java:1999)
>>	at gr.omadak.leviathan.asp.VbsTree.expr_statement(VbsTree.java:1979)
>>	at gr.omadak.leviathan.asp.VbsTree.if_statement(VbsTree.java:797)
>>	at gr.omadak.leviathan.asp.VbsTree.statement(VbsTree.java:354)
>>	at gr.omadak.leviathan.asp.VbsTree.start_rule(VbsTree.java:193)
>>	at Main2.main(Main2.java:33)
>>
>>
>>found if and expr and statements
>>found if
>>AST
>>
>> [73]
>>if [23]
>>   EXPR [50]
>>      iMode [77]
>>      = [51]
>>      1 [61]
>>   if [23]
>>      EXPR [50]
>>         and [4]
>>            not [29]
>>               rs [59]
>>                  . [63]
>>                     eof [60]
>>            not [29]
>>               rs [59]
>>                  . [63]
>>                     bof [60]
>>      for [21]
>>         FOR_INIT [66]
>>            = [78]
>>               i [77]
>>               EXPR [50]
>>                  0 [61]
>>            to [43]
>>               - [90]
>>                  rs [59]
>>                     . [63]
>>                        Fields [60]
>>                     . [63]
>>                        Count [60]
>>                  1 [61]
>>         rs [59]
>>            . [63]
>>               Fields [60]
>>               PARAMETERS [56]
>>                  EXPR [50]
>>                     i [77]
>>            . [63]
>>               Name [60]
>>         next [28]
>>      = [78]
>>         j [77]
>>         0 [61]
>>      Do [9]
>>         While [46]
>>            EXPR [50]
>>               Not [29]
>>                  rs [59]
>>                     . [63]
>>                        EOF [60]
>>         for [21]
>>            FOR_INIT [66]
>>               = [78]
>>                  i [77]
>>                  EXPR [50]
>>                     0 [61]
>>               to [43]
>>                  - [90]
>>                     rs [59]
>>                        . [63]
>>                           Fields [60]
>>                        . [63]
>>                           Count [60]
>>                     1 [61]
>>            rs [59]
>>               . [63]
>>                  Fields [60]
>>                  PARAMETERS [56]
>>                     EXPR [50]
>>                        i [77]
>>            next [28]
>>         rs [59]
>>            . [63]
>>               MoveNext [60]
>>         = [78]
>>            j [77]
>>            + [89]
>>               j [77]
>>               1 [61]
>>
>>
>> [73]
>><%@Language="VBSCript"%>
>><%
>>    if iMode = 1 then
>>			if not rs.eof and not rs.bof then
>>        for i=0 to rs.Fields.Count-1
>>            rs.Fields(i).Name
>>        next
>>        j=0
>>				Do While Not rs.EOF
>>            for i=0 to rs.Fields.Count-1
>>                rs.Fields(i)
>>            next
>>            rs.MoveNext()
>>            j=j+1
>>				Loop
>>      end if
>>		end if
>>%>
>>
>>
> --
> Co-founder, http://www.jguru.com
> Creator, ANTLR Parser Generator: http://www.antlr.org
> Lecturer in Comp. Sci., University of San Francisco
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 
> 
> 



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/