[antlr-interest] Tree parser changes Token type?

Anakreon Mejdi amejdi at ertonline.gr
Mon Dec 2 03:15:57 PST 2002


Mea Culpa!
instead of d.setType(WHILE_DO); should write #d.setType(WHILE_DO);
The Token's type was changed but the AST node created for token d
has the original type of d which is DO.

Anakreon Mejdi wrote:
> There is only one DO in the test file.
> The System.out should be for that "DO" only.
> In the tree printed, DO has the original type (unmodified) and
> the tree structure is the expected one.
> If the lexer had a problem with DO how could the parser have matched the
> rule do_statement.
> In the lexer the DO is defined anyway.
> 
> I solved the problem by modifying the tree parser
> not #(DO_WHILE... but (#DO ...
> 
> If you want the grammar for further investigation I can post it.
> 
> Anakreon
> 
> Terence Parr wrote:
> 
>>Howdy.  It looks like you have a problem in the lexer not the parser.  
>>You probably reference a "Do" somewhere or don't define DO in the lexer.
>>
>>Terence
>>
>>On Friday, November 29, 2002, at 04:09  AM, Anakreon Mejdi wrote:
>>
>>
>>
>>>I observed somthing really strange.
>>>The parser changes the type of the Token (which I verified with a
>>>System.out) but when the tree is printed (with ASTVisitor) the Token
>>>apears to have the original type.
>>>
>>>The mail has two attachments.
>>>The file used for testing and the output of the program.
>>>Here is the rule for the do statement.
>>>do_statement
>>>  { boolean whileFound = false;}
>>>  :
>>>  d:DO^
>>>  (
>>>  /*
>>>    The problem is when
>>>    do while a < 0
>>>        while z > 3
>>>          .....
>>>        wend
>>>        ...
>>>    loop
>>>    can not distinguish betwen do while .. while .. wend loop
>>>    It should be ok
>>>   */
>>>    options {
>>>        warnWhenFollowAmbig = false;
>>>    }
>>>    :
>>>    STATEMENT_END!
>>>    | do_while {whileFound = true; d.setType(DO_WHILE);} 
>>>(STATEMENT_END!)?
>>>  )?
>>>  (statements)*
>>>  LOOP!
>>>  (
>>>    {!whileFound}? do_while {d.setType(WHILE_DO);}
>>>    |
>>>  )
>>>  {
>>>      System.out.println("DO type is:" + d.getType());
>>>  }
>>>  ;
>>>
>>>If interested in investigating the phainomenon I can send all the 
>>>gramar.
>>>
>>>By.
>>>Anakreon
>>>
>>>
>>>
>>>Your use of Yahoo! Groups is subject to 
>>>http://docs.yahoo.com/info/terms/
>>>
>>>Vbs Parser
>>>DO type is:67
>>>found obj_member:eof
>>>found obj_name: rs
>>>found not
>>>found obj_member:bof
>>>found obj_name: rs
>>>found not
>>>found and
>>>found obj_member:Fields
>>>found obj_member:Count
>>>found obj_name: rs
>>>found minus
>>>found obj_member:Fields
>>>found obj_member:Name
>>>found obj_name: rs
>>>found if and expr and statements
>>>ANTLR Tree Parsing RecognitionException Error:
>>>File Name:<AST>
>>>[-1:-1]
>>><AST>: unexpected AST node: Do
>>>	at gr.omadak.leviathan.asp.VbsTree.if_statement(VbsTree.java:841)
>>>	at gr.omadak.leviathan.asp.VbsTree.statement(VbsTree.java:354)
>>>	at gr.omadak.leviathan.asp.VbsTree.statements(VbsTree.java:1999)
>>>	at gr.omadak.leviathan.asp.VbsTree.expr_statement(VbsTree.java:1979)
>>>	at gr.omadak.leviathan.asp.VbsTree.if_statement(VbsTree.java:797)
>>>	at gr.omadak.leviathan.asp.VbsTree.statement(VbsTree.java:354)
>>>	at gr.omadak.leviathan.asp.VbsTree.start_rule(VbsTree.java:193)
>>>	at Main2.main(Main2.java:33)
>>>
>>>
>>>found if and expr and statements
>>>found if
>>>AST
>>>
>>>[73]
>>>if [23]
>>>  EXPR [50]
>>>     iMode [77]
>>>     = [51]
>>>     1 [61]
>>>  if [23]
>>>     EXPR [50]
>>>        and [4]
>>>           not [29]
>>>              rs [59]
>>>                 . [63]
>>>                    eof [60]
>>>           not [29]
>>>              rs [59]
>>>                 . [63]
>>>                    bof [60]
>>>     for [21]
>>>        FOR_INIT [66]
>>>           = [78]
>>>              i [77]
>>>              EXPR [50]
>>>                 0 [61]
>>>           to [43]
>>>              - [90]
>>>                 rs [59]
>>>                    . [63]
>>>                       Fields [60]
>>>                    . [63]
>>>                       Count [60]
>>>                 1 [61]
>>>        rs [59]
>>>           . [63]
>>>              Fields [60]
>>>              PARAMETERS [56]
>>>                 EXPR [50]
>>>                    i [77]
>>>           . [63]
>>>              Name [60]
>>>        next [28]
>>>     = [78]
>>>        j [77]
>>>        0 [61]
>>>     Do [9]
>>>        While [46]
>>>           EXPR [50]
>>>              Not [29]
>>>                 rs [59]
>>>                    . [63]
>>>                       EOF [60]
>>>        for [21]
>>>           FOR_INIT [66]
>>>              = [78]
>>>                 i [77]
>>>                 EXPR [50]
>>>                    0 [61]
>>>              to [43]
>>>                 - [90]
>>>                    rs [59]
>>>                       . [63]
>>>                          Fields [60]
>>>                       . [63]
>>>                          Count [60]
>>>                    1 [61]
>>>           rs [59]
>>>              . [63]
>>>                 Fields [60]
>>>                 PARAMETERS [56]
>>>                    EXPR [50]
>>>                       i [77]
>>>           next [28]
>>>        rs [59]
>>>           . [63]
>>>              MoveNext [60]
>>>        = [78]
>>>           j [77]
>>>           + [89]
>>>              j [77]
>>>              1 [61]
>>>
>>>
>>>[73]
>>><%@Language="VBSCript"%>
>>><%
>>>   if iMode = 1 then
>>>			if not rs.eof and not rs.bof then
>>>       for i=0 to rs.Fields.Count-1
>>>           rs.Fields(i).Name
>>>       next
>>>       j=0
>>>				Do While Not rs.EOF
>>>           for i=0 to rs.Fields.Count-1
>>>               rs.Fields(i)
>>>           next
>>>           rs.MoveNext()
>>>           j=j+1
>>>				Loop
>>>     end if
>>>		end if
>>>%>
>>>
>>>
>>--
>>Co-founder, http://www.jguru.com
>>Creator, ANTLR Parser Generator: http://www.antlr.org
>>Lecturer in Comp. Sci., University of San Francisco
>>
>>
>> 
>>
>>Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 
>>
>>
> 
> 
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 
> 
> 



 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list