[antlr-interest] Fundamental question

[Mari Matinlassi]

Jim,

Thanks for the link. 

I see my problem was 'too early exit' and I was able to translate the
example to C sharp and make my parser to ignore erroneous input and continue
after that. That is, for input
	int test;
	strig another;
	int variable;
	string here;

Parser makes AST as follows
	int test;
	int variable;
	string here;

It is what I needed.

I include the C sharp example below if anyone would like to reuse that.

Mari

***********************************
grammar List;

options {
	language = CSharp3;
	TokenLabelType = CommonToken;
	output=AST;
	ASTLabelType = CommonTree; 
}

@lexer::namespace{ConsoleApplication4}
@parser::namespace{ConsoleApplication4}
@parser::header{using System;}

@members
{
	
    protected void syncToSet()
    {
       
         BitSet follow = state.following[state._fsp]; 

         syncToSet(follow);
    }

    protected void syncToSet(BitSet follow)
    {
        int mark = -1;

        try {

            mark = input.Mark();

          
            while (! follow.Member(input.LA(1)) ) {

                if  (input.LA(1) == EOF) {

                    input.Rewind();
                    mark = -1;
                    return;
                }
                input.Consume();

                // Now here, because you are consuming some tokens, yu will
probably want
                // to raise an error message such as "Spurious elements
after the class member were discarded"
                // using whatever your override of displayRecognitionError()
routine does to record
                // error messages. The exact error my depend on context etc.
                //
            }
        } catch (Exception e) {

          // Just ignore any errors here, we will just let the recognizer
          // try to resync as normal - something must be very screwed.
          //
        }
        finally {

            // Always release the mark we took
            //
            if  (mark != -1) {
                input.Release(mark);
            }
        }
    }
}

public r
    : syncClass( variables syncClass)* EOF
    ;

variables
    : presuffix ID ';'
    ;

presuffix
    : 'int'
    | 'string'
    ;

syncClass//[ListBuffer<JFXTree> mems]
@init
{ syncToSet();}
    :   
    ;

ID : 'a'..'z' + ;

WS : (' ' |'\n' |'\r' ) {$channel=Hidden;} ;
********************************************
-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Jim Idle
Sent: Thursday, November 10, 2011 4:16 PM
Cc: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Fundamental question

Take a look at:

http://www.antlr.org/wiki/display/ANTLR3/Custom+Syntax+Error+Recovery

as it may help. This is also useful for the other person asking about AST
construction on errors, though in the end that probably comes down to custom
coding.


Jim


> -----Original Message-----
> From: Mari Matinlassi [mailto:mmatinlassi at icinetic.com]
> Sent: Thursday, November 10, 2011 1:01 AM
> To: 'Jim Idle'; 'Ian Kaplan'
> Cc: antlr-interest at antlr.org
> Subject: RE: [antlr-interest] Fundamental question
>
> Jim, Ian,
>
> Thanks for pointing me to missing EOF.
>
> However, it did not .. kind of.. solve my problem.
>
> Resulting tree will now (after adding EOF and with same input) be
>
> int test ;
> <missing EOF>
>
> Whereas, I would _like to have_ a tree something like below:
>
> int test;
> <error>
> int variable;
> string here;
>
> How to refactor my grammar to make a tree like above?
>
> Thanks in advance,
>
> mari
>
>
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org 
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Jim Idle
> Sent: Wednesday, November 09, 2011 6:18 PM
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Fundamental question
>
> Except that the rule "type" does not allow an ID as the type, and so 
> this is a plain syntax error that is not picked up because of the lack 
> of EOF at the end of the start rule.
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest- 
> > bounces at antlr.org] On Behalf Of Ian Kaplan
> > Sent: Wednesday, November 09, 2011 9:09 AM
> > To: Mari Matinlassi
> > Cc: antlr-interest at antlr.org
> > Subject: Re: [antlr-interest] Fundamental question
> >
> >   There is nothing wrong as far as the syntax goes with this.  There 
> > could, for example, be a user defined type called strig (strings for 
> > trigonometry :).  So this is grammatically correct.  What is not 
> > correct is the semantics.  In fact, there probably is no user 
> > defined type strig.  So this is an error may be issued after the 
> > syntax tree is built, when a semantic phase makes a pass over the 
> > tree.  The semantic phase would discover that there was no type 
> > strig and report an error ("No type 'strig' on line 42").
> >
> >
> > On Wed, Nov 9, 2011 at 8:18 AM, Mari Matinlassi
> > <mmatinlassi at icinetic.com>wrote:
> >
> > > Hi,
> > >
> > > There is something fundamental and important that I have not 
> > > understood with ANTLR grammars.
> > >
> > > If I need to parse something like...
> > >
> > >        int test;
> > >        strig another;
> > >        int variable;
> > >        string here;
> > >
> > > You notice there is a typing error on the second line ('strig'
> > instead
> > > of 'string').
> > >
> > > How do I make a grammar that will not stop parsing but, continues 
> > > after an error??
> > >
> > > Below is an example how I CANNOT make it work the right way
> (created
> > > AST contains only 'int test ;')..
> > >
> > > grammar List;
> > >
> > > options {
> > >        language = CSharp3;
> > >        TokenLabelType = CommonToken;
> > >        output=AST;
> > >        ASTLabelType = CommonTree;
> > > }
> > >
> > > @lexer::namespace{ConsoleApplication4}
> > > @parser::namespace{ConsoleApplication4}
> > >
> > >
> > > public r
> > >        : variables*
> > >        ;
> > >
> > > variables
> > >        : type ID ';'
> > >        ;
> > >
> > > type
> > >        : 'int'
> > >        | 'string'
> > >        ;
> > >
> > > ID : 'a'..'z' + ;
> > >
> > > WS : (' ' |'\n' |'\r' ) {$channel=Hidden;} ;
> > >
> > >
> > > Many thanks for your time and help,
> > >
> > > Mari
> > >
> > >
> > >
> > > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > > Unsubscribe:
> > > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> > address
> > >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-
> interest/your-
> > email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address