[antlr-interest] BUG - Rewrite rules make ANTLR dumber in label=(a|b) rule

Austin Hastings Austin_Hastings at Yahoo.com
Sun Oct 7 09:03:46 PDT 2007


I have the following rule which checks for redundancy in a list of type 
specifiers and emits a diagnostic if a qualifier is found to be redundant.

decl_access_qualifier
    @init { boolean allowed = true; }
    : qual=(Kconst
    | Kvolatile)
    {
        if ($decl_specifiers::accessQuals.contains($qual.text))
        {
            say("Redundant access qualifier " + $qual.text);
            allowed = false;
        }
        else
        {
            say("Adding acc.qual. " + $qual.text);
            $decl_specifiers::accessQuals.add($qual.text);
        }
    }
    ;

I decided to make it drop the qualifier if it was found to be redundant, 
simplifying my eventual AST:

        -> {allowed}? $qual
        ->
    ;

Below is the code generated by the first and second cases. I can see how 
the presence of rewrite rules might change the way that individual 
values in the alternation are recognized and handled. But the fact is 
that without the rewrite rule, the $qual variable is being set 
correctly, whereas the presence of rewrite rules causes $qual to be 
completely unset.

If I distribute the assignment to qual, using

decl_access_qualifier
    @init { boolean allowed = true; }
    : (qual=Kconst
    | qual=Kvolatile)

then the value is stored correctly for the rewrite case. If I leave the 
assignment distributed and remove the rewrite code, the code is 
generated in the same "shape" as for the rewrite case.

I infer from this that the qual=(a|b) case is somehow receiving special 
treatment in the code genseration mechanism, but that this special 
treatment has not been extended to the case when rewriting is present. 
As a result, the rule begins as a valid rule, but adding rewrite rules 
to the end makes ANTLR dumber, causing the formerly-working code to 
suddenly break.

=Austin


    public final decl_access_qualifier_return decl_access_qualifier() 
throws RecognitionException {
        decl_access_qualifier_return retval = new 
decl_access_qualifier_return();
        retval.start = input.LT(1);
        CommonTree root_0 = null;
        Token qual=null;
        CommonTree qual_tree=null;
         boolean allowed = true;
        try {
            // src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:206:2: 
(qual= ( Kconst | Kvolatile ) )
            // src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:206:4: qual= 
( Kconst | Kvolatile )
            {
            root_0 = (CommonTree)adaptor.nil();
            qual=(Token)input.LT(1);
            if ( input.LA(1)==Kconst||input.LA(1)==Kvolatile ) {
                input.consume();
                if ( backtracking==0 ) adaptor.addChild(root_0, 
adaptor.create(qual));
                errorRecovery=false;failed=false;
            } else {
                if (backtracking>0) {failed=true; return retval;}
                MismatchedSetException mse = new 
MismatchedSetException(null,input);
                
recoverFromMismatchedSet(input,mse,FOLLOW_set_in_decl_access_qualifier497);    
throw mse;
            }

            if ( backtracking==0 ) {
                        if 
(((decl_specifiers_scope)decl_specifiers_stack.peek()).accessQuals.contains(qual.getText()))
                        {
                                say("Redundant access qualifier " + 
qual.getText());
                                allowed = false;
                        } else {
                                say("Adding acc.qual. " + qual.getText());
                                
((decl_specifiers_scope)decl_specifiers_stack.peek()).accessQuals.add(qual.getText());
                        }              
            } }
            retval.stop = input.LT(-1);
            if ( backtracking==0 ) {
                retval.tree = 
(CommonTree)adaptor.rulePostProcessing(root_0);
                adaptor.setTokenBoundaries(retval.tree, retval.start, 
retval.stop);
            }
        }







    public final decl_access_qualifier_return decl_access_qualifier() 
throws RecognitionException {
        decl_access_qualifier_return retval = new 
decl_access_qualifier_return();
        retval.start = input.LT(1);
        CommonTree root_0 = null;
        Token qual=null;
        Token Kconst18=null;
        Token Kvolatile19=null;
        CommonTree qual_tree=null;
        CommonTree Kconst18_tree=null;
        CommonTree Kvolatile19_tree=null;
        RewriteRuleTokenStream stream_Kconst=new 
RewriteRuleTokenStream(adaptor,"token Kconst");
        RewriteRuleTokenStream stream_Kvolatile=new 
RewriteRuleTokenStream(adaptor,"token Kvolatile");
         boolean allowed = true;
        try {
            // src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:206:2: 
(qual= ( Kconst | Kvolatile ) -> {allowed}? $qual ->)
            // src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:206:4: qual= 
( Kconst | Kvolatile )
            {
            // src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:206:9: ( 
Kconst | Kvolatile )
            int alt9=2;
            int LA9_0 = input.LA(1);
            if ( (LA9_0==Kconst) ) { alt9=1; }
            else if ( (LA9_0==Kvolatile) ) { alt9=2; }
            else {
                if (backtracking>0) {failed=true; return retval;}
                NoViableAltException nvae = new 
NoViableAltException("206:9: ( Kconst | Kvolatile )", 9, 0, input);
                throw nvae;
            }
            switch (alt9) {
                case 1 : // 
src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:206:10: Kconst
                    {
                    Kconst18=(Token)input.LT(1);
                    
match(input,Kconst,FOLLOW_Kconst_in_decl_access_qualifier498); if 
(failed) return retval;
                    if ( backtracking==0 ) stream_Kconst.add(Kconst18);
                    }
                    break;
                case 2 : // 
src/org/parrotcode/hlasm/antlr/HlasmSyntax.g:207:4: Kvolatile
                    {
                    Kvolatile19=(Token)input.LT(1);
                    
match(input,Kvolatile,FOLLOW_Kvolatile_in_decl_access_qualifier503); if 
(failed) return retval;
                    if ( backtracking==0 ) 
stream_Kvolatile.add(Kvolatile19);
                    }
                    break;
            }
            if ( backtracking==0 ) {
                        if 
(((decl_specifiers_scope)decl_specifiers_stack.peek()).accessQuals.contains(qual.getText()))
                        {
                                say("Redundant access qualifier " + 
qual.getText());
                                allowed = false;
                        } else {
                                say("Adding acc.qual. " + qual.getText());
                                
((decl_specifiers_scope)decl_specifiers_stack.peek()).accessQuals.add(qual.getText());
                        }
            }
            // AST REWRITE
            // elements: qual
            // token labels: qual
            // rule labels: retval
            // token list labels:
            // rule list labels:
            if ( backtracking==0 ) {
            retval.tree = root_0;
            RewriteRuleTokenStream stream_qual=new 
RewriteRuleTokenStream(adaptor,"token qual",qual);
            RewriteRuleSubtreeStream stream_retval=new 
RewriteRuleSubtreeStream(adaptor,"token 
retval",retval!=null?retval.tree:null);
            root_0 = (CommonTree)adaptor.nil();
            // 220:3: -> {allowed}? $qual
            if (allowed) { adaptor.addChild(root_0, stream_qual.next()); }
            else // 221:3: ->
            { root_0 = null; }
            } }

            retval.stop = input.LT(-1);
            if ( backtracking==0 ) {
                retval.tree = 
(CommonTree)adaptor.rulePostProcessing(root_0);
                adaptor.setTokenBoundaries(retval.tree, retval.start, 
retval.stop);
            }
        }



More information about the antlr-interest mailing list