[antlr-interest] fragments..

Tobias Pape Das.Linux at gmx.de
Mon Mar 26 03:20:18 PDT 2007


Dear All,

today I've got a Problem w/ fragmens in b6 and up
(occuring in C Target.)

Consider the following Grammar:

----in the Parser ----
// Grammar productions
classdef returns [ pASTClass c ]
   @init {
	 // seems to be buggy in \$label.text implementation
	 // THIS IS A DIRTY HACK!
	 #define toString(A,B) toStringTT(theInput(),(A),(B))
	}
     : name=identifier Equal (  s_name=identifier
				{ $c->set_names($c, (char *) $name.text->chars, (char *)  
$s_name.text->chars); }
			    |	{ $c->set_names($c, (char *) $name.text->chars, "Object"); }
			    )
     NewTerm
     instanceFields[$c]
     ( m=method { $c->add_instance_method($c, m.m); } )*

     ( Separator
       classFields[$c]
       ( m=method { $c->add_class_method($c, m.m); } )*
     )?
     EndTerm
   ;
-----------------------

-----in the Lexer-----
BinarySelector :
( Minus Minus Minus Minus ) =>
   Separator {$type=Separator;}
|
( SingleOperator SingleOperator ) =>
   OperatorSequence {$type=OperatorSequence;}
|
( Minus SingleOperator ) =>
   Minus OperatorSequence {$type=OperatorSequence;}
|
( Minus ) =>
   Minus {$type=Minus;}
|
( Comma ) =>
   Comma {$type=Comma;}
|
( Or ) =>
   Or {$type=Or;}
|
( Equal ) =>
   Equal {$type=Equal;}
|
   (SingleOperator ) => SingleOperator {$type=SingleOperator;}
;

/* ... */
fragment SingleOperator :
   Not | And | Or | Star | Div | Mod | Plus |
   Equal | More | Less | Comma | At | Per
;

fragment Not   : '~' ;
fragment And   : '&' ;
fragment Or    : '|' ;
fragment Star  : '*' ;
fragment Div   : '/' ;
fragment Mod   : '\\' ;
fragment Plus  : '+' ;
fragment Minus : '-' ;
fragment Equal : '=' ;
fragment More  : '>' ;
fragment Less  : '<' ;
fragment Comma : ',' ;
fragment At    : '@' ;
fragment Per   : '%' ;
------------------------


The Problem occurring is: I don't know how to macht classdef the way  
I want:

I've written (merely copy&pastes Jim's b4 example ) a litle tester.
======
/*...*/
while   ((token = tstream->tstream->LT(tstream->tstream, 1))->getType 
(token) != ANTLR3_TOKEN_EOF)
     {
		printf("Token is %s\n", token->toString(token)->chars);
		tstream->tstream->istream->consume(tstream->tstream->istream);
     }
	
	tstream->free(tstream);
	psr->free(psr);
	lxr->free(lxr);
	input->close(input);
/*...*/
======

for every Input I test, it only prints out the first non-hidden  Token,
then, when matching "Equal" comes to
========
    /* End: ruleRef(rule,label,elementIndex,args) */
             /* tokenRef(token,label,elementIndex) */
             Equal1 = (pANTLR3_COMMON_TOKEN)LT(1);
             tmatch(Equal, FOLLOW_Equal_in_classdef96);
             if (hasFailed())
             {
                 return retval;
             }
=======
in the parser, which stepps through and goes to the lexer
=======
    /* SOM.g:461:9: ( '=' ) */
     /* SOM.g:461:9: '=' */
     {
         matchc('=');
         if (hasFailed())
         {
             return ;
         }

     }


     if ( backtracking == 0 )
     {

             if ( ltoken() == NULL && !hasException() && ctx->pLexer- 
 >ruleNestingLevel==1)  //////////////-----------------
             {
         		emitNew(_type,_line,_charPosition,_channel,_start,  
(_end==0 ? getCharIndex()-1 : _end));
             }


     }    ctx->pLexer->ruleNestingLevel--;
========
ctx->pLexer->ruleNestingLevel will never be 1 at the marked point,
due to Equal was defined as a Fragment, as Jim pointed out to me.

My problem is, when I change the Equal Rule, to be non-Fragment,  
antlr warns me about Token Equal not beeing accessible.
--------
h248:~/dev/parsertest tobias$ antlr_ SOM.g
no such locale file org/antlr/tool/templates/messages/languages/ 
de.stg retrying with English locale
ANTLR Parser Generator  Version 3.0b7 (??, 2007)  1989-2007
warning(208): SOM.g:461:1: The following token definitions are  
unreachable: Equal
--------


weird, huh?


anybody there who wants to enlighten me in this very case?

thanks,
	-Tobias 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: Signierter Teil der Nachricht
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20070326/0e4f3641/attachment.bin 


More information about the antlr-interest mailing list