[antlr-interest] C-Target $label.text / toString malfunction

Tobias Pape Das.Linux at gmx.de
Sat Mar 31 09:25:56 PDT 2007


Hi guys, hi Jim in particular.


I've recently got a nasty behavoir in my generated Lexer/Parser.

example grammar snippet:
==============================================
unaryPattern returns [ pASTPattern p ]
   : s=unarySelector
     { $p = (pASTPattern) ASTUnaryPattern_new((char *) $s.text- 
 >chars); }
   ;

binaryPattern returns [ pASTPattern p  ]
   : s=binarySelector v=argument
     { $p = (pASTPattern) ASTBinaryPattern_new((char *) $s.text- 
 >chars, v.v); }
   ;

keywordPattern returns [ pASTKeywordPattern p ]
   : (s=keyword v=argument {
   	$p = ASTKeywordPattern_new();
   	$p->add_argument($p,(char *)  $s.text->chars, v.v);
   } )+
   ;
==============================================

as you see, i reference the s-label (as suggested to me by Jim) to  
then get the
chars out of it.

At this point, i have particular serious problems:
 >>
 >> warning: passing argument 1 of ‘ctx->pParser->tstream->toString’  
from incompatible pointer type
 >>
I looked into the code and found:
==============================================
            	            /* End: ruleRef 
(rule,label,elementIndex,args) */
             	            if ( backtracking == 0 )
             	            {
             	
             	                  	retval.p= ASTKeywordPattern_new();
             	                  	retval.p->add_argument(retval.p, 
(char *)  strStream->toString(s.start,s.stop)->chars, v.v);
             	
             	            }
==============================================

as you see, it's from the keywordPattern-rule.
the problem is, as of b6/b7 (which I tried) one can't call toString  
like this.
I found some toString functions in the antlr3tokenstream,h connected  
to what strStream should return:
==============================================
    /** Return the text of all the tokens in the stream, as the old  
tramp in
      *  Leeds market used to say; "Get the lot!"
      */
     pANTLR3_STRING	    (*toString)		(struct  
ANTLR3_TOKEN_STREAM_struct * tokenStream);

     /** Return the text of all tokens from start to stop, inclusive.
      *  If the stream does not buffer all the tokens then it can just
      *  return an empty ANTLR3_STRING or NULL;  Grammars should not  
access $ruleLabel.text in
      *  an action in that case.
      */
     pANTLR3_STRING	    (*toStringSS)	(struct  
ANTLR3_TOKEN_STREAM_struct * tokenStream, ANTLR3_UINT64 start,  
ANTLR3_UINT64 stop);

     /** Because the user is not required to use a token with an  
index stored
      *  in it, we must provide a means for two token objects  
themselves to
      *  indicate the start/end location.  Most often this will just  
delegate
      *  to the other toString(int,int).  This is also parallel with
      *  the pTREENODE_STREAM->toString(Object,Object).
      */
     pANTLR3_STRING	    (*toStringTT)	(struct  
ANTLR3_TOKEN_STREAM_struct * tokenStream, pANTLR3_COMMON_TOKEN start,  
pANTLR3_COMMON_TOKEN stop);

==============================================



therefore I included
==============================================
	 //seems to be buggy in \$label.text implementation
	 //THIS IS A DIRTY HACK!
	 #define toString(A,B) toStringTT(strStream,(A),(B))
==============================================

into the @init block of my first parser rule,
but later patched the C.stg to say
==============================================
<else>
strStream->toStringSS(strStream,(ANTLR3_INT32)(<scope>.start- 
 >getTokenIndex(<scope>.start)),(ANTLR3_INT32)(<scope>.stop- 
 >getTokenIndex(<scope>.stop)))
<endif>
==============================================
instead of
==============================================
<else>
strStream->toString(<scope>.start,<scope>.stop)
<endif>
==============================================


this maked my Source compile, but nasty, nasty, now _every_ reference
in the form of $label.text (irrespective of the label) turns out to  
be the same
string (i.e. for me, the Text of the first non-hidden Token).

can somebody try to reproduce or tell me, where im wrong?
I suspect this a bug in either the C.stg or in the libantlr3c (or both).

thaks for listening to me,
have a nice Sunday,
	-Tobias








-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: Signierter Teil der Nachricht
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20070331/77bff6a6/attachment.bin 


More information about the antlr-interest mailing list