[antlr-interest] C-Target $label.text / toString malfunction

Jim Idle jimi at temporal-wave.com
Tue Apr 3 12:29:45 PDT 2007


OK - I see what you are getting at - should be an easy fix (I think) -
have to give me a few days though as I just got back from vacation and
have lots of things piled up.

Jim

-----Original Message-----
From: Tobias Pape [mailto:Das.Linux at gmx.de] 
Sent: Saturday, March 31, 2007 9:26 AM
To: antlr-interest at antlr.org
Cc: Jim Idle
Subject: C-Target $label.text / toString malfunction

Hi guys, hi Jim in particular.


I've recently got a nasty behavoir in my generated Lexer/Parser.

example grammar snippet:
==============================================
unaryPattern returns [ pASTPattern p ]
   : s=unarySelector
     { $p = (pASTPattern) ASTUnaryPattern_new((char *) $s.text-
>chars); }
   ;

binaryPattern returns [ pASTPattern p  ]
   : s=binarySelector v=argument
     { $p = (pASTPattern) ASTBinaryPattern_new((char *) $s.text-
>chars, v.v); }
   ;

keywordPattern returns [ pASTKeywordPattern p ]
   : (s=keyword v=argument {
   	$p = ASTKeywordPattern_new();
   	$p->add_argument($p,(char *)  $s.text->chars, v.v);
   } )+
   ;
==============================================

as you see, i reference the s-label (as suggested to me by Jim) to then
get the chars out of it.

At this point, i have particular serious problems:
 >>
 >> warning: passing argument 1 of 'ctx->pParser->tstream->toString'  
from incompatible pointer type
 >>
I looked into the code and found:
==============================================
            	            /* End: ruleRef
(rule,label,elementIndex,args) */
             	            if ( backtracking == 0 )
             	            {
             	
             	                  	retval.p=
ASTKeywordPattern_new();
             	                  	retval.p->add_argument(retval.p,
(char *)  strStream->toString(s.start,s.stop)->chars, v.v);
             	
             	            }
==============================================

as you see, it's from the keywordPattern-rule.
the problem is, as of b6/b7 (which I tried) one can't call toString like
this.
I found some toString functions in the antlr3tokenstream,h connected to
what strStream should return:
==============================================
    /** Return the text of all the tokens in the stream, as the old
tramp in
      *  Leeds market used to say; "Get the lot!"
      */
     pANTLR3_STRING	    (*toString)		(struct  
ANTLR3_TOKEN_STREAM_struct * tokenStream);

     /** Return the text of all tokens from start to stop, inclusive.
      *  If the stream does not buffer all the tokens then it can just
      *  return an empty ANTLR3_STRING or NULL;  Grammars should not
access $ruleLabel.text in
      *  an action in that case.
      */
     pANTLR3_STRING	    (*toStringSS)	(struct  
ANTLR3_TOKEN_STREAM_struct * tokenStream, ANTLR3_UINT64 start,
ANTLR3_UINT64 stop);

     /** Because the user is not required to use a token with an index
stored
      *  in it, we must provide a means for two token objects themselves
to
      *  indicate the start/end location.  Most often this will just
delegate
      *  to the other toString(int,int).  This is also parallel with
      *  the pTREENODE_STREAM->toString(Object,Object).
      */
     pANTLR3_STRING	    (*toStringTT)	(struct  
ANTLR3_TOKEN_STREAM_struct * tokenStream, pANTLR3_COMMON_TOKEN start,
pANTLR3_COMMON_TOKEN stop);

==============================================



therefore I included
==============================================
	 //seems to be buggy in \$label.text implementation
	 //THIS IS A DIRTY HACK!
	 #define toString(A,B) toStringTT(strStream,(A),(B))
==============================================

into the @init block of my first parser rule, but later patched the
C.stg to say ==============================================
<else>
strStream->toStringSS(strStream,(ANTLR3_INT32)(<scope>.start-
 >getTokenIndex(<scope>.start)),(ANTLR3_INT32)(<scope>.stop-
 >getTokenIndex(<scope>.stop)))
<endif>
==============================================
instead of
==============================================
<else>
strStream->toString(<scope>.start,<scope>.stop)
<endif>
==============================================


this maked my Source compile, but nasty, nasty, now _every_ reference in
the form of $label.text (irrespective of the label) turns out to be the
same string (i.e. for me, the Text of the first non-hidden Token).

can somebody try to reproduce or tell me, where im wrong?
I suspect this a bug in either the C.stg or in the libantlr3c (or both).

thaks for listening to me,
have a nice Sunday,
	-Tobias










More information about the antlr-interest mailing list