[antlr-interest] [C] code to change Token type, use char* and loose data when buffer destroyed

[Douglas Godfrey]

You need to modify your string lexing rules to use sub-rules for the
elementary
strings and return the concatenated string as the lexer token value.

The value of 

StringConstant: QuotedString
		{RemoveQuotePairs($QuotedString);};

fragment
QuotedString: 	( StringTerm )+;

fragment
StringTerm: 	Dquote ( Character )* Dquote;

fragment
Character:	( ' ' | AlphaChar | Punctuation | Digit );






On 9/27/11 5:16 AM, "Ruslan Zasukhin" <ruslan_zasukhin at valentina-db.com>
wrote:

>Hi All,
>
>===== TASK ======
>
>In SQL we must be able write
>      SELECT 'aaa' 'bbbb'
>
>And this should be same as
>      SELECT 'aaabbbb'
>
>I.e. Parser must concatenate literals self.
>This was quite easy do in ANTLR 2,
>and I already have kill 5-6 hours in ANTLR 3.  :-((((((
>
>
>I have try many tricks for ANTLR3 itself trying to use its tokens and
>ANTLR_STRING class but no luck.
>
>Finally I have give up and have try to use simple code as in v2 using
>STD::string as place to accumulate literal.
>
>=================================
>character_string_literal
>@init{ 
>    STD::string st;
>}
>    :    ( STRING_LITERAL
>            { 
>                st.append(
>                    (const char*) $STRING_LITERAL.text->chars,
>                    $STRING_LITERAL.text->len );
>            } 
>        )+
>            -> ^( CONST_STR[ st.c_str() ] )
>    ;    
>=================================
>
>But this not works, because new Token object stores just pointer
>
>        newToken->textState        = ANTLR3_TEXT_CHARP;
>        newToken->tokText.chars = (pANTLR3_UCHAR)text;
> 
>And as only STD::string dies we get problem.
>
>
>Jim, how this simple task can be solved in the C TARGET ?
>
>Also I see that for Java code they can contruct dynamic text
>And produce token using that text. For example on this page
>
>http://www.antlr.org/wiki/display/ANTLR3/Tree+construction
>
>                            -> ^('+' $p
>INT[String.valueOf($a.int+$b.int)])
>
>
>But C target tryies to work only which char*
>
>
>I guess that ANTLR_STRING setText() can help me,
>But I cannot see how I can call that from my
>
>            -> ^( CONST_STR[ st.c_str() ] )
>
>???
>
>Thank you for points ...
>
>
>-- 
>Best regards,
>
>Ruslan Zasukhin
>VP Engineering and New Technology
>Paradigma Software, Inc
>
>Valentina - Joining Worlds of Information
>http://www.paradigmasoft.com
>
>[I feel the need: the need for speed]
>
>
>
>List: http://www.antlr.org/mailman/listinfo/antlr-interest
>Unsubscribe: 
>http://www.antlr.org/mailman/options/antlr-interest/your-email-address