[antlr-interest] fragments..
Tobias Pape
Das.Linux at gmx.de
Mon Mar 26 03:20:18 PDT 2007
Dear All,
today I've got a Problem w/ fragmens in b6 and up
(occuring in C Target.)
Consider the following Grammar:
----in the Parser ----
// Grammar productions
classdef returns [ pASTClass c ]
@init {
// seems to be buggy in \$label.text implementation
// THIS IS A DIRTY HACK!
#define toString(A,B) toStringTT(theInput(),(A),(B))
}
: name=identifier Equal ( s_name=identifier
{ $c->set_names($c, (char *) $name.text->chars, (char *)
$s_name.text->chars); }
| { $c->set_names($c, (char *) $name.text->chars, "Object"); }
)
NewTerm
instanceFields[$c]
( m=method { $c->add_instance_method($c, m.m); } )*
( Separator
classFields[$c]
( m=method { $c->add_class_method($c, m.m); } )*
)?
EndTerm
;
-----------------------
-----in the Lexer-----
BinarySelector :
( Minus Minus Minus Minus ) =>
Separator {$type=Separator;}
|
( SingleOperator SingleOperator ) =>
OperatorSequence {$type=OperatorSequence;}
|
( Minus SingleOperator ) =>
Minus OperatorSequence {$type=OperatorSequence;}
|
( Minus ) =>
Minus {$type=Minus;}
|
( Comma ) =>
Comma {$type=Comma;}
|
( Or ) =>
Or {$type=Or;}
|
( Equal ) =>
Equal {$type=Equal;}
|
(SingleOperator ) => SingleOperator {$type=SingleOperator;}
;
/* ... */
fragment SingleOperator :
Not | And | Or | Star | Div | Mod | Plus |
Equal | More | Less | Comma | At | Per
;
fragment Not : '~' ;
fragment And : '&' ;
fragment Or : '|' ;
fragment Star : '*' ;
fragment Div : '/' ;
fragment Mod : '\\' ;
fragment Plus : '+' ;
fragment Minus : '-' ;
fragment Equal : '=' ;
fragment More : '>' ;
fragment Less : '<' ;
fragment Comma : ',' ;
fragment At : '@' ;
fragment Per : '%' ;
------------------------
The Problem occurring is: I don't know how to macht classdef the way
I want:
I've written (merely copy&pastes Jim's b4 example ) a litle tester.
======
/*...*/
while ((token = tstream->tstream->LT(tstream->tstream, 1))->getType
(token) != ANTLR3_TOKEN_EOF)
{
printf("Token is %s\n", token->toString(token)->chars);
tstream->tstream->istream->consume(tstream->tstream->istream);
}
tstream->free(tstream);
psr->free(psr);
lxr->free(lxr);
input->close(input);
/*...*/
======
for every Input I test, it only prints out the first non-hidden Token,
then, when matching "Equal" comes to
========
/* End: ruleRef(rule,label,elementIndex,args) */
/* tokenRef(token,label,elementIndex) */
Equal1 = (pANTLR3_COMMON_TOKEN)LT(1);
tmatch(Equal, FOLLOW_Equal_in_classdef96);
if (hasFailed())
{
return retval;
}
=======
in the parser, which stepps through and goes to the lexer
=======
/* SOM.g:461:9: ( '=' ) */
/* SOM.g:461:9: '=' */
{
matchc('=');
if (hasFailed())
{
return ;
}
}
if ( backtracking == 0 )
{
if ( ltoken() == NULL && !hasException() && ctx->pLexer-
>ruleNestingLevel==1) //////////////-----------------
{
emitNew(_type,_line,_charPosition,_channel,_start,
(_end==0 ? getCharIndex()-1 : _end));
}
} ctx->pLexer->ruleNestingLevel--;
========
ctx->pLexer->ruleNestingLevel will never be 1 at the marked point,
due to Equal was defined as a Fragment, as Jim pointed out to me.
My problem is, when I change the Equal Rule, to be non-Fragment,
antlr warns me about Token Equal not beeing accessible.
--------
h248:~/dev/parsertest tobias$ antlr_ SOM.g
no such locale file org/antlr/tool/templates/messages/languages/
de.stg retrying with English locale
ANTLR Parser Generator Version 3.0b7 (??, 2007) 1989-2007
warning(208): SOM.g:461:1: The following token definitions are
unreachable: Equal
--------
weird, huh?
anybody there who wants to enlighten me in this very case?
thanks,
-Tobias
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: Signierter Teil der Nachricht
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20070326/0e4f3641/attachment.bin
More information about the antlr-interest
mailing list