[antlr-interest] missing getTokenType(string) in ANTLR3C?
Jim Idle
jimi at temporal-wave.com
Tue Feb 1 09:05:57 PST 2011
It looks like you have used 'strings' in your parser and not "real" tokens
in the lexer. That is the reason I always advise to use:
p: A ;
A : 'a' ;
And not
p: 'a' ;
There is no way for you to create a cross correlation. In the former you
will get:
#define A 5
And the latter:
#define T__5 5
With the former, you can build a static table yourself:
{
A, "An", "a token";
...
But with the latter you can do nothing because you do not know what token
number is defined. In error messages, you will have the token number and
can look in your table for the string that represents it and so on.
The C target does not tell you about the parser rules you are in, as this
is only kept by the Java runtime, so you have to create a stack yourself
and push the rule in actions. However the parser rule is rarely that
useful expect in some error reporting situations.
As you say - use a tree and real tokens and it will all make sense to you.
Jim
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Bastian Asam
> Sent: Tuesday, February 01, 2011 12:56 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] missing getTokenType(string) in ANTLR3C?
>
> Thank you Jim and Justin,
> the only usefull token constants in my header files are the ones from
> the lexer, not the parser. It looks like this:
>
> /** Symbolic definitions of all the tokens that the parser will work
> with.
> * \{
> *
> * Antlr will define EOF, but we can't use that as it it is too common
> in
> * in C header files and that would be confusing. There is no way to
> filter this out at the moment
> * so we just undef it here for now. That isn't the value we get back
> from C recognizers
> * anyway. We are looking for ANTLR3_TOKEN_EOF.
> */
> #ifdef EOF
> #undef EOF
> #endif
> #ifdef Tokens
> #undef Tokens
> #endif
> #define EOF -1
> #define T__23 23
> #define T__24 24
> #define T__25 25
> #define T__26 26
> #define T__27 27
> #define T__28 28
> #define T__29 29
> #define T__30 30
> #define T__31 31
> #define T__32 32
> #define T__33 33
> #define T__34 34
> #define T__35 35
> #define T__36 36
> #define T__37 37
> #define T__38 38
> #define T__39 39
> #define T__40 40
> #define T__41 41
> #define T__42 42
> #define T__43 43
> #define T__44 44
> #define T__45 45
> #define T__46 46
> #define T__47 47
> #define T__48 48
> #define T__49 49
> #define T__50 50
> #define T__51 51
> #define T__52 52
> #define T__53 53
> #define T__54 54
> #define T__55 55
> #define T__56 56
> #define T__57 57
> #define T__58 58
> #define T__59 59
> #define T__60 60
> #define T__61 61
> #define T__62 62
> #define T__63 63
> #define T__64 64
> #define T__65 65
> #define T__66 66
> #define T__67 67
> #define T__68 68
> #define T__69 69
> #define T__70 70
> #define T__71 71
> #define T__72 72
> #define T__73 73
> #define T__74 74
> #define T__75 75
> #define T__76 76
> #define T__77 77
> #define T__78 78
> #define T__79 79
> #define T__80 80
> #define T__81 81
> #define T__82 82
> #define T__83 83
> #define T__84 84
> #define T__85 85
> #define T__86 86
> #define T__87 87
> #define T__88 88
> #define T__89 89
> #define T__90 90
> #define T__91 91
> #define T__92 92
> #define T__93 93
> #define T__94 94
> #define T__95 95
> #define T__96 96
> #define T__97 97
> #define T__98 98
> #define T__99 99
> #define T__100 100
> #define T__101 101
> #define T__102 102
> #define DEF 4
> #define COMMENT 5
> #define STRING 6
> #define INT 7
> #define DATETIME 8
> #define DURATION 9
> #define METRIC 10
> #define BEHAVIORS 11
> #define RESULTS 12
> #define ID 13
> #define RESULT_TYPES 14
> #define NUMBER 15
> #define CHAR 16
> #define WS 17
> #define SPECIALS 18
> #define DATE 19
> #define TIME 20
> #define TZD 21
> #define DAYS 22
> #ifdef EOF
> #undef EOF
> #define EOF ANTLR3_TOKEN_EOF
> #endif
>
> Sadly there are no parser rulenames. Is this correct or should there be
> some for the parser as well?
> Anyway, I guess its probably better to use a tree parser...
>
> But it's good to know, that you guys try to help :) I believe you will
> hear from me again ;) Thanks Bastian
>
>
>
>
> From: Jim Idle <jimi at temporal-wave.com>
> To: antlr-interest at antlr.org
> Date: 28-01-11 18:09
> Subject: Re: [antlr-interest] missing getTokenType(string) in
> ANTLR3C?
> Sent by: antlr-interest-bounces at antlr.org
>
>
>
> The token constants are in the .h file. If you are looking in a tree,
> you
> get the tree node, then get its payload token and get the type from
> there.
> You can include the .h file and write code to create a string map if
> you
> want to use "dddd" but I think that you mean you want DDDD, which is
> the
> #define in the generated .h file.
>
> Jim
>
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Bastian Asam
> > Sent: Friday, January 28, 2011 1:04 AM
> > To: antlr-interest at antlr.org
> > Subject: Re: [antlr-interest] missing getTokenType(string) in
> ANTLR3C?
> >
> > Hi Hiran,
> > unfortunately there aren't any generated constants for my tokens or
> at
> > least I can't find any. Since this also looks kind of odd to me I'm
> > wondering if something's going wrong on my system.
> > Can anyone confirm please, that for the C target no token constants
> are
> > generated or that there is no getTokenType Method?
> >
> > And parser.test is only a Function: GrammarParser_test_return (*test)
> > (struct GrammarParser_Ctx_struct * ctx) . I don't know how this could
> > give the type of the token test.
> >
> > Thanks
> > Bastian
> >
> >
> >
> > From: "Hiran Chaudhuri" <Hiran.Chaudhuri at web.de>
> > To: "Bastian Asam" <bastian.asam at amadeus.com>
> > Date: 27-01-11 16:29
> > Subject: Re: [antlr-interest] missing getTokenType(string) in
> > ANTLR3C?
> >
> >
> >
> > Hi, Bastian.
> >
> > I've never used the C target, but in Java I would not go for
> > getFirstChildWithType(BASE_TREE, UINT32) Function with (tree,"test")
> >
> > but instead
> >
> > getFirstChildWithType(BASE_TREE, UINT32) Function with (tree,
> > parser.TEST)
> >
> > Are there no constants generated for your tokens?
> >
> > Hiran
> >
> >
> > -----Ursprüngliche Nachricht-----
> > Von: "Bastian Asam" <bastian.asam at amadeus.com>
> > Gesendet: 27.01.2011 15:20:57
> > An: antlr-interest at antlr.org
> > Betreff: [antlr-interest] missing getTokenType(string) in ANTLR3C?
> >
> > >Hello everybody,
> > >I spend all day to find a function for translating token names to
> > their
> > >integer type value.
> > >
> > >Basically what I want is to be able to call the
> > >getFirstChildWithType(BASE_TREE, UINT32) Function with (tree,"test")
> > >instead of (tree, 23). Otherwise I would always have to change my
> code
> > >after altering the grammar a bit...
> > >
> > >I found the getTokenType(string) function in java, but not in C. Is
> it
> > >missing in C? And what can I do to get it anyways?
> > >
> > >Any help would be greatly appreciated!
> > >Thanks
> > >Bastian
> > >
> > >
> > >
> > >IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended
> only
> > >for
> >
> > >the use of the individual or entity shown above as addressees. It
> may
> > >contain information which is privileged, confidential or otherwise
> > >protected from disclosure under applicable laws. If the reader of
> > this
> > >transmission is not the intended recipient, you are hereby notified
> > >that any dissemination, printing, distribution, copying, disclosure
> or
> > >the taking of any action in reliance on the contents of this
> > >information is strictly prohibited. If you have received this
> > >transmission in error, please immediately notify us by reply e-mail
> or
> > >using the address below and delete the message and any attachments
> > from your system.
> > >
> > >Amadeus Data Processing GmbH
> > >Geschäftsführer: Eberhard Haag
> > >Sitz der Gesellschaft: Erding
> > >HR München 48 199
> > >Berghamer Strasse 6
> > >85435 Erding
> > >Germany
> > >
> > >List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > >Unsubscribe:
> > http://www.antlr.org/mailman/options/antlr-interest/your-email-
> address
> > ___________________________________________________________
> > Empfehlen Sie WEB.DE DSL Ihren Freunden und Bekannten und wir
> belohnen
> > Sie mit bis zu 50,- Euro! https://freundschaftswerbung.web.de
> >
> >
> >
> >
> >
> > IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only
> > for the use of the individual or entity shown above as addressees. It
> > may contain information which is privileged, confidential or
> otherwise
> > protected from disclosure under applicable laws. If the reader of
> this
> > transmission is not the intended recipient, you are hereby notified
> > that any dissemination, printing, distribution, copying, disclosure
> or
> > the taking of any action in reliance on the contents of this
> > information is strictly prohibited. If you have received this
> > transmission in error, please immediately notify us by reply e-mail
> or
> > using the address below and delete the message and any attachments
> from
> > your system.
> >
> > Amadeus Data Processing GmbH
> > Geschäftsführer: Eberhard Haag
> > Sitz der Gesellschaft: Erding
> > HR München 48 199
> > Berghamer Strasse 6
> > 85435 Erding
> > Germany
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-
> interest/your-
> > email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
>
>
>
>
> IMPORTANT - CONFIDENTIALITY NOTICE - This e-mail is intended only
> for
> the use of the individual or entity shown above as addressees. It may
> contain information which is privileged, confidential or otherwise
> protected from disclosure under applicable laws. If the reader of this
> transmission is not the intended recipient, you are hereby notified
> that
> any dissemination, printing, distribution, copying, disclosure or the
> taking of any action in reliance on the contents of this information is
> strictly prohibited. If you have received this transmission in error,
> please immediately notify us by reply e-mail or using the address below
> and delete the message and any attachments from your system.
>
> Amadeus Data Processing GmbH
> Geschäftsführer: Eberhard Haag
> Sitz der Gesellschaft: Erding
> HR München 48 199
> Berghamer Strasse 6
> 85435 Erding
> Germany
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address
More information about the antlr-interest
mailing list