[antlr-interest] C codegen change request: Expose TokenNames array
Jim Idle
jimi at temporal-wave.com
Sun Jun 22 15:19:14 PDT 2008
On Mon, 2008-06-23 at 08:47 +1200, Gavin Lambert wrote:
> At 06:59 23/06/2008, Brent Yates wrote:
>
> > // Returns a pointer to the TokenNames array
> > pANTLR3_UINT8* <name>TokenNamesArray()
> > {
> >
> > return(<grammar.composite.rootGrammar.recognizerName>TokenNames);
> > }
>
>
> ... which is exactly what getTokenNames does. Why redefine it?
>
>
> > // Returns the number of elements in the TokenNames array
> > ANTLR3_UINT32 <name>TokenNamesArraySize()
> > {
> > return(<length(tokenNames)>+4);
> > }
>
>
> A more robust implementation would be:
>
> static ANTLR3_UINT32 getTokenCount(void)
> {
> return
> sizeof(<grammar.composite.rootGrammar.recognizerName>TokenNames) /
> sizeof(<grammar.composite.rootGrammar.recognizerName>TokenNames[0]);
> }
>
> Then all they need is to be linked in as overridable class methods of
> ANTLR3_PARSER. Or even the base recogniser, in fact. (Also, the
> array pointer is already accessible through
> yourParser->pParser->rec->state->tokenNames; it might be simpler to
> similarly expose the token count rather than make an actual method for
> it.) Alternatively, the TokenNames table could just be made
> null-terminated, so that the size of the table is discoverable given a
> pointer to it. (Though that's not quite as tidy.)
Originally I just used the deferences too, as it wasn't used by the Java
example error routines and so on and I wanted to see the token table.
But as this is now used a lot in the new recovery code, and error
messages, I decided to implement the function anyway as it is probably
more obvious.
>
>
> And incidentally, Jim: why isn't an empty parameter list specified as
> 'void' in the C templates? The meaning of "()" differs between C and
> C++, so it's better to be explicit.
I usually do code that way in fact, but I will double check all that
before the next release.
Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080622/9dc43fe9/attachment.html
More information about the antlr-interest
mailing list