[antlr-interest] C codegen change request: Expose TokenNames array

Gavin Lambert antlr at mirality.co.nz
Sun Jun 22 13:47:33 PDT 2008


At 06:59 23/06/2008, Brent Yates wrote:
>// Returns a pointer to the TokenNames array
>pANTLR3_UINT8* <name>TokenNamesArray()
>     {
> 
>return(<grammar.composite.rootGrammar.recognizerName>TokenNames);
>     }

... which is exactly what getTokenNames does.  Why redefine it?

>// Returns the number of elements in the TokenNames array
>ANTLR3_UINT32 <name>TokenNamesArraySize()
>     {
>     return(<length(tokenNames)>+4);
>     }

A more robust implementation would be:

static ANTLR3_UINT32 getTokenCount(void)
{
     return 
sizeof(<grammar.composite.rootGrammar.recognizerName>TokenNames) / 
sizeof(<grammar.composite.rootGrammar.recognizerName>TokenNames[0]);
}

Then all they need is to be linked in as overridable class methods 
of ANTLR3_PARSER.  Or even the base recogniser, in fact.  (Also, 
the array pointer is already accessible through 
yourParser->pParser->rec->state->tokenNames; it might be simpler 
to similarly expose the token count rather than make an actual 
method for it.)  Alternatively, the TokenNames table could just be 
made null-terminated, so that the size of the table is 
discoverable given a pointer to it.  (Though that's not quite as 
tidy.)


And incidentally, Jim: why isn't an empty parameter list specified 
as 'void' in the C templates?  The meaning of "()" differs between 
C and C++, so it's better to be explicit.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080623/343fb382/attachment.html 


More information about the antlr-interest mailing list