[antlr-interest] C codegen change request: Expose TokenNames array

[Jim Idle]

On Mon, 2008-06-23 at 08:47 +1200, Gavin Lambert wrote:

> At 06:59 23/06/2008, Brent Yates wrote:
> 
> > // Returns a pointer to the TokenNames array
> > pANTLR3_UINT8* <name>TokenNamesArray()
> >     {
> > 
> > return(<grammar.composite.rootGrammar.recognizerName>TokenNames);
> >     }
> 
> 
> ... which is exactly what getTokenNames does.  Why redefine it?
> 
> 
> > // Returns the number of elements in the TokenNames array
> > ANTLR3_UINT32 <name>TokenNamesArraySize()
> >     {
> >     return(<length(tokenNames)>+4);
> >     }
> 
> 
> A more robust implementation would be:
> 
> static ANTLR3_UINT32 getTokenCount(void)
> {
>     return
> sizeof(<grammar.composite.rootGrammar.recognizerName>TokenNames) /
> sizeof(<grammar.composite.rootGrammar.recognizerName>TokenNames[0]);
> }
> 
> Then all they need is to be linked in as overridable class methods of
> ANTLR3_PARSER.  Or even the base recogniser, in fact.  (Also, the
> array pointer is already accessible through
> yourParser->pParser->rec->state->tokenNames; it might be simpler to
> similarly expose the token count rather than make an actual method for
> it.)  Alternatively, the TokenNames table could just be made
> null-terminated, so that the size of the table is discoverable given a
> pointer to it.  (Though that's not quite as tidy.)


Originally I just used the deferences too, as it wasn't used by the Java
example error routines and so on and I wanted to see the token table.
But as this is now used a lot in the new recovery code, and error
messages, I decided to implement the function anyway as it is probably
more obvious.

> 
> 
> And incidentally, Jim: why isn't an empty parameter list specified as
> 'void' in the C templates?  The meaning of "()" differs between C and
> C++, so it's better to be explicit.


I usually do code that way in fact, but I will double check all that
before the next release.

Jim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20080622/9dc43fe9/attachment.html