[antlr-interest] Re: Request enhancement ANTLR

[Dennis Marsa]

Terence Parr wrote:
> 
> On Thursday, June 20, 2002, at 03:11  PM, micheal_jor wrote:
> 
> > Hi,
> >
> >> Problem:
> >> ANTLR does not register filename information with tokens;
> >> only line/column information.
> >>
> >> This becomes a problem when using include files in which
> >> case it is unknown which source file a token originates from.
> >>
> >> Solution:
> >> Add setFilename/getFilename to Token class and change
> >> other classes (including code generators) to use this method.
> >>
> >> The impacted classes are: Token, CommonToken,
> >> CharScanner, LexerSharedInputState, JavaCodeGenerator
> >> and CppCodeGenerator.
> >>
> >> I'd be willing to implement these changes if we agree that this
> >> enhancement is useful.
> >
> > Care to post the code? And same message to you too John Allen Green.
> > It helps if others can try out the changes you propose...
> >
> > Quick note, is it possible to use a shared string table and just
> > store a compact handle in each token?
> 
> This should happen by default as the filename will be one string that
> gets pointed to by every token.  I like this concept by the way, but I'm
> nervous about adding 4 bytes to every single token. 'course you can just
> make a subclass of CommonToken and then tell antlr to build those, right?
> 
> Ter

Why not implement setFilename/getFilename in class Token exactly like
setText/getText are currently? 

That is, class Token doesn't store the filename itself, but it does define
the interface for setting and getting the filename, with do-nothing default
implementations.

Of course, one would still have to subclass Token to add storage for the
filename, and override setFilename/getFilename to manipulate it.

Dennis

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/