Fwd: [antlr-interest] How to set filename in parser

Ric Klaren ric.klaren at gmail.com
Fri Jul 15 10:21:05 PDT 2005


Paul Johnson wrote:
>> If you set the line info correct in the lexer inputstate then tokens
>> should be created with the right line/column information. I don't
>> recall of the top of my head if CommonToken had a filename attribute.
>> You might have to make a custom token class and override makeToken in
>> the lexer to put the filename in the token as well.
> 
> CommonToken/Token only have type, line, col, and text fields (for C++, 
> anyway). Token.hpp declares:
> 
> virtual void setFilename( const std::string& file );
> virtual const ANTLR_USE_NAMESPACE(std)string& getFilename() const;
> 
> so the OP will need to derive from CommonToken, adding a filename field, 
> and defining these routines.
> 
> Incidentally, why does 'getFilename' return a string reference? This 
> doesn't make sense to me, because:
> 
> 1) The token is reference-counted, and might be deleted without your 
> knowledge, invalidating the string reference
> 
> 2) The idea was presumably to allow a single common filename to be 
> shared among multiple instances of Token, but Token has no way to handle 
> destruction of the filename anyway

If you store a reference to the string it won't get destroyed. You can 
implement a static hashtable in the token class and only return 
references without trouble AFAIK. I use the setup in a few projects.

> My solution was to ignore the setFilename/getFilename I/F completely, 
> and to use a ptr to the filename in my derived token. Does anything in 
> the rest of ANTLR use setFilename/getFilename? If not, wouldn't it 
> better to remove the I/F completely?

Not 100% sure if it is only there to provide a hook. Would have to check.

Cheers,

Ric


More information about the antlr-interest mailing list