Fwd: [antlr-interest] How to set filename in parser

Paul Johnson gt54-antlr at cyconix.com
Thu Jul 14 01:26:30 PDT 2005


Ric Klaren wrote:

> If you set the line info correct in the lexer inputstate then tokens
> should be created with the right line/column information. I don't
> recall of the top of my head if CommonToken had a filename attribute.
> You might have to make a custom token class and override makeToken in
> the lexer to put the filename in the token as well.

CommonToken/Token only have type, line, col, and text fields (for C++, 
anyway). Token.hpp declares:

virtual void setFilename( const std::string& file );
virtual const ANTLR_USE_NAMESPACE(std)string& getFilename() const;

so the OP will need to derive from CommonToken, adding a filename field, 
and defining these routines.

Incidentally, why does 'getFilename' return a string reference? This 
doesn't make sense to me, because:

1) The token is reference-counted, and might be deleted without your 
knowledge, invalidating the string reference

2) The idea was presumably to allow a single common filename to be 
shared among multiple instances of Token, but Token has no way to handle 
destruction of the filename anyway

My solution was to ignore the setFilename/getFilename I/F completely, 
and to use a ptr to the filename in my derived token. Does anything in 
the rest of ANTLR use setFilename/getFilename? If not, wouldn't it 
better to remove the I/F completely?

Paul



More information about the antlr-interest mailing list