Fwd: [antlr-interest] How to set filename in parser
Ric Klaren
ric.klaren at gmail.com
Fri Jul 15 10:21:05 PDT 2005
Paul Johnson wrote:
>> If you set the line info correct in the lexer inputstate then tokens
>> should be created with the right line/column information. I don't
>> recall of the top of my head if CommonToken had a filename attribute.
>> You might have to make a custom token class and override makeToken in
>> the lexer to put the filename in the token as well.
>
> CommonToken/Token only have type, line, col, and text fields (for C++,
> anyway). Token.hpp declares:
>
> virtual void setFilename( const std::string& file );
> virtual const ANTLR_USE_NAMESPACE(std)string& getFilename() const;
>
> so the OP will need to derive from CommonToken, adding a filename field,
> and defining these routines.
>
> Incidentally, why does 'getFilename' return a string reference? This
> doesn't make sense to me, because:
>
> 1) The token is reference-counted, and might be deleted without your
> knowledge, invalidating the string reference
>
> 2) The idea was presumably to allow a single common filename to be
> shared among multiple instances of Token, but Token has no way to handle
> destruction of the filename anyway
If you store a reference to the string it won't get destroyed. You can
implement a static hashtable in the token class and only return
references without trouble AFAIK. I use the setup in a few projects.
> My solution was to ignore the setFilename/getFilename I/F completely,
> and to use a ptr to the filename in my derived token. Does anything in
> the rest of ANTLR use setFilename/getFilename? If not, wouldn't it
> better to remove the I/F completely?
Not 100% sure if it is only there to provide a hook. Would have to check.
Cheers,
Ric
More information about the antlr-interest
mailing list