[antlr-interest] Re: Line numbers

[micheal_jor <open.zone at virgin.net>]

> The AST generated by the parser however do not include the line and 
column number for the token it represents. I need that information to 
customize run-time error messages, i.e. include line numbers.
> 
> Has anybody got an (simple) example of how I could do that - extend 
the .g file to make it include line numbers in the AST class? I 
believe I'll have to extend the AST class, which is fine.

With the latest version of ANTLR, generated Lexers already track line 
and column information and they are preserved in Token objects. They 
can be accessible via getColumn() and getLine() in Token objects.

The Lexers also track File information intenally but that info isn't 
transferred to Tokens automatically. If you want to track line, 
column and file information and transfer them to your AST, you will 
need to:

- subclass the appropriate xxxToken class
  a. add a string member for filename (or add File/FileInfo fields)
  b. add accessor/mutator for the field (or use a C# property)

- override nextToken()/makeToken() in yourLexer using code like 
(example is for C#):
     protected internal override Token makeToken(int t)
     {
        YourToken newToken = (YourToken) base.makeToken(t);
        if (newToken != Token.batToken)
           newToken.setFilename(getFilename());
     }
  This could be added to your *.g grammar file.
- subclass the appropriate xxxAST class
  a. add fields (and related accessors/mutators) for line, column and 
file info
  b. override initialize(Token tok) to transfer the tracked info from 
the Token to your AST

I have literally just type this from memory but I think it does 
outline what needs to be done. Other ANTLR veterans would be able to 
advice you on which of nextToken() or makeToken() should be 
overridden in this instance - my gut feeling says nextToken().

Cheers,

Micheal


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/