[antlr-interest] Strategy for mapping output to line numbers from a tree walker

Gavin Lambert antlr at mirality.co.nz
Fri Aug 21 14:19:05 PDT 2009


At 08:47 22/08/2009, Stanislav Sokorac wrote:
>What is the best way to handle this problem when the children of 
>a node are coming from different CharStreams (include files, 
>macros, what have you...), and you could expect to have the first 
>or last token be from another stream?
[...]
>It seems to me that the tree walker has no way of determining the 
>location of the first character in PROGRAM without us tracking 
>the locations of char stream switches during lexing, but that 
>creates a special case to be checked for every one of the nice 
>and simple methods below. Is there a more elegant solution 
>available?

You're going to want to track that information anyway, in order to 
provide good error reporting (people will want to know that an 
error is on line 16 of include file B, not on line 152 of the 
input after preprocessing).

ANTLR already includes a line number with each token, and the 
filename is in the char stream.  You just need to link the token 
with the char stream it came from (which might even already be 
done; I haven't checked) and you should be good to go.


If you don't care about the error reporting, then the simplest 
thing to do is to run a complete preprocessing pass to merge all 
the include files before starting the "real" lex/parse/treewalk.



More information about the antlr-interest mailing list