[antlr-interest] passing stuff from lexer to parser

[Gavin Lambert]

At 10:02 2/01/2008, siemsen at ucar.edu wrote:
>To handle the include statements, I use the mechanism described 
>in the ANTLR Wiki page titled "How do I implement include 
>files?".  It works great.  It does its magic during the lexer 
>phase.  So all the source files are lexed first into one big 
>token stream, then the parser starts.
>
>Currently, my translator just emits output to standard out, as 
>one text stream.  Now I'm ready to make it put the output into 
>directories and files.  The source text is a set of things with 
>names like CIM_DatabaseResourceStatistics, so I know what to name 
>each output file.  I just need to know what directory to put each 
>output file in.

If you're generating separate outputs for each file, then treating 
it as a single lexer stream seems a little odd.  (Not necessarily 
wrong, especially if there are dependencies between the 
files.  Just odd.)

If there aren't order-based dependencies between the files, then 
it might be better to treat them as separate translation units and 
invoke separate instances of the lexer/parser to deal with them.

>During the lexer phase, I store the name-to-directory information 
>in a HashMap.  So for example, the HashMap tells me that the 
>output file named CIM_DatabaseResourceStatistics.java belongs in 
>the output subdirectory named "Database".
>
>I need to pass the HashMap from the lexer to the parser.  Is 
>there a good way to do it?  Am I thinking about the problem 
>correctly?

Probably the easiest way to do this is to pass an INCLUDE token up 
to the parser that contains the full filename, and let the parser 
reconstruct the HashMap itself.  Or you could use it in a scope 
instead, since presumably everything else is logically contained 
within one or more INCLUDEs.