[antlr-interest] passing stuff from lexer to parser

Gavin Lambert antlr at mirality.co.nz
Wed Jan 2 14:59:08 PST 2008


At 10:54 3/01/2008, siemsen at ucar.edu wrote:
>The top-level file contains nothing but include 
>statements, and none of the other files contain 
>include statementàphkµÿÿÿÿlZb s, but the first 2 
>included files contain code that is needed by 
>each of the other files.  The files are included 
>in order such that superclasses are defined 
>before subclasses, but that isn't really 
>important for translation.

What sort of 
code?  Constants?  Superclasses?  Support 
classes/methods?  Any of those could be dealt 
with as a separate file easily enough.

>Would it be possible to inject a token into the 
>token stream just before I switch to the include 
>file and call reset?  In the PragmaInclude lexer 
>rule, can I call "emit" to do it, and make the 
>token contain the include file name?  I haven't 
>done anything like this before, I just wonder if it's reasonable.

Lexer operation is basically just calling 
nextToken to retrieve one token at a 
time.  Calling emit sets the data for that token; 
not calling it will lead to generating a default 
token based on all the characters matched by the rule.

I'm not really familiar with the Java runtime, so 
I'm not sure what the reset call affects.  It 
might destroy an emit as well (and you probably 
can't emit afterwards successfully 
either).  Still, it could be worth a try.

The rule must currently be returning *something*, 
though, since every top-level lexer rule called 
must return a token.  Trace it through with a 
debugger and see what's going on.



More information about the antlr-interest mailing list