[antlr-interest] Multiplexed lexers and stream insertion problem

[silke7772003]

Hi Torkild, 

> The 
> problem occurs when the lexer is done with the macro 
> contents. I'm getting an EOF and can't get back to the original lexer.

The solution I have worked out uses an extended LexerSharedInputState class that keeps a stack of preprocessors. I have only one lexer that uses this input state to get its input. The preprocessor extends a Reader to actually feed the lexer. They all share also the current macro table, include pathes etc.

What happens is when an include macro is found then the current preprocessor creates a sub preprocessor and calls pushState on the shared input state, which will put the current pp on the stack and use the new one as active input. This new pp now feeds the lexer. If there is no more input then the pp calls popState and the previous pp is made active again (beside other things by setting the input buffer in the input state). It also returns a newline char, which is skipped in my lexer, so the lexer will immediately read the next input char. This next one is now read from the restored previous (and now active) pp. 

There are a few minor things to consider, but this is the base principle.

Mike
--
www.soft-gems.net 



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/