[antlr-interest] Question about implementation 'include' directive (v3)

[Dmitry Marienko]

Hi All !
   I've found solution for realisation includes in ANTLR v3 lexers. Hope  
it'll be usefull.

// First we override nextToken() method
  @lexer::members {
       class SaveStruct {
          public CharStream input;
          public int tokenStartCharIndex, tokenStartLine,  
tokenStartCharPositionInLine;
       }

      Stack<SaveStruct> includes = new Stack<SaveStruct>();
  	
      public Token nextToken(){
        Token token = super.nextToken();

         // we've got EOF and have non empty stack
         if(token==Token.EOF_TOKEN && !includes.empty()){
            SaveStruct ss = includes.pop();
            setCharStream(ss.input);
            tokenStartCharIndex = ss.tokenStartCharIndex;
            tokenStartLine = ss.tokenStartLine;
            tokenStartCharPositionInLine = ss.tokenStartCharPositionInLine;
            token = super.nextToken();
         }

        // skip first token after switching on another input
        if(((CommonToken)token).getStartIndex()<0)
           token = super.nextToken();

        return token;
     }
}
.....
.....
.....
// in lexer's rule:
INCLUDE
        : ('#'|'.') (WS)? 'include' (WS)? i=STRING_LITERAL ';'  {
                  String name = i.getText();
                  name = i.getText().substring(1,name.length()-1);
                  try {

                      // save current lexer's state
                       SaveStruct ss = new SaveStruct();
                       ss.input = input;
                       ss.tokenStartCharIndex = tokenStartCharIndex;
	   ss.tokenStartLine = tokenStartLine;
	   ss.tokenStartCharPositionInLine = tokenStartCharPositionInLine;
                       includes.push(ss);

                     // switch on new input stream
                      setCharStream(new ANTLRFileStream(name));
                      reset();

                 } catch(Exception fnf) { System.err.println("cannot find  
file " + name); }
        }
        ;


> Hi All !
>    I've implemented c-like language with "include" directive support  
> using  ANTLR v2.
>    Now I've decided to do it with ANTLR v3 but I didn't find any methods  
> for switching my lexer's on the fly (like TokenStreamSelector in v2).
>
  ... skip ...
>     This lexer processes included file well but doesn't  set previouse  
> input stream.  Can anyone points me on my fault or gives any advice  
> about this problem ?
>


-- 
  Dmitry E. Marienko