[antlr-interest] Question about implementation 'include' directive (v3)

Dmitry Marienko dima at rts.ua
Mon May 14 06:27:05 PDT 2007


Hi All !
   I've implemented c-like language with "include" directive support using   
ANTLR v2.
   Now I've decided to do it with ANTLR v3 but I didn't find any methods  
for switching my lexer's on the fly (like TokenStreamSelector in v2).

   I attempted to save current state  in my lexer class:

@lexer::members {
    class SaveStruct {   // structure for save
       public CharStream input;
       public int tokenStartCharIndex;
       public int tokenStartLine, tokenStartCharPositionInLine, channel,  
type;
    }

   Stack<SaveStruct> includes = new Stack<SaveStruct>();
   public Token nextToken(){
      Token token = super.nextToken();
       if(token==Token.EOF_TOKEN && !includes.empty()){

       // try to restore previouse state and input stream here
          SaveStruct ss = includes.pop();
          tokenStartCharIndex = ss.tokenStartCharIndex;
          tokenStartLine = ss.tokenStartLine;
          tokenStartCharPositionInLine = ss.tokenStartCharPositionInLine;
          channel = ss.channel;
          type = ss.type;
         setCharStream(ss.input);
      }
      return token;
   }
}
  .......
  ......
  .......
INCLUDE
        : ('#') (WS)? 'include' (WS)? i=STRING_LITERAL (';')? 	{
            String name = i.getText();
           name = i.getText().substring(1,name.length()-1);
           try {
             SaveStruct ss = new SaveStruct();
             ss.input = input;
             ss.tokenStartCharIndex = tokenStartCharIndex;
             ss.tokenStartLine = tokenStartLine;
             ss.tokenStartCharPositionInLine = tokenStartCharPositionInLine;
             ss.channel = channel;
             ss.type = type;
             includes.push(ss);                                                             
// save current state here
             setCharStream(new ANTLRFileStream(name));  // and set new  
input stream
           } catch(Exception fnf) {
              System.err.println("cannot find file " + name);
           }
    }
    ;

    This lexer processes included file well but doesn't  set previouse  
input stream.  Can anyone points me on my fault or gives any advice about  
this problem ?

-- 
  Dmitry


More information about the antlr-interest mailing list