[antlr-interest] unget a character

Terence Parr parrt at cs.usfca.edu
Wed Dec 22 11:28:36 PST 2004


On Dec 22, 2004, at 10:25 AM, Paul J. Lucas wrote:

> 	Is there any way to "unget" a character back onto the head of
> 	the input stream?
>
> 	I have multiple lexers and when lexer 'B' encounters a character
> 	is shouldn't, I want it to pop itself, "unget" the character,
> 	and then lexer 'A' try it.

Hi Paul.

You don't so much unget as "rewind", which is more sophisticated.  From 
TokenStreamSelector:

     /** Abort recognition of current Token and try again.
      *  A stream can push a new stream (for include files
      *  for example, and then retry(), which will cause
      *  the current stream to abort back to this.nextToken().
      *  this.nextToken() then asks for a token from the
      *  current stream, which is the new "substream."
      */
     public void retry() throws TokenStreamRetryException {
         throw new TokenStreamRetryException();
     }

Here is where that retry exception thing comes in.  Slow using an 
exception.

Alternatively, you can just use rewind yourself.  The following grammar

class Lex extends Lexer;

options {
         filter=Skip;
}

A : "apple" ;
B : "banana" ;

protected
Skip : . ;

has the interesting tidbit:

                     rewind(_m);
                     resetText();
                     try {mSkip(false);}

upon RecognitionException in a token rule where _m is set in the 
nextToken() rule.

         int _m;
         _m = mark();

I suggest something like this.

Ter
--
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again!





More information about the antlr-interest mailing list