[antlr-interest] Performance and LA(int)

[Paul J. Lucas]

On Wed, 23 Feb 2005, Paul J. Lucas wrote:

> On Wed, 23 Feb 2005, Paul J. Lucas wrote:
>
>> 	Thought: could I override LA(int) in my own lexer and implement
>> 	caching?  When/how would I know to invalidate the cache?  By
>> 	also overriding consume()?
>
> 	OK, so I did this.  (I also overrode rewind() and made sure to
> 	flush the cache whenever I push/pop a lexer.)
>
> 	I got a 40% performance increase!

 	Below is the relevant code, FYI.

 	- Paul


abstract class XQueryLexer extends CharScanner {

     /**
      * Consume a single character.  Additionally, "slide" the cached
      * look-ahead characters down one.
      */
     public void consume() throws CharStreamException {
         super.consume();
         int i;
         for ( i = 1; i < MAX_LA_K; ++i )
             m_la[i-1] = m_la[i];
         m_la[i-1] = NO_LA;
     }

     /**
      * Look-ahead to the <i>i</i>'th character.  We first check to see whether
      * its value is cached.
      * @param i The <i>i</i>'th character to look-ahead at (starting at 1).
      * @return Returns the <i>i</i>'th character.
      */
     public char LA( int i ) throws CharStreamException {
         assert i >= 1 && i <= MAX_LA_K;
         final int j = i - 1;
         if ( m_la[j] == NO_LA )
             m_la[j] = super.LA(i);
         return m_la[j];
     }

     /**
      * Rewind the input stream.  Additionally, we invalidate the LA cache.
      * @param pos The amount to rewind.
      */
     public void rewind( int pos ) {
         super.rewind( pos );
         invalidateLACache();
     }

     /**
      * Invalidate the look-ahead cache.
      */
     final void invalidateLACache() {
         for ( int i = 0; i < MAX_LA_K; ++i )
             m_la[i] = NO_LA;
     }

     /**
      * Construct an <code>XQueryLexer</code>.
      * @param state The {@link LexerSharedInputState} to use.
      */
     protected XQueryLexer( LexerSharedInputState state ) {
         super( state );
         // ...
         invalidateLACache();
     }

     /**
      * The maximum 'k' value of all the lexers.
      */
     private static final int MAX_LA_K = 4;

     /**
      * A special value that indicates a given look-ahead cache character is
      * invalid.
      */
     private static final char NO_LA = (char)-1;

     /**
      * The look-ahead cache.
      */
     private char m_la[] = new char[ MAX_LA_K ];
}