[antlr-interest] lexing: case insensitivity

[Edwards, Waverly]

Outstanding!

I will definitely have to look into this.

Thanks,


W.

-----Original Message-----
From: Sam Ellis [mailto:Sam.Ellis at arm.com] 
Sent: Monday, August 20, 2007 4:34 AM
To: Edwards, Waverly; antlr
Subject: RE: [antlr-interest] lexing: case insensitivity

antlr-interest-bounces at antlr.org wrote:
> Now the question remains:
> In ANTLR Is it possible to manage the file stream *before* the lexer 
> handles it?

Yes. My solution to this involves extending ANTLRInputStream, and then
implementing the load() method to load the data and then process the
data[] array:

public class MyInputStream extends ANTLRInputStream {

   ... other methods ...

   public void load(Reader r, int sizem int readChunkSize) throws
IOException {
       super.load(r, size, readChunkSize);

       /* Now process data[0..n-1]. */
   }
}

The length of the data is stored in a field called 'n'. I find the
choice of name 'n' slightly annoying as I commonly use 'n' as a loop
iterator. Perhaps a better name could be used, 'dataLength'.

In any case, you can then pass an instance of your new class to the
constructor of your lexer. An alternative, which I was using previously,
was to process the stream on the fly before it is read by
ANTLRInputStream, e.g. extend InputStream. However, in my case, I found
it more efficient to use the apparach outlined above as I can operate on
the whole input in a single operation.


--
Sam Ellis, RVDK Team Leader,
Product Engineering Group,                 Tel: +44 (0) 1223 400516
System Design Division,                    Fax: +44 (0) 1223 400887
ARM Ltd., 110 Fulbourn Road,               skype:armsamellis
Cambridge, CB1 9NJ                         mailto:sam.ellis at arm.com

--
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy
the information in any medium.  Thank you.