[antlr-interest] ANTLR 2.x or 3 feature request re: input buffers (source available)

[Richard Clark]

I've been working on an Eclipse editor based on ANTLR and would like to 
request a small change in how the Lexer gets its input.

At the moment, there's an abstract InputBuffer class that is subclassed 
by CharBuffer and ByteBuffer. InputBuffer's design assumes it's getting 
input from a sequential device (via the fill command) and puts the 
result into a queue to allow limited backtracking.

However, if I'm building an editor, I already have a big fat juicy 
character buffer in memory -- it's awkward and inefficient to have to 
emulate a character-at-a-time sequential device just to feed 
InputBuffer.

So here's my request:
1) Refactor InputBuffer to use an abstracted "character source" 
interface and move the queue management stuff into a subclass.
2) Refactor CharQueue to use the new character source interface
2) Make CharBuffer and ByteBuffer subclasses of the queue-managing 
buffer.

This allows those of us who already have big buffers in memory to give 
the lexer direct read access to the buffer through a simple shim. This 
simplifies the code, saves memory, and eliminates a possible coherency 
issue.

I've already modified a private copy to do all this and would be happy 
to contribute the code back. My modified version defines an 
ICharacterSource interface w/ elementAt(idx) and length(), pushes the 
sequential-input assumption down to QueuedInputBuffer, and includes an 
example ConstantStringInputBuffer that can be fed a string when 
instantiated (which also simplifies unit testing. :))

  Any takers?

  ...Richard


  



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/