[antlr-interest] ANTLR 2.x or 3 feature request re: input buffers (source available)
Richard Clark
rd_clark at sbcglobal.net
Wed Aug 4 14:23:31 PDT 2004
I've been working on an Eclipse editor based on ANTLR and would like to
request a small change in how the Lexer gets its input.
At the moment, there's an abstract InputBuffer class that is subclassed
by CharBuffer and ByteBuffer. InputBuffer's design assumes it's getting
input from a sequential device (via the fill command) and puts the
result into a queue to allow limited backtracking.
However, if I'm building an editor, I already have a big fat juicy
character buffer in memory -- it's awkward and inefficient to have to
emulate a character-at-a-time sequential device just to feed
InputBuffer.
So here's my request:
1) Refactor InputBuffer to use an abstracted "character source"
interface and move the queue management stuff into a subclass.
2) Refactor CharQueue to use the new character source interface
2) Make CharBuffer and ByteBuffer subclasses of the queue-managing
buffer.
This allows those of us who already have big buffers in memory to give
the lexer direct read access to the buffer through a simple shim. This
simplifies the code, saves memory, and eliminates a possible coherency
issue.
I've already modified a private copy to do all this and would be happy
to contribute the code back. My modified version defines an
ICharacterSource interface w/ elementAt(idx) and length(), pushes the
sequential-input assumption down to QueuedInputBuffer, and includes an
example ConstantStringInputBuffer that can be fed a string when
instantiated (which also simplifies unit testing. :))
Any takers?
...Richard
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list