[antlr-interest] [v3] not including text in token. Still possible?
Terence Parr
parrt at cs.usfca.edu
Mon Feb 6 15:50:23 PST 2006
On Feb 6, 2006, at 3:32 PM, Kay Roepke wrote:
>
> On 7. Feb 2006, at 0:20 Uhr, Terence Parr wrote:
>
>> Hmm...well i used to just record the start index before every
>> element (token or rule) and then if not a bang, add chars from
>> saved index to current index into buffer. If a bang, then don't
>> add. Would that work?
>
> It sure would, but wouldn't this mean we'd always create a buffer
> and copy the string? I'd rather not lose the ability to use the
> indexing into the input
> buffer.
Well, you need a pure flat simple string for the overall token you
are creating. You need to copy from the buffer anyway into the new
stirng, right?
> Just recording the indices and copy to buffer once I see a bang and
> pick up after that was the first idea I had...
> I could probably set some flag if I've seen a bang and only then
> really copy the buffer. Yeah, that might work...
True, but messier I suppose. Perhaps we should think about the
common case like removing the quotes off of string and char
literals...how complicated do you want the average case to be...
Ter
More information about the antlr-interest
mailing list