[antlr-interest] (follow up) setting, altering text in lexer rules

Terence Parr parrt at cs.usfca.edu
Mon Jun 12 17:44:34 PDT 2006


On Jun 12, 2006, at 5:42 PM, shmuel siegel wrote:

> Terence Parr wrote:
>
>> Well the ! thing has always been a pain in the ass ;)  I'd rather  
>> opt for speed for now with a workable solution and see what  
>> happens in the future. :)
>> Ter
> I, for one will really miss "the ! thing" when dealing with  
> strings. It is much easier to adjust the end pointers to strings  
> than it is to create a new string as a substring. Can we at least  
> have that feature for 3.0, i.e., the ability to throw away  
> beginning and ending character sequences. Forcing the programmer to  
> manipulate strings will more than throw away any perceived time  
> savings that you think you will be achieving by avoiding this case,  
> not to mention making the programmer's job harder.

So you are saying that STRING token dominates the tokenizing time?  I  
doubt it.  Build your own token and emit that uses char indexes into  
the char buff that are one off.  No char creation at all.

I fixing like 2 bugs an hour right now...adding more bugs and  
breaking all targets is not the way to get a product shipped ;)

Ter



More information about the antlr-interest mailing list