[antlr-interest] [v3] not including text in token. Still possible?

[Terence Parr]

On Feb 7, 2006, at 1:51 PM, Kay Roepke wrote:
>> If there are no bangs lexically present in the rule, don't  
>> generate the special code. :)
>
> In my infinite naiveté I actually succeded ;)

Hooray!

>>> So in the general case without any bangs, I don't want to create  
>>> any string. I want to rely on the indices into the input buffer.
>>
>> Yep.  Only do new stuff if you see a !.  Even if it's in a (..)?  
>> block.
>
> Alright, the first version is now working.

Great!

> What I do is the following:
> Upon entering a rule (in codegen.g) I do a #rule.findFirstType 
> (BANG) to determine whether this rule has a bang somewhere inside.
> If that's true (i.e. the result non-null) I turn on a flag that  
> gets set as an attribute in the lexerRule template.

Hi wonder if you would be better to create a separate template called  
lecture lexerRuleModifiedTest or something.

> Then I create a new empty StringBuffer.
> If I see a bang attached to an atom I grab the text from start to  
> getCharIndex()-1 and put that into the StringBuffer. After the
> match I set start=getCharIndex(). Repeat.

Nice!

> At the end of the rule I emit a token with the new emit method that  
> takes a String as parameter instead of start,stop.

hooray!

> Done. Works beautifully. Does it sound the least sensible, or did I  
> do something horrible? ;)

  Sensible :)  and correct minus the caveat above about the separate  
rule.  You can toss me your modified codegen.g and template file if  
you want and I'll take a look for inclusion.

Ter