[antlr-interest] Fragments and setText appear to not work at all - not even with code from the book

[Harald Mueller]

Thanks - honestly, that (and the bold example in the book) are everything I found about this issue.

I have been using ANTLR 2 now for years and years (I started with PCCTS ...), I have written very complex grammars in ANTLR 2 with "jumping lexers" (what now would be channels) parsing horrendous languages mixing fixed-column and free-text layout in the same file, I have lectured people about LL(k) and LR(k) parsing etc.; up to now I was quite happy with ANLTR 2 - but maybe I sinned and used too many features ;-)

ANTLR 3, with its LL(*) analysis and its even better performance, is something I long for (and we have changed over a few projects to it - especially complex ones) - but still, arbitrarily modifying the text of returned symbols - including fragments - is a feature I'd very much need ...

Maybe a trade-off can be made: If no ! is in a rule, then it is very fast; if ! are in there, the code creates a replacement object and hence gets only fast ;-). The surrounding (calling) rules would have to know this (but this is not a real problem, I'd say) so that they would correctly assemble their text ... ...  but probably I'm just re-discussing what you already discussed.

Still ... (right now, it takes 9 minutes to parse the dozen or so million lines I have to parse - with ANTLR 3 this could well be 2 minutes, I reckon).

Regards & thx
Harald

 and I managed

-------- Original-Nachricht --------
> Datum: Thu, 22 Nov 2007 15:55:16 +0100
> Von: "Kay Röpke" <kroepke at classdump.org>
> An: Harald Mueller <harald_m_mueller at gmx.de>
> CC: antlr-interest at antlr.org
> Betreff: Re: [antlr-interest] Fragments and setText appear to not work at	all - not even with code from the book

> Hi!
> 
> On Nov 22, 2007, at 1:37 PM, Harald Mueller wrote:
> 
> > I tried to track discussions of this issue in all of 2007. What I  
> > came up with after about 2 hours search (for "fragment", "setText",  
> > "!"(!)) in all text is only the following:
> 
> I'm sure there's more, because I vividly remember those dicussions.  
> sorry I have no links right now. Maybe it was back 2006, dunno. The  
> first beta release wasn't the beginning of development on v3. I've  
> been involved since late 2005 and at the time Ter had been working on  
> it for quite some time, already.
> 
> > --------------
> > On Apr 20, 2007, at 4:36 AM, Wincent Colaiuta wrote:
> >
> >> Replying to my own post with a more fleshed-out example grammar. My
> >> question is, does the ! operator no longer have any effect in lexer
> >> rules in ANTLR 3?
> >
> >  At the moment, it does not work in version 3.
> > Ter
> > --------------
> >
> > Is this to mean that it will work one day?
> 
> 
> 
> The ! operator is pretty ugly to implement when all you have is  
> indices. If you are copying text like v2 does, it's trivial, because  
> then you just don't copy the text for a token/char if there's the !  
> operator tacked on.
> With indices there can be holes in your character index ranges which  
> makes it non-trivial to assemble the text later on, aside from the  
> complexity of storing these non-contiguous ranges.
> 
> It is possible, but there are workarounds for most of the cases where  
> you might want it. If everything else fails, it is possible to  
> implement (and I think the ! is properly lexed and parsed by ANTLR).  
> So in theory you could write it yourself ;)
> 
> IIRC Ter was against implementing it for ANTLR v3 because it is hard  
> to get it right *and* efficient.
> 
> cheers,
> -k
> -- 
> Kay Röpke
> http://classdump.org/
> 
> 
> 
> 
> 

-- 
Psssst! Schon vom neuen GMX MultiMessenger gehört?
Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger