[antlr-interest] Serious doubts on usage of incremental parsinginides

Martin Probst mail at martin-probst.com
Mon Apr 25 10:35:30 PDT 2005


Hi,
why do you directly update your views? Shouldn't it be sufficient to
"invalidate" them and have them repaint themselves in the next UI
redraw? I'm not that familiar with SWT, but all GUI toolkits I've used
have such an invalidate flag (as it just makes sense performance wise).
If that still doesn't work you could also set a timeoff like e.g. 500ms
on the views to query if the parse tree has changed and redraw
themselves. Make updating the views asynchronous from actual editing,
noone needs an outline that is completly in synch with typing, as noone
uses both the editor and the outline in the very same second.

Also I'm not sure if you really need a parse tree for things like
auto-indent. I would guess it's sufficient to check that if you press
ENTER after a line that opens a new block (e.g. ending in "{",
containing an if etc.) you make sure the next line is indented. Lots of
editors (think vim) handle such stuff without a real parser. 

Also of course edit support (think indentation) controlled by a parser
would not work if there is an unrecoverable error before the statement,
e.g.
> String foo = "bar"
> if (true) ...
When I edit sourcefiles they are very often not valid until I finish.

Completly unrelated to this: my experience with a parser for a fairly
complex language to parse (about as complex as Java) was that parsing
took <20ms for average sourcefiles even on really old boxes. So you
might be right as there is no real need for incremental parsing, things
that take less than 20ms are not noticable for the user anyways. 

And you can do asynchronouse GUI updates with or without an incremental
parser ofcourse.

Regards,
Martin

On Mo, 2005-04-25 at 18:05 +0530, Prashant Deva wrote:
> Part 3  of my reply ...
> 
> Also notice that i am not saying that the incremental parser slows the
> stuff down.
> No, its updating all the other stuff attached to the parser that slows
> everything down.
> 
> Note that things like a tree view, if you are gonna update on every
> keystroke, it will flash like hell.
> Will be almost like a blinking tree ;)
> 
> PRASHANT
> 
> On 4/25/05, Prashant Deva <prashant.deva at gmail.com> wrote:
> > Part 2 of my reply .....
> > 
> > >Unless
> > > you have a very slow machine or very low RAM causing swapping, this should
> >  > be pretty much unnoticeable.
> > 
> > I have a 2.4 ghz processor and 512 mb ram and believe me my
> > incremental parser is FAST. all my tests are standalone swt apps so
> > they take like 6 mb in mem max. But still i notice sluggishness if i
> > run it on every keystroke.
> > 
> > Also eclipse has insanely complex code in ALL its ui elements to
> > handle those deltas.
> > Look at the outline view for example. They had to almost completly
> > reimplement that tree view to make it NOT work on domain objects.
> > 
> > Same goes for stuff like folding and semantic highlighting.
> > PRASHANT
> > 
> > 
> > On 4/25/05, Prashant Deva <prashant.deva at gmail.com> wrote:
> > > Scott,
> > > >
> > > > AFAICS, the user can only be editing one thing at a time, so there's really
> > > > only one model to update (no matter how many views on that model). Unless
> > > > you have a very slow machine or very low RAM causing swapping, this should
> > > > be pretty much unnoticeable.
> > >
> > > Again you are not getting me.
> > > Lets say we have the following which needs to be updated on every keystroke-
> > > 1.) Tree View
> > > 2.) Search results view
> > > 3.) 3d (hell, lets say OpenGL) dig
> > > 4.) Editor highlighting
> > > 5.) Symbol table entries
> > > 6.) Auto indentation
> > > 7.) Folding structure
> > >
> > > Now if you are going to update all these things on every single
> > > keystoke  ,then the editor is gonna seem real slow.
> > >
> > > And mind you if you don't run the parser on every keystroke, then you
> > > are limiting its power to as good as a batch parser.
> > > Hell, a batch parser also takes jsut arnd 30 ms to parse a 10,0000
> > > line file, and we are going to be updating after say 500ms anyway, so
> > > what diff does it make whether we run a batch parser in between or an
> > > incremental parser.
> > >
> > > As you are talking so much about eclipse, well look at the source.
> > > they run a FULL parse each time, and just get the deltas of the diff
> > > between 2 asts.
> > > They DONT have an incremental parser.
> > >
> > > Take a look at 'JavaSourceViewerConfiguration', in the getReconciler()
> > > method. Hell even the reconciler there is not incremental. There
> > > stands no question of incremental parsing.
> > >
> > > They just do incremental compilation, for which they anyway have to
> > > parse the full file, and as its done only each time you save, the
> > > little slowness does not get noticed.
> > >
> > > So here is my problem stated once again -
> > > You can't run an incremental parser on every keystroke cause no matter
> > > how fast the parser is, the amount of 'other' stuff to update will
> > > slow the editor.
> > > If you can't run the parser on each keystroke, the incremental parser
> > > is as good as a batch parser cause the time interval you are gonna run
> > > it in, you can run the batch parser 10 times in it anyway.
> > >
> > > So what's the use of an incremental parser.
> > >
> > > Also i was looking at this blog of the guy who handles intellisense in
> > > visual studio. even they don't run their parser on each keystroke,
> > > cause it will be too slow.
> > > And eclipse is not fast! its just that parsing is not that slow after all ;)
> > >
> > > PRASHANT
> > >
> > >
> > > On 4/25/05, Scott Stanchfield <scott at javadude.com> wrote:
> > > > Ahhh... I took "incremental parsing" for "incremental compilation".
> > > >
> > > > Never mind ;)
> > > >
> > > > AFAICS, the user can only be editing one thing at a time, so there's really
> > > > only one model to update (no matter how many views on that model). Unless
> > > > you have a very slow machine or very low RAM causing swapping, this should
> > > > be pretty much unnoticeable.
> > > >
> > > > I guess I'll come back to eclipse anyway. I've never noticed it stopping me
> > > > just because it's updating its ast. Occasionally I'll have to wait a few
> > > > seconds for a large build (due to some validation weirdness in the
> > > > webtools), but even that's minimal.
> > > >
> > > > I'm not sure if eclipse is totally rebuilding the ast each time or not (it
> > > > may be). I'm working on a plugin that filters trivial get/set methods and
> > > > displays an editable table of bean properties (which is why I want the ANTLR
> > > > tree support to be more flexible), and I had it rebuild the ast each time
> > > > and it wasn't noticeable to me. (In other words, the ast build was getting
> > > > called at least twice per refresh; once by eclipse, once by me. However,
> > > > eclipse may be caching it and detecting changes so it can simply return it).
> > > >
> > > > Later,
> > > > -- Scott
> > > >
> > > >
> > >
> >
> 



More information about the antlr-interest mailing list