[antlr-interest] Serious doubts on usage of incremental parsinginides

[Jean Bovet]

Hi,

I have been reading your last e-mails with great interest because I 
have similar problems in ANTLRWorks. I would like to share my comments 
about that and if you have any suggestions or comments, I would be 
delighted to read them!

Currently, I have to update the following things at each keystroke:

1) background parser (thread A)
2) syntax coloring (thread B)
3) list of all rules/tokens (thread C)
4) syntax diagram/NFA (thread D)
5) gutter information (thread E)

There is not as many elements as Prashant is talking about, but this is 
enough to slow down the editor depending on how you implement that. I 
am currently using the following methods which doesn't slow the editor 
(except when the syntax diagram is displayed - see my comments below) 
while trying to keep everything in sync as fast as possible:

1) At each keystroke, I awake my parsing thread (thread A) and, if the 
user doesn't press any key before a certain amount of time (let says a 
threshold of 100 ms), thread A runs. If the user presses a key before 
this threshold, thread A is put back to sleep again. Because the parser 
in thread A is running fast (full parsing, not incremental), the user 
won't notice any slow down.

2) When the parsing thread A is done, it notifies the other threads (B, 
C, D and E) so they starts to work. Again, if the user presses a key 
before a certain threshold in time, threads B, C, D and E are 
cancelled. If the user pauses long enough (let say more than 500 ms), 
threads B, C, D and E will have time to complete there work and the 
window will be updated.

3) On thread B (syntax coloring), I have the following problem: I need 
to lock the editor while the colorization is being made because 
otherwise the user can insert/delete a character and the color will not 
be applied at the right place. I currently cache the previous tokens 
list so when the new tokens list is available, I only colorize the 
modified tokens (using a fast diff comparaison). This is fast enough so 
the user doesn't notice anything. But this is maybe not the best 
solution - any idea ?

4) As Prashant wrote, the only time I notice a slowdown is when Swing 
updates the syntax diagram view. Because it is doing that in the events 
thread, if the display is taking too much time, the text zone will be 
"frozen" for a few ms and the user will notice that. I didn't find a 
nice way to solve that (is Swing able to update a view in another 
thread than the event thread?): I am currently caching the syntax 
diagram in a image (in thread C) so when Swing updates the view, only 
the image has to be drawn (and this is done only when the user pauses 
long enough).

In short, everything is fast and doesn't alter the edit experience of 
the user except when Swing is updating part of the window. How are 
other IDEs doing to update graphic elements without delaying the user 
input ? Are they updating them in another thread than the event thread 
or is it simply a matter of speed when rendering these elements ?

Best regards,

Jean