[antlr-interest] Freemarker grammer w/ JavaScript target

Sam Harwell sam at tunnelvisionlabs.com
Wed Nov 14 13:21:51 PST 2012


This naïve approach is not scalable, and will introduce the following limitations:

1. Typing characters within a large token such as a block comment spanning many lines will be "laggy".
2. As the document grows in size, the editor will progressively slow down.

For even medium sized documents, running *just* the lexer on the entire document in response to keystrokes will be noticeably slow. Not all editors treat syntax highlighting as a line-at-a-time problem. Even in those editors I use the line-at-a-time approach to greatly improve performance of my IDEs.

The new lexer modes in ANTLR 4 make it much easier to break up tokens which would otherwise span multiple lines. It can be done in ANTLR 3 by manually tracking the mode information and using an override of nextToken that calls a fragment rule for the current mode instead of always calling mTokens. I haven't used the JavaScript target or worked with CodeMirror so I don't have any examples of how to implement this strategy in that environment.

--
Sam Harwell
Owner, Lead Developer
http://tunnelvisionlabs.com

-----Original Message-----
From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Gerald Rosenberg
Sent: Wednesday, November 14, 2012 12:29 PM
To: rodedb at gmail.com
Cc: ANTLR-Interest Interest
Subject: Re: [antlr-interest] Freemarker grammer w/ JavaScript target

Although your editor's approach is line at a time, no reason to try and force Antlr to do the same.  Antlr is more than fast and light enough to re-parse the entire source file between each keystroke and walk the AST to provide highlighting info (and walk the AST to adjust error markers and to collect code assist hints and ... ).

On 11/14/2012 12:26 AM, Roded wrote:
> Hi list,
> I'm planning on using ANTLR 3.3's JavaScript target for creating a 
> Freemarker <http://freemarker.sourceforge.net/> parser for the sake of 
> syntax highlighting (and auto-completion at a later stage) in a 
> web-based editor. Considering my lacking experience in ANTLR, I 
> thought I'd ask for any input or tips on accomplishing my goal.
> As for highlighting, using a generated AST is simple enough, however 
> my editor component's (CodeMirror <http://codemirror.net/>) syntax 
> highlighting mechanism works on individual lines of the source. Is 
> there a way to use the ANTLR parser in an interruptible mode so it 
> could be called for every line separately while retaining its state? 
> and perhaps even remedying the last parsing error in view of the new 
> input (as not all source lines pass parsing by themselves)?
> Any help and points in the right direction (whether in regards to the 
> JS target or ANTLR in general) would be much appreciated.
> Many thanks,
> Roded
>
> P.S. anyone encountered a Freemarker grammar?
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: 
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>



List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address




More information about the antlr-interest mailing list