[antlr-interest] Context-Sensitive Scanning proposal

["Paul Bouché (NSN)"]

Scott Stanchfield schrieb:
> Hey all - I've written up a little proposal for a context-sensitive
> scanning idea. Hopefully my brain is on a good track with this; I
> think it could be a major win for ANTLR.
>
> Please check out
>    http://javadude.com/articles/antlr-context-sensitive-scanner.html
>
> and let me know what you think.
>
> Looks like something similar has been done before (rats! thought I had
> an original thought!).
>
> Thoughts?
> -- Scott
>
> ----------------------------------------
> Scott Stanchfield
> http://javadude.com
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>   
Hi Scott,

sounds like very much what I would have needed when I extended the 
parser for our DSL to allow syntax coloring and auto completion as well 
as some tricky lexer issues (in some cases what is usually part of an 
identifier is not but an extra token). I ended up building a custom 
state in the lexer and write semantic predicates in ANTLR for that rule. 
This made a lexer grammar look pretty ugly and the advantage of 
easy-to-ready-ANTLR grammars was lost.... maybe I should have used the 
yapp parser generator. Anyway my current opinion on this issue is that I 
would always prefer a context sensitive lexer which already builds the 
structures I need (no parser, no ast) for speed reasons and for 
flexiblity reasons, of course then one does not have automatic error 
recovery, but if ANTLR was build differently one would have error 
recovery. Let's look ahead for ANTLR v4 and error recovery in lexers.

BR,
Paul