[antlr-interest] 4.0 daily builds

[Terence Parr]

On Jan 1, 2012, at 1:05 PM, Eric wrote:

> Ter,
> 
> 
>> The most obvious differences with v4 are:
>> 
>> * (directly) left recursive grammars that works great for expressions
>> * modes in the lexer
>> 
> 
> What are modes and how do they work. I noticed them looking at the antlr
> parser grammar, but could not find any *.g4 file that gave me any help.

Hi Eric, I'm working on an XML grammar that will demonstrate things nicely. They work well when you have multiple languages within the same file such as inside and outside XML tags. One can also consider the stuff inside of strings to be a different language than the outside. so here is a simple example that treats the 2 differently

lexer grammar L;

STRING_START : '"' {pushMode(STRING_MODE); more();} ;
WS : ' '|'
' {skip();} ;

mode STRING_MODE;
STRING : '"' {popMode();} ;
ANY : . {more();} ;

We start out in default mode and when it sees a doublequote it switches to the string mode and asks the lexer to go get more input. Because we asked for more, lexer looks for more matches and matches a bunch of stuff to ANY and keeps looking.  It's only when we see the final double quote that we pop the modes and return an actual token.

This mode stuff is  ancient as far as I can tell. For example, I see another tool doing it 

http://quex.sourceforge.net/doc/html/basics/modes.html

> 
> 
>> * the lexer rules behave as you'd expect lexer rules to behave now
>> * it takes any grammar you give it that isn't indirectly left recursive;
>> there are no static grammar analysis errors
> 
> 
> There seems to be more steps performed using *.g4 grammars such a tree
> validation and refactoring. Could you elaborate on them.

v4 generates the listener interface and a blank listener for you automatically. To use it, you create a listener and override the method you want and then pass it to the parser. Or, create a visitor that triggers the events on the resulting parse tree. take a look at TestRig because it shows you how to print out the tree and display it after parsing.

Ter