[antlr-interest] newlines in v3 (codegen, runtime solutions)

Terence Parr parrt at cs.usfca.edu
Sat Nov 26 11:40:52 PST 2005


Howdy,

My philosophy behind v3 is that it should do by default what most  
people will want it to do.  Most people just want it to work and are  
not that concerned with speed.  Those that are concerned have the  
flexibility to do amazing things by changing templates so that  
overhead is reduced and method calls are eliminated etc...  For  
example, here is how I generate token references for the Java target:

/** match a token optionally with a label in front */
tokenRef(token,label,elementIndex) ::= <<
<if(label)>
<label>=(<labelType>)input.LT(1);<\n>
<endif>
match(input,<token>,FOLLOW_<token>_in_<ruleName><elementIndex>);
 >>

For references with += labels, I just do this:

/** ids+=ID */
tokenRefAndListLabel(token,label,elementIndex) ::= <<
<tokenRef(...)>
<listLabel(...)>
 >>

listLabel(label) ::= <<
if (list_<label>==null) list_<label>=new ArrayList();
list_<label>.add(<label>);<\n>
 >>

So, if you want, you can totally inline the invocation of match.   
Most people won't want this.  If you do, then change it.   You will  
even be able to do it within the grammar file itself by specifying  
these overridden templates! :) All this w/o recompiling/rebuilding  
ANTLR.

On to newlines.  I'm not sure this is worth arguing about as along as  
you can make it do what you want.  I check for \n automatically as  
that is what most people will want.  In my stream classes I have:

public void consume() {
     if ( p < data.length ) {
         charPositionInLine++;
         if ( data[p]=='\n' ) {
             line++;
             charPositionInLine=0;
         }
         p++;
     }
}

If you want to avoid that data[p] test for newline, I'm pretty sure  
people will know how to subclass the stream and override.  I could  
even provide a default one.

For binary, I'm sure people will simply implement my CharStream and  
be done with it. :)

Anyway, don't worry folks.  You will be able to do just about  
anything you want either by changing code generation or by tweaking  
some classes. :)

Ter


More information about the antlr-interest mailing list