[antlr-interest] Re: Help with Java grammar

lgcraymer lgc at mail1.jpl.nasa.gov
Mon Mar 8 15:16:42 PST 2004


Removing the newlines from the // rule is probably the right move for now--the NEWLINE rule should be set to "SKIP".  You, however, 
have uncovered an ANTLR bug--there is no way to match EOF in the lexer as far as I can see:  InputBuffer absorbs the EOF_CHAR.  
Alternatively, you can do something like replacing the newline at the end with

(     { LA(1) == EOF_CHAR}?
      |  <usual "\r\n" and alternatives>
)

and modifying the "match anything but newline" as well.  It's also possible that the language spec says you must end a file with a 
newline--that used to be true of C--but I don't think that that is the case.

--Loring


--- In antlr-interest at yahoogroups.com, "cliftonccraig" <ccc at i...> wrote:
> Hello all,
> 
> I am a novice at ANTLR and have managed to get a Java source code
> augmenting tool working after reading the article on
> TokenStreamRewriting. What I'm having trouble with is the Java grammar
> that I've modified from the ANTLR site. In order to get my augmenter
> working I had to modify the JavaLexer and tell it not to ignore
> whitespace and single line comments. I needed these tokens to flow
> through to the rewrite engine. Now I have a scenario where a Java
> source file is terminated with a single line comment"//". This seems
> to confuse the Lexer and cause an OutOfMemory error. Could one of you
> gurus help me tweak the grammar to avoid this? I'm not sure if I
> should code an alternate for the single line comment rule like this:
> 
> // Single-line comments
> SL_COMMENT
> 	:	"//"
> 		(~('\n'|'\r'))* ('\n'|'\r'('\n')?)
> 		{
> //*CCC- Allow comments to flow through to the rewrite engine
> //		    $setType(Token.SKIP);
> 		    newline();
> 		}
>                 |
>                 "//" EOF
> 	;
> or even if that makes sense. I was also considering removing the
> newline characters from the definition but I don't know what kind of
> side effects that would have on the grammar. I'm just too nervous to
> disturb the original grammar too much without know what's going on. Help! 
> 
> Thanks in advance,
> Cliff



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list