[antlr-interest] Re: Help with Java grammar

cliftonccraig ccc at icsaward.com
Tue Mar 9 06:08:09 PST 2004


I just tried this:
SL_COMMENT
	:	"//"
		(~('\n'|'\r'))* ('\n'|'\r'('\n')?)
		{
//*CCC- Allow comments to flow through to the rewrite engine
//		    $setType(Token.SKIP);
		    newline();
		}
		|
		"//" (~('\n'|'\r'))*
	;

And I got an warning saying:
D:\scm\tools\parsers\grammar\ANTLR\java.g:1235: warning:lexical
nondeterminism between alts 1 and 2 of block upon
D:\scm\tools\parsers\grammar\ANTLR\java.g:1235:     k==1:'/'
D:\scm\tools\parsers\grammar\ANTLR\java.g:1235:     k==2:'/'
D:\scm\tools\parsers\grammar\ANTLR\java.g:1235:    
k==3:'\u0003'..'\t','\u000b','\u000c','\u000e'..'\uffff'
D:\scm\tools\parsers\grammar\ANTLR\java.g:1235:    
k==4:<end-of-token>,'\u0003'..'\t','\u000b','\u000c','\u000e'..'\uffff'

I think I'm really lost. Help!

Cliff
--- In antlr-interest at yahoogroups.com, "cliftonccraig" <ccc at i...> wrote:
> Thank you Loring,
> 
> I tried the following to no avail:
> // Single-line comments
> SL_COMMENT
> 	:	"//"
> 		(~('\n'|'\r'))* ('\n'|'\r'('\n')?)
> 		{
> //*CCC- Allow comments to flow through to the rewrite engine
> //		    $setType(Token.SKIP);
> 		    newline();
> 		}
>         |
>                 "//" { LA(1) == EOF_CHAR}?
> 	;
> 
> I'm still struggling. I'm not sure if I understand how to do this.
> 
> Cliff
> 
> 
> --- In antlr-interest at yahoogroups.com, "lgcraymer" <lgc at m...> wrote:
> > Removing the newlines from the // rule is probably the right move
> for now--the NEWLINE rule should be set to "SKIP".  You, however, 
> > have uncovered an ANTLR bug--there is no way to match EOF in the
> lexer as far as I can see:  InputBuffer absorbs the EOF_CHAR.  
> > Alternatively, you can do something like replacing the newline at
> the end with
> > 
> > (     { LA(1) == EOF_CHAR}?
> >       |  <usual "\r\n" and alternatives>
> > )
> > 
> > and modifying the "match anything but newline" as well.  It's also
> possible that the language spec says you must end a file with a 
> > newline--that used to be true of C--but I don't think that that is
> the case.
> > 
> > --Loring
> > 
> > 
> > --- In antlr-interest at yahoogroups.com, "cliftonccraig" <ccc at i...>
wrote:
> > > Hello all,
> > > 
> > > I am a novice at ANTLR and have managed to get a Java source code
> > > augmenting tool working after reading the article on
> > > TokenStreamRewriting. What I'm having trouble with is the Java
grammar
> > > that I've modified from the ANTLR site. In order to get my augmenter
> > > working I had to modify the JavaLexer and tell it not to ignore
> > > whitespace and single line comments. I needed these tokens to flow
> > > through to the rewrite engine. Now I have a scenario where a Java
> > > source file is terminated with a single line comment"//". This seems
> > > to confuse the Lexer and cause an OutOfMemory error. Could one
of you
> > > gurus help me tweak the grammar to avoid this? I'm not sure if I
> > > should code an alternate for the single line comment rule like this:
> > > 
> > > // Single-line comments
> > > SL_COMMENT
> > > 	:	"//"
> > > 		(~('\n'|'\r'))* ('\n'|'\r'('\n')?)
> > > 		{
> > > //*CCC- Allow comments to flow through to the rewrite engine
> > > //		    $setType(Token.SKIP);
> > > 		    newline();
> > > 		}
> > >                 |
> > >                 "//" EOF
> > > 	;
> > > or even if that makes sense. I was also considering removing the
> > > newline characters from the definition but I don't know what kind of
> > > side effects that would have on the grammar. I'm just too nervous to
> > > disturb the original grammar too much without know what's going
> on. Help! 
> > > 
> > > Thanks in advance,
> > > Cliff



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list