[antlr-interest] Re: Help with Java grammar
cliftonccraig
ccc at icsaward.com
Tue Mar 9 05:55:26 PST 2004
Thank you Loring,
I tried the following to no avail:
// Single-line comments
SL_COMMENT
: "//"
(~('\n'|'\r'))* ('\n'|'\r'('\n')?)
{
//*CCC- Allow comments to flow through to the rewrite engine
// $setType(Token.SKIP);
newline();
}
|
"//" { LA(1) == EOF_CHAR}?
;
I'm still struggling. I'm not sure if I understand how to do this.
Cliff
--- In antlr-interest at yahoogroups.com, "lgcraymer" <lgc at m...> wrote:
> Removing the newlines from the // rule is probably the right move
for now--the NEWLINE rule should be set to "SKIP". You, however,
> have uncovered an ANTLR bug--there is no way to match EOF in the
lexer as far as I can see: InputBuffer absorbs the EOF_CHAR.
> Alternatively, you can do something like replacing the newline at
the end with
>
> ( { LA(1) == EOF_CHAR}?
> | <usual "\r\n" and alternatives>
> )
>
> and modifying the "match anything but newline" as well. It's also
possible that the language spec says you must end a file with a
> newline--that used to be true of C--but I don't think that that is
the case.
>
> --Loring
>
>
> --- In antlr-interest at yahoogroups.com, "cliftonccraig" <ccc at i...> wrote:
> > Hello all,
> >
> > I am a novice at ANTLR and have managed to get a Java source code
> > augmenting tool working after reading the article on
> > TokenStreamRewriting. What I'm having trouble with is the Java grammar
> > that I've modified from the ANTLR site. In order to get my augmenter
> > working I had to modify the JavaLexer and tell it not to ignore
> > whitespace and single line comments. I needed these tokens to flow
> > through to the rewrite engine. Now I have a scenario where a Java
> > source file is terminated with a single line comment"//". This seems
> > to confuse the Lexer and cause an OutOfMemory error. Could one of you
> > gurus help me tweak the grammar to avoid this? I'm not sure if I
> > should code an alternate for the single line comment rule like this:
> >
> > // Single-line comments
> > SL_COMMENT
> > : "//"
> > (~('\n'|'\r'))* ('\n'|'\r'('\n')?)
> > {
> > //*CCC- Allow comments to flow through to the rewrite engine
> > // $setType(Token.SKIP);
> > newline();
> > }
> > |
> > "//" EOF
> > ;
> > or even if that makes sense. I was also considering removing the
> > newline characters from the definition but I don't know what kind of
> > side effects that would have on the grammar. I'm just too nervous to
> > disturb the original grammar too much without know what's going
on. Help!
> >
> > Thanks in advance,
> > Cliff
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list