[antlr-interest] Behavior of LA(n) at EOF?

[Terence Parr]

On Oct 2, 2004, at 11:04 AM, Richard Clark wrote:
> I'm trying to get the antlr/eclipse bridge working and I've hit a
> glitch. I had to create an adaptor that uses an Eclipse document as an
> input source, and I'm running into an infinite loop in the lexer.

Hi Richard,

Looking forward a great deal to your presentation on this next week at 
the workshop!  Woohoo!

> The lexer construct
>
> COMMENT: "//" (~('\n' | '\r'))* EOLN;
>
> generates
>
> match("//");
> {
> _loop10:
> do {
>    if ((_tokenSet_0.member(LA(1))) {
>      { match(_token_set_0); }
>    }
>    else {
>      break _loop10;
>    }
> } while (true);
>
> The problem is, if LA(1) returns CharScanner.EOF_CHAR before the
> comment is done (before an EOLN), the member(LA(1)) call returns true
> and the loop repeats forever.

Really?  Hmm...Oh, EOF is a member of "not newline".  Hmm...probably 
stupid on my part; I believe I do NOT do this in my 3.0 prototype.  Ok, 
try this:

CMT : "//" (options {greedy=false;}:.)* '\n' ;

which generates:

                 _loop3:
                 do {
                         // nongreedy exit test
                         if ((LA(1)=='\n')) break _loop3;
                         if (((LA(1) >= '\u0001' && LA(1) <= '\u00ff'))) 
{
                                 matchNot(EOF_CHAR);
                         }
                         else {
                                 break _loop3;
                         }

                 } while (true);

The set test will fail for EOF and break out of the loop and then fail 
to match EOLN.

Let me/antlr-interest know if this works. :)

Ter

>
> Thus, what is the proper behavior for InputBuffer::LA(int i) when i is
> at or past the end of the file? Should it return EOF_CHAR (in which
> case, what should I do with the loop above) or throw an exception?
>
>   ...Richard (directions: apply forehead to wall. repeat.)
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
--
CS Professor & Grad Director, University of San Francisco
Creator, ANTLR Parser Generator, http://www.antlr.org
Cofounder, http://www.jguru.com
Cofounder, http://www.knowspam.net enjoy email again!





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/