Re: Réf. : Re: [antlr-interest] java.lang.OutOfMemoryError

Alexey Demakov demakov at ispras.ru
Mon Jul 25 06:31:24 PDT 2005


antlr casts -1 read from input reader:
CharBuffer.fill(i): queue.append( (char)input.read() );

to FFFF

When charVocabulary includes '\uFFFF':

charVocabulary='\u0003'..'\uFFFF';

FFFF is included in the set ~('\n'|'\r') and is successfully matched in the loop.

You can remove FFFF from vocabulary:

charVocabulary='\u0003'..'\uFFFE';

But I don't like this solution because it doesn't handle unicode '\uFFFF'.

Regards,
Alexey

-----
Alexey Demakov
TreeDL: Tree Description Language: http://treedl.sourceforge.net
RedVerst Group: http://www.unitesk.com


----- Original Message ----- 
From: <loic.lefevre at bnpparibas.com>
To: <antlr-interest at antlr.org>
Sent: Monday, July 25, 2005 4:28 PM
Subject: Réf. : Re: [antlr-interest] java.lang.OutOfMemoryError



Seems I found the origin of the problem but I can't find how to resolve it
yet:

All comes from the implementation of the Reader I use, if you change your
code with:

---8
<-----------------------------------------------------------------------------------------------------------------------------------
----------
    public static void main( String[]  args ) throws TokenStreamException
    {
        RosterExpressionLexer lexer = new RosterExpressionLexer( new
java.io.StringReader("//") );
        while( true )
        {
            Token t = lexer.nextToken();
            if( t.getType() == RosterExpressionTokenTypes.EOF ) break;
            System.out.println( "'" + t.getText() + "'" );
        }
    }
---8
<-----------------------------------------------------------------------------------------------------------------------------------
----------

then you'll get the OutOfMemoryError!

Argh, I'm investigating on the behaviour of this java class!

Loïc




Internet
demakov at ispras.ru@antlr.org - 25/07/2005 13:19


Envoyé par :      antlr-interest-bounces at antlr.org



Pour : antlr-interest

cc :


Objet :     Re: [antlr-interest] java.lang.OutOfMemoryError

Here is shortened version of your parser with main() reading System.in
I don't see any problems with it

Regards,
Alexey

-----
Alexey Demakov
TreeDL: Tree Description Language: http://treedl.sourceforge.net
RedVerst Group: http://www.unitesk.com


----- Original Message -----
From: <loic.lefevre at bnpparibas.com>
To: <antlr-interest at antlr.org>
Sent: Monday, July 25, 2005 2:33 PM
Subject: [antlr-interest] java.lang.OutOfMemoryError



Hello,
My grammar to handle expressions is now finished but now I'm facing a
strange behaviour:
When I try to validate hand typed expressions, I get the terrible
java.lang.OutOfMemoryError when
parsing the expression: //    (two chars)

Here is my grammar:
(See attached file: RosterExpression.g)

It appears that the lexer loops forever:

      public final void mSL_COMMENT(boolean _createToken) throws
RecognitionException, CharStreamException, TokenStreamException {
            int _ttype; Token _token=null; int _begin=text.length();
            _ttype = SL_COMMENT;
            int _saveIndex;

            match("//");
            {
            _loop59:


//--> begin of loop forever
            do {
                  if ((_tokenSet_0.member(LA(1)))) {
                        {
                        match(_tokenSet_0);
                        }
                  }
                  else {
                        break _loop59;
                  }

            } while (true);
//--> end of loop forever


            }
            {
            switch ( LA(1)) {
            case '\n':
            {
                  match('\n');
                  break;
            }
            case '\r':
            {
                  match('\r');
                  {
                  if ((LA(1)=='\n')) {
                        match('\n');
                  }
                  else {
                  }

                  }
                  break;
            }
            default:
                  {
                  }
            }
            }
            if ( inputState.guessing==0 ) {
                  _ttype = Token.SKIP; newline();
            }
            if ( _createToken && _token==null && _ttype!=Token.SKIP ) {
                  _token = makeToken(_ttype);
                  _token.setText(new String(text.getBuffer(), _begin,
text.length()-_begin));
            }
            _returnToken = _token;
      }


Thanks in advance for your help.
Loïc

This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.

                ---------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.

(See attached file: RosterExpression.g)

This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.

                ---------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.




More information about the antlr-interest mailing list