Réf. : Re: Réf. : Re: [antlr-interest] java.lang.OutOfMemoryError

loic.lefevre at bnpparibas.com loic.lefevre at bnpparibas.com
Mon Jul 25 06:56:34 PDT 2005


Indeed, I was just finishing running all my testcases but I've found a
solution slightly similar than yours:

// Single-line comments
SL_COMMENT
      :     "//"
            (~('\uffff'|'\n'|'\r'))* ('\n'|'\r'('\n')?)?
            {$setType(Token.SKIP); newline();}
      ;

I've just added this and now it works.
I also added it here:

// string literals
STRING_LITERAL
      :     '"'! (REGEXP_ESC|~('\uffff'|'"'|'\\'|'\n'|'\r'))* '"'!
      ;

Thanks a lot!
Loïc




Internet
demakov at ispras.ru@antlr.org - 25/07/2005 15:31


Envoyé par :      antlr-interest-bounces at antlr.org



Pour : antlr-interest

cc :


Objet :     Re: Réf. : Re: [antlr-interest] java.lang.OutOfMemoryError

antlr casts -1 read from input reader:
CharBuffer.fill(i): queue.append( (char)input.read() );

to FFFF

When charVocabulary includes '\uFFFF':

charVocabulary='\u0003'..'\uFFFF';

FFFF is included in the set ~('\n'|'\r') and is successfully matched in the
loop.

You can remove FFFF from vocabulary:

charVocabulary='\u0003'..'\uFFFE';

But I don't like this solution because it doesn't handle unicode '\uFFFF'.

Regards,
Alexey

-----
Alexey Demakov
TreeDL: Tree Description Language: http://treedl.sourceforge.net
RedVerst Group: http://www.unitesk.com


----- Original Message -----
From: <loic.lefevre at bnpparibas.com>
To: <antlr-interest at antlr.org>
Sent: Monday, July 25, 2005 4:28 PM
Subject: Réf. : Re: [antlr-interest] java.lang.OutOfMemoryError



Seems I found the origin of the problem but I can't find how to resolve it
yet:

All comes from the implementation of the Reader I use, if you change your
code with:

---8
<-----------------------------------------------------------------------------------------------------------------------------------

----------
    public static void main( String[]  args ) throws TokenStreamException
    {
        RosterExpressionLexer lexer = new RosterExpressionLexer( new
java.io.StringReader("//") );
        while( true )
        {
            Token t = lexer.nextToken();
            if( t.getType() == RosterExpressionTokenTypes.EOF ) break;
            System.out.println( "'" + t.getText() + "'" );
        }
    }
---8
<-----------------------------------------------------------------------------------------------------------------------------------

----------

then you'll get the OutOfMemoryError!

Argh, I'm investigating on the behaviour of this java class!

Loïc




Internet
demakov at ispras.ru@antlr.org - 25/07/2005 13:19


Envoyé par :      antlr-interest-bounces at antlr.org



Pour : antlr-interest

cc :


Objet :     Re: [antlr-interest] java.lang.OutOfMemoryError

Here is shortened version of your parser with main() reading System.in
I don't see any problems with it

Regards,
Alexey

-----
Alexey Demakov
TreeDL: Tree Description Language: http://treedl.sourceforge.net
RedVerst Group: http://www.unitesk.com


----- Original Message -----
From: <loic.lefevre at bnpparibas.com>
To: <antlr-interest at antlr.org>
Sent: Monday, July 25, 2005 2:33 PM
Subject: [antlr-interest] java.lang.OutOfMemoryError



Hello,
My grammar to handle expressions is now finished but now I'm facing a
strange behaviour:
When I try to validate hand typed expressions, I get the terrible
java.lang.OutOfMemoryError when
parsing the expression: //    (two chars)

Here is my grammar:
(See attached file: RosterExpression.g)

It appears that the lexer loops forever:

      public final void mSL_COMMENT(boolean _createToken) throws
RecognitionException, CharStreamException, TokenStreamException {
            int _ttype; Token _token=null; int _begin=text.length();
            _ttype = SL_COMMENT;
            int _saveIndex;

            match("//");
            {
            _loop59:


//--> begin of loop forever
            do {
                  if ((_tokenSet_0.member(LA(1)))) {
                        {
                        match(_tokenSet_0);
                        }
                  }
                  else {
                        break _loop59;
                  }

            } while (true);
//--> end of loop forever


            }
            {
            switch ( LA(1)) {
            case '\n':
            {
                  match('\n');
                  break;
            }
            case '\r':
            {
                  match('\r');
                  {
                  if ((LA(1)=='\n')) {
                        match('\n');
                  }
                  else {
                  }

                  }
                  break;
            }
            default:
                  {
                  }
            }
            }
            if ( inputState.guessing==0 ) {
                  _ttype = Token.SKIP; newline();
            }
            if ( _createToken && _token==null && _ttype!=Token.SKIP ) {
                  _token = makeToken(_ttype);
                  _token.setText(new String(text.getBuffer(), _begin,
text.length()-_begin));
            }
            _returnToken = _token;
      }


Thanks in advance for your help.
Loïc

This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.

                ---------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.

(See attached file: RosterExpression.g)

This message and any attachments (the "message") is
intended solely for the addressees and is confidential.
If you receive this message in error, please delete it and
immediately notify the sender. Any use not in accord with
its purpose, any dissemination or disclosure, either whole
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message.
BNP PARIBAS (and its subsidiaries) shall (will) not
therefore be liable for the message if modified.

                ---------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le
"message") sont etablis a l'intention exclusive de ses
destinataires et sont confidentiels. Si vous recevez ce
message par erreur, merci de le detruire et d'en avertir
immediatement l'expediteur. Toute utilisation de ce
message non conforme a sa destination, toute diffusion
ou toute publication, totale ou partielle, est interdite, sauf
autorisation expresse. L'internet ne permettant pas
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce
message, dans l'hypothese ou il aurait ete modifie.





This message and any attachments (the "message") is
intended solely for the addressees and is confidential. 
If you receive this message in error, please delete it and 
immediately notify the sender. Any use not in accord with 
its purpose, any dissemination or disclosure, either whole 
or partial, is prohibited except formal approval. The internet
can not guarantee the integrity of this message. 
BNP PARIBAS (and its subsidiaries) shall (will) not 
therefore be liable for the message if modified. 

                ---------------------------------------------

Ce message et toutes les pieces jointes (ci-apres le 
"message") sont etablis a l'intention exclusive de ses 
destinataires et sont confidentiels. Si vous recevez ce 
message par erreur, merci de le detruire et d'en avertir 
immediatement l'expediteur. Toute utilisation de ce 
message non conforme a sa destination, toute diffusion 
ou toute publication, totale ou partielle, est interdite, sauf 
autorisation expresse. L'internet ne permettant pas 
d'assurer l'integrite de ce message, BNP PARIBAS (et ses
filiales) decline(nt) toute responsabilite au titre de ce 
message, dans l'hypothese ou il aurait ete modifie.


More information about the antlr-interest mailing list