[antlr-interest] Can lex token return value?

[Jim Idle]

ANTLR3 does not allow lexer rules to return values I am afraid, but generally you won't need to do this. If you really want to do it (though it is probably just as easy to do this in the parser, but there are varying opinions on this ;-), then try:


@lexer::members {
 public static int value;
}

DECIMAL	:	('0d')?('0'..'9') ('_'? '0'..'9')* {value = parseInt($text);};

Jim

-----Original Message-----
From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of femtowin1
Sent: Wednesday, April 18, 2007 8:15 PM
To: antlr-interest at antlr.org
Subject: [antlr-interest] Can lex token return value?

Hi, I was trying to define the following grammar:

int
	:	'-'?(DECIMAL|HEX|BINARY|OCTAL | ESCAPE_INT) {token = createIntToken();}
	;

DECIMAL returns [int value]	:	('0d')?('0'..'9') ('_'? '0'..'9')* {$value = parseInt($text);};

HEX	:	...;
BINARY	:	...;
OCTAL	:	...;
ESCAPE_INT
	:       ...  
	;

the generated Lexer file is:

public final void mDECIMAL() throws RecognitionException {
        int value = 0;

        try {
            int _type = DECIMAL;
            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:31: ( ( '0d' )? ( '0' .. '9' ) ( ( '_' )? '0' .. '9' )* )
            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:31: ( '0d' )? ( '0' .. '9' ) ( ( '_' )? '0' .. '9' )*
            {
            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:31: ( '0d' )?
            int alt3=2;
            int LA3_0 = input.LA(1);

            if ( (LA3_0=='0') ) {
                int LA3_1 = input.LA(2);

                if ( (LA3_1=='d') ) {
                    alt3=1;
                }
            }
            switch (alt3) {
                case 1 :
                    // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:32: '0d'
                    {
                    match("0d"); 


                    }
                    break;

            }

            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:38: ( '0' .. '9' )
            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:39: '0' .. '9'
            {
            matchRange('0','9'); 

            }

            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:49: ( ( '_' )? '0' .. '9' )*
            loop5:
            do {
                int alt5=2;
                int LA5_0 = input.LA(1);

                if ( ((LA5_0>='0' && LA5_0<='9')||LA5_0=='_') ) {
                    alt5=1;
                }


                switch (alt5) {
            	case 1 :
            	    // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:50: ( '_' )? '0' .. '9'
            	    {
            	    // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:50: ( '_' )?
            	    int alt4=2;
            	    int LA4_0 = input.LA(1);

            	    if ( (LA4_0=='_') ) {
            	        alt4=1;
            	    }
            	    switch (alt4) {
            	        case 1 :
            	            // C:\\toolset\\ruby\\xruby-trunk\\src\\com\\xruby\\compiler\\parser\\Rubyv3.g:162:50: '_'
            	            {
            	            match('_'); 

            	            }
            	            break;

            	    }

            	    matchRange('0','9'); 

            	    }
            	    break;

            	default :
            	    break loop5;
                }
            } while (true);

            value = parseInt(getText());

            }

            this.type = _type;
        }
        finally {
        }

it seems can't return value from Lex, is that true?
______________________________________
Sent from my www.pageflakes.com startpage