[antlr-interest] behaviour of lexer

Philippe Frankson Philippe.Frankson at Frsglobal.com
Tue Mar 2 10:13:07 PST 2010


Thank you for your answer, Bart.

But if I do what you are suggesting, my parser rules will look like this:
intexpr:		INTTOKEN arithexpr CB { ... };

Instead of this (which is more readable):
intexpr:		INTTOKEN OB arithexpr CB { ... };

But, ok, I have a solution to make it work.
But my questions were more about understanding how it is working internally.
Why having 2 lines in the lexer makes it different as having a single line ?
AT              : '@';
NAME            : ALPHA (ALPHA | DIGIT)*;   ...

Instead of

ATNAME: 		: '@' NAME;


Regards,
Philippe


-----Original Message-----
From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Bart Kiers
Sent: 02 March 2010 19:01
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] behaviour of lexer

On Tue, Mar 2, 2010 at 5:25 PM, Philippe Frankson <
Philippe.Frankson at frsglobal.com> wrote:

> ...
> @int('444') is a function converting a string into integer.
> If I don't have parentheses, then it is not a function, it is only a column
> name. Ex.: @test, @integer, @in, ....
>
> Here is a part of lexer rules:
>
> fragment DIGIT  : ('0'..'9');
> fragment ALPHA  : ('a'..'z'|'A'..'Z'|'_');
>
> OB              : '(';
> INTTOKEN        : ('@int' OB)=> '@int'; // so I check if there is an open
> parenthesis to return INTTOKEN.
> AT              : '@';
> NAME            : ALPHA (ALPHA | DIGIT)*;   ...
>

Why not just include the OB in your lexer rule?
Something like this:

INT_METHOD      : AT 'int' OB; // or: AT 'int' OB STRING CB;
COLUMN          : AT NAME;

OB              : '(';
AT              : '@';
NAME            : ALPHA (ALPHA | DIGIT)*;

fragment DIGIT  : ('0'..'9');
fragment ALPHA  : ('a'..'z'|'A'..'Z'|'_');

and because lexer rules are matched from top to bottom, '@int' will be
matched as 'AT NAME'.

Regards,

Bart.

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address


More information about the antlr-interest mailing list