[antlr-interest] Is '!' operator working with ANTLR 3.1.2

[Philippe Frankson]

I'm sorry but I'm not sure to understand what you mean by 'change the
start and end points of the token'.
Do you mean I should split my lexer this way:

fragment INT 	: ('0'..'9');
fragment ALPHA    : ('a'..'z'|'A'..'Z'|'_');

ID	: ALPHA (ALPHA | INT)*;
EXT	: 'EXT.';

rule: EXT ID '(' { stack.push(new FuncName($ID.getText())); }
(expressList { ... } )? ')'	expressList: ...;


In fact I cannot do that because I have the following lexer rule (I
didn't put it in my previous mail to make it lighter):

IDENTIFIER		: ID (('.' ID)=> '.' ID { $type = CELLNAME; })*;

Because of that rule, 'EXT.Myfunc' would return the token CELLNAME
instead of EXT + ID;

Now, of course, like you said, I can strip the characters out but I
thought it was possible to do that at lexer side and that would have
been better.

Thank you for your feedback.
Philippe Frankson

-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Jim Idle
Sent: 17 November 2010 23:24
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Is '!' operator working with ANTLR 3.1.2

In the lexer, but the context of the question tells you this. The !
operator
in the parser is used to prevent being auto written in to the tree on an
AST
producing parser. That is not the same thing.

Ignoring the token in the parser is not the same thing as it allows
whitespace and any other characters that are not passed to the parser to
be
placed in between.

All you need do is change the start and end points of the token or just
strip the characters out when you come to need the string value of the
token.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Colin Macdonald
> Sent: Wednesday, November 17, 2010 12:43 PM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Is '!' operator working with ANTLR 3.1.2
> 
> Jim,
> 
> Don't you mean that the ! operator is not supported in the Lexer?
> 
> Phil would need a parser rule to discard the token.  Something like:
> 
> //Convert ID from a fragment to a token
> ID: ALPHA (ALPHA | INT)*;
> EXT: 'EXT.';
> external_call: EXT! ID;
> 
> rule: call=external_call '(' { stack.push(new
>         FuncName($call.text)); } (expressList { ... } )? ')'
>         expressList: ...;
> 
> 
> 
> Colin Macdonald
> Senior Consultant
> _____________________________________
> 
> GROUP Business Software
> Phone: 770 720 1300 ext: 6132
> 
> http://www.gbs.com/
> 
> 
> 
> 
> From:   "Jim Idle" <jimi at temporal-wave.com>
> To:     <antlr-interest at antlr.org>
> Date:   2010-11-17 11:19 AM
> Subject:        Re: [antlr-interest] Is '!' operator working with
ANTLR
> 3.1.2
> Sent by:        antlr-interest-bounces at antlr.org
> 
> 
> 
> Remember antlr.markmail.org
> 
> This operator is not supported in ANTLR3 for performance reasons. But
> if the pieces you don't want are at the start or end, then you can
just
> change the start and/or end points of the token.
> 
> Jim
> 
> > -----Original Message-----
> > From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> > bounces at antlr.org] On Behalf Of Philippe Frankson
> > Sent: Wednesday, November 17, 2010 4:51 AM
> > To: antlr-interest at antlr.org
> > Subject: [antlr-interest] Is '!' operator working with ANTLR 3.1.2
> >
> >
> > When I call my parser with -> EXT.MyFunc() The value returned by
> > $EXTERNAL_CALL.getText() is 'EXT.MyFunc' where I would expect to
have
> > only 'MyFunc' (because I'm using the '!' operator in the lexer here
> > below).
> >
> > fragment INT                           : ('0'..'9');
> > fragment ALPHA                                 :
> ('a'..'z'|'A'..'Z'|'_');
> > fragment ID                                            : ALPHA
(ALPHA
> |
> INT)*;
> >
> > EXTERNAL_CALL: 'EXT.'! ID;
> >
> >
> > rule: EXTERNAL_CALL '(' { stack.push(new
> > FuncName($EXTERNAL_CALL.getText())); } (expressList { ... } )? ')'
> > expressList: ...;
> >
> >
> > Any idea why the '!' operator looks like not working ? What am I
> doing
> > wrong ?
> >
> > Thank you.
> > Philippe Frankson
> >
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe: http://www.antlr.org/mailman/options/antlr-
> interest/your-
> > email-address
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> 
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address


List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address