[antlr-interest] Token stream filter

Anthony Youngman Anthony.Youngman at ECA-International.com
Wed Jun 2 06:40:38 PDT 2004


Except yours is (clearly?) a *lexer* rule. I need a *parser* rule.

The problem is I can only tell if REM means a comment once I've started
parsing. Because it is also a function name, and can be a variable name.
There's various ways of doing it, but the obvious way to use a filter
AFTER the lexer has done its work.

Cheers,
Wol 

-----Original Message-----
From: Anakreon [mailto:anakreonmejdi at yahoo.gr] 
Sent: 02 June 2004 14:19
To: antlr-interest at yahoogroups.com
Subject: Re: [antlr-interest] Token stream filter

Anthony Youngman wrote:
Hello Anthony.
This is the whay I did it although the comments are lost so it might not
be what 
you need.

In the rule which matches identifiers i have this code:
     if (_ttype == REM) {
         mLINE(false);
         _ttype = STATEMENT_END;
     }
The  mLINE is generated by the rule:
protected LINE : (~'\n')* '\n' {newline();};
> This sounds like it should be very easy, but I can't see how to do it
> ...
> 
> I want to filter the token stream coming out of the lexer (and I can't
> see how to do it using the lexer's filter - can I do the same thing in
> the parser?). Anyways, I want to match the following sequence
> 
> (newline | semi) ("rem" | "*" | "!" ) ~( "=" | "(" ) ({greedy=false}
.)
> newline)
> 
> eating everything EXCEPT the newline, which needs to be passed through
> to the parser. Note also, that while "*" and "!" are tokens, "rem" is
an
> ident. Of course, if I need "rem" to be a token, presumably if this
rule
> fails I can convert it back to an ident?
> 
> Or do I need to write my own token stream to do this (if so, where do
I
> look for an example?)
> 
> Cheers,
> Wol
> 


 
Yahoo! Groups Links



 



****************************************************************************

This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system.

Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333.

****************************************************************************



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list