[antlr-interest] Token stream filter

Anthony Youngman Anthony.Youngman at ECA-International.com
Thu Jun 3 00:09:44 PDT 2004


Great! Talk about stating the bleeding obvious ... :-)

Actually, I was going to ask you and Ter something ...

Having looked at your filter stuff, I don't remember seeing a LB() (look
before) function. Is there one, and if not, how easy would it be to
implement and add to Antlr?

While it might not be used much, it seems to me to be perfect for
dealing with this "is it an identifier or token" problem. Rather than
the mods you made to NextToken and so on, we could then simply have a
rule

commentst : {LB(1) == EOL || LB(1) == SEMI} (
	("*" | "!") ...
	| id:IDENT {if id.getText != "REM" throw recognition-exception}
...

Okay, handling "REM" would be messy :-) Antlr's rule system is great for
dealing with tokens having different meanings when they follow other
stuff WITHIN a rule, but as in this case it doesn't always work when one
of the permitted positions is the first token in the rule ...

Cheers,
Wol

-----Original Message-----
From: Monty Zukowski [mailto:monty at codetransform.com] 
Sent: 02 June 2004 18:10
To: antlr-interest at yahoogroups.com
Cc: Monty Zukowski
Subject: Re: [antlr-interest] Token stream filter

On Jun 2, 2004, at 7:37 AM, Anthony Youngman wrote:

> commentst : REMARK ( (LA(1) != newline) => . )* ;

What's wrong with this?

commentst: REMARK (~(NEWLINE))* ;

Monty Zukowski

ANTLR & Java Consultant -- http://www.codetransform.com
ANSI C/GCC transformation toolkit -- 
http://www.codetransform.com/gcc.html
Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html


****************************************************************************

This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system.

Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333.

****************************************************************************



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list