[antlr-interest] Token stream filter

Anthony Youngman Anthony.Youngman at ECA-International.com
Thu Jun 3 01:24:07 PDT 2004


Thanks. Actually, Monty's solution should work ...

but seeing as you seem to know these things, taking this line from my
original post

	(id:IDENT {if text != "REM" throw tokenmatchexception}|"*"|"!")

which is the exception I need to throw here?

I think if I've got this, I've got enough to write my filter :-) While
the LB() function might be useful, further thought on what Monty said
made me think it might not be needed.

So - I can feed the lexer output into my deremer parser - and I can then
feed the output from that into my main parser?

And if I have a rule like

commentst : (EOL | SEMI) ("*" | "!")! (~(EOL)*)! ;

it will then eat everything between the initial eol/semi and final eol,
but it will let those two tokens through to the next parser?

Cheers,
Wol

-----Original Message-----
From: Ric Klaren [mailto:klaren at cs.utwente.nl] 
Sent: 02 June 2004 16:46
To: antlr-interest at yahoogroups.com
Subject: Re: [antlr-interest] Token stream filter

On Wed, Jun 02, 2004 at 03:37:15PM +0100, Anthony Youngman wrote:
> If I don't want to eat up the newline at the end, is the following
> likely to be a good/sensible parser rule?
> 
> commentst : REMARK ( (LA(1) != newline) => . )* ;
> 
> in other words, having found a REMARK, eat everything up to but not
> including the next newline. Or is LA a lexer-only thing as well?

That one is a bit inefficient due to an exception per character:

This is a dirty one but it works:

comment: REMARK ( { if( LA(1) == NEWLINE ) break; } : . )* ;

Tip: read the generated code.

Cheers,

Ric
-- 
-----+++++*****************************************************+++++++++
-------
    ---- Ric Klaren ----- j.klaren at utwente.nl ----- +31 53 4893755  ----
-----+++++*****************************************************+++++++++
-------
 'And this 'rebooting' business? Give it a good kicking, do you?' 'Oh,
no,
  of course, we ... that is ... well, yes, in fact,' said Ponder.
'Adrian
    goes round the back and ... er ... prods it with his foot. But in a
     technical way,' he added. --- From: Hogfather by Terry Pratchett.


****************************************************************************

This transmission is intended for the named recipient only. It may contain private and confidential information. If this has come to you in error you must not act on anything disclosed in it, nor must you copy it, modify it, disseminate it in any way, or show it to anyone. Please e-mail the sender to inform us of the transmission error or telephone ECA International immediately and delete the e-mail from your information system.

Telephone numbers for ECA International offices are: Sydney +61 (0)2 8272 5300, Hong Kong + 852 2121 2388, London +44 (0)20 7351 5000 and New York +1 212 582 2333.

****************************************************************************



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list