[antlr-interest] distinction between newline and ws

[Jim Idle]

Basically, you don’t do that :-)

Your WS rule is just:

WS : (' '|'\t')+ { $type = HIDDEN; };

And NEWLINE is then

NEWLINE : '\r'? '\n' ;

There is not context, so you choose which you want. Now, this is only true
if you want NEWLINE to be seen by the parser, but WS to be hidden from it.
IF you don't want either, then just delete the NEWLINE rule and leave \n and
\r in the WS rule so that they are all discarded.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Clifford Heath
> Sent: Saturday, October 20, 2007 5:45 PM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] distinction between newline and ws
> 
> Sven Busse wrote:
> > something. The grammar looks like this:
> > grammar Expr;
> ...
> > NEWLINE     :     '\r'? '\n';
> > WS    :     (' '|'\t'|'\n'|'\r')+ {skip();};
> > My Question now is, how does antrl know, that “\n” should match to a
> > NEWLINE instead
> 
> I have some understanding of this, but I'm still struggling with
> a similar case also.
> 
> Your grammar parses the string "a=3\n" just fine, but
> the string "a=3 \n" (with space) doesn't parse... why?
> 
> If I redefine NEWLINE as (' '|'\t')* '\r'? '\n'
> it doesn't parse either form... why?
> 
> > I would have thought, this grammar is ambiguous, but apparantly, it
> isn’t. Why not?
> 
> Lexical rules are often ambiguous, and all rules always apply
> regardless of the grammar rule context (this is the answer to
> Simon West's question). The Lexer is supposed to choose the
> longest token that matches the current input... but I don't
> see that principle applying here.
> 
> Clifford Heath.
> 
> 
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.5.488 / Virus Database: 269.15.3/1081 - Release Date:
> 10/19/2007 5:41 PM
> 

No virus found in this outgoing message.
Checked by AVG Free Edition. 
Version: 7.5.488 / Virus Database: 269.15.3/1081 - Release Date: 10/19/2007
5:41 PM
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20071020/83213880/attachment.html