[antlr-interest] ambigous lexer tokens

[Jim Idle]

The example python parser shows this because it needs it to make the
ridiculous context sensitive whitespace easier to parse.

Jim 

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Gavin Lambert
> Sent: Thursday, June 28, 2007 5:25 AM
> To: Wincent Colaiuta; Torsten Curdt
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] ambigous lexer tokens
> 
> At 22:20 28/06/2007, Wincent Colaiuta wrote:
>  >Well, here's one (untested) idea: one way might be to modify
> your
>  >WS rule to increment a "column" counter whenever a run of spaces
> 
>  >is seen; you'd have to set up the column counter in your
>  >@lexer::members section (exactly how you set up and initialize
>  >that variable is dependent on your target language):
>  >
>  >   WS : ' '+ { column++; };
> 
> Actually, I'm pretty sure that v3 tracks the column for you, so
> there's no need to do it manually.  How you get hold of it varies
> by target language, though, but it's usually a property/field of
> the lexer class, I think.
> 
> In fact I vaguely remember there being an example script for a
> column-sensitive language (Fortran, maybe?  Not sure) either in
> the v3 examples bundle or on the Wiki.
> 
> (Sorry for being so vague; it's after midnight.)