[antlr-interest] Re: Is there an ANTLR trick/hack to specify "NEWLINE or EOF" in Lexer

[Terence Parr]

On Tuesday, February 4, 2003, at 04:39 PM, micheal_jor 
<open.zone at virgin.net> wrote:

> --- In antlr-interest at yahoogroups.com, Terence Parr <parrt at j...>
> wrote:
>> Hi Michael,
>>
>> What if you just made NEWLINE normal and then subclass your lexer
> and
>> override nextToken so that you detect the EOF_TYPE coming in the
> token
>> and replace it with a NEWLINE token?
>
> I did considered this option. At the time I felt it was equivalent to
> simply appending a NEWLINE to all input files automatically. I think
> I'll go with this now since Ric suggested something similar too.
> Thanks guys.
>
> Incidentally, what's your opinion of point (2) below. You know, about
> ANTLR supporting a "virtual EOF char" that Lexers can match in rules.
>
> NEWLINE
> :  '\n'
> |  '\r' ('\n')?
> |  EOF                 // or $EOF or $eof
> ;

Hmm....yeah, I'm not sure.  What character would it be?  We already use 
(char)-1 in Java, which I think is wrong since 0xFFFF is a valid char 
in some script.  Any unicode geniuses out there?

Ter
--
Co-founder, http://www.jguru.com
Creator, ANTLR Parser Generator: http://www.antlr.org
Lecturer in Comp. Sci., University of San Francisco


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/