[antlr-interest] EOF in Lexer- how to?

[Bryan Ewbank]

Hi,

It does not directly address your question, but I think the "//"
comment is defined to terminate at the next newline character;
therefore, the situation you describe means that the comment flows
into the next file.

Of course, the user might like to know if that happens, but it's not
quite the same as the situation you describe.

- Bryan

On 1/16/06, Tomasz Jastrzebski <tdjastrzebski at yahoo.com> wrote:
> Is it possible to recognize EOF in the lexer?
>
> Ok, why would someone wanted to do it in the first place?
> Lets suppose I want my lexer to recognize a SingleLineComment, let's say
> Java "// com  ment"  style. My lexer rules should look more or less like this:
> NewLine :(options{greedy=true;}:"\r\n" | '\r' | '\n' ) ;
> SingleLineComment :"//" ( ~('\r' | '\n') )* NewLine ;
>
> But there is a problem here. What if my input stream consists of only single
> comment and no NewLine? E.g.
> // comment text <EOF>
> This lexer will not recognize such an input correctly.
> That is why I w  ant my  lexer to be able to treat EOF as NewLine.