[antlr-interest] EOF in Lexer- how to?
Bryan Ewbank
ewbank at gmail.com
Mon Jan 16 01:57:08 PST 2006
Hi,
It does not directly address your question, but I think the "//"
comment is defined to terminate at the next newline character;
therefore, the situation you describe means that the comment flows
into the next file.
Of course, the user might like to know if that happens, but it's not
quite the same as the situation you describe.
- Bryan
On 1/16/06, Tomasz Jastrzebski <tdjastrzebski at yahoo.com> wrote:
> Is it possible to recognize EOF in the lexer?
>
> Ok, why would someone wanted to do it in the first place?
> Lets suppose I want my lexer to recognize a SingleLineComment, let's say
> Java "// com ment" style. My lexer rules should look more or less like this:
> NewLine :(options{greedy=true;}:"\r\n" | '\r' | '\n' ) ;
> SingleLineComment :"//" ( ~('\r' | '\n') )* NewLine ;
>
> But there is a problem here. What if my input stream consists of only single
> comment and no NewLine? E.g.
> // comment text <EOF>
> This lexer will not recognize such an input correctly.
> That is why I w ant my lexer to be able to treat EOF as NewLine.
More information about the antlr-interest
mailing list