[antlr-interest] EOF in Lexer- how to?

[Tomasz Jastrzebski]

  Hi,
   
  You are 100% right.
  What I described is the problem I need a solution for.
  That is; I want SingleLineComment to terminate either at the NewLine or <EOF>.
  In another words: I would like to be able to define NewLine as:
  NewLine :(options{greedy=true;}:"\r\n" | '\r' | '\n' )  | EOF;
but I do not how to do it - the above definition will not work.
   
-Tomasz

Bryan Ewbank <ewbank at gmail.com> wrote:  Hi,

It does not directly address your question, but I think the "//"
comment is defined to terminate at the next newline character;
therefore, the situation you describe means that the comment flows
into the next file.

Of course, the user might like to know if that happens, but it's not
quite the same as the situation you describe.

- Bryan

On 1/16/06, Tomasz Jastrzebski wrote:
> Is it possible to recognize EOF in the lexer?
>
> Ok, why would someone wanted to do it in the first place?
> Lets suppose I want my lexer to recognize a SingleLineComment, let's say
> Java "// com ment" style. My lexer rules should look more or less like this:
> NewLine :(options{greedy=true;}:"\r\n" | '\r' | '\n' ) ;
> SingleLineComment :"//" ( ~('\r' | '\n') )* NewLine ;
>
> But there is a problem here. What if my input stream consists of only single
> comment and no NewLine? E.g.
> // comment text 
> This lexer will not recognize such an input correctly.
> That is why I w ant my lexer to be able to treat EOF as NewLine.



		
---------------------------------
Yahoo! Photos
 Ring in the New Year with Photo Calendars. Add photos, events, holidays, whatever.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20060116/9fa709a0/attachment.html