[antlr-interest] Lexer issues when block ends with EOF instead ofEOL

Brent Yates brent.yates at gmail.com
Wed Feb 11 11:09:21 PST 2009


Thanks for the comments guys.  I like Jim's idea of adding an EOL to the
input source.  I can do that very easily and then I don't have to fix all my
block rules in the lexer.  Line comments were just an easy example, but
there are several more places where the same thing happens.
Regards,

Brent Yates

On Wed, Feb 11, 2009 at 2:05 PM, Sam Harwell <sharwell at pixelminegames.com>wrote:

>  This works because lexer tokens are greedy:
>
>
>
> LINE_COMMENT
>
>         :       '//' (~('\n'|'\r'))* '\r'? '\n'?
>
>                 { $channel=HIDDEN; }
>
>         ;
>
>
>
> Sam
>
>
>
> *From:* antlr-interest-bounces at antlr.org [mailto:
> antlr-interest-bounces at antlr.org] *On Behalf Of *Brent Yates
> *Sent:* Wednesday, February 11, 2009 11:52 AM
> *To:* antlr-interest at antlr.org
> *Subject:* [antlr-interest] Lexer issues when block ends with EOF instead
> ofEOL
>
>
>
> Assuming a standard LINE comment form such as:
>
>
>
> SL_COMMENT
>
>     : '//'  ( options {greedy=false;} : . )*  '\r'? '\n' {$channel=HIDDEN;}
>
>     ;
>
>
>
> What is the best way to handle a file which ends with a single line comment
> but no EOL?
>
>
>
> If I add the EOF to the rule I get the following error:
>
>
>
> SL_COMMENT
>
>     : '//'  ( options {greedy=false;} : . )*  '\r'? ('\n'|EOF)
> {$channel=HIDDEN;}
>
>     ;
>
>
>
> ANTLR Parser Generator  Version 3.1.1
>
> error(201): SystemVerilogLexer.g:592:43: The following alternatives can
> never be matched: 1
>
>
>
> This problem occurs with other rules as well.  Is it expected that files
> which end with no EOL are bad or should the lexer handle it?
>
>
>
> Thanks.
>
>
>
> Brent Yates
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090211/dc00680a/attachment.html 


More information about the antlr-interest mailing list