[antlr-interest] Lexer issues when block ends with EOF instead of EOL

[Loring Craymer]

No--this is a factoring issue.  Unless your parser needs to see EOL tokens, they should be in a separate rule and marked as "hidden".  If you take out the
'\r/? '\n' from this rule, I expect that your grammar will work (provided that you separately recognize newlines).

--Loring



________________________________
From: Brent Yates <brent.yates at gmail.com>
To: "antlr-interest at antlr.org" <antlr-interest at antlr.org>
Sent: Wednesday, February 11, 2009 9:52:23 AM
Subject: [antlr-interest] Lexer issues when block ends with EOF instead of EOL

Assuming a standard LINE comment form such as:

SL_COMMENT
    : '//'  ( options {greedy=false;} : . )*  '\r'? '\n' {$channel=HIDDEN;}
    ;

What is the best way to handle a file which ends with a single line comment but no EOL
If I add the EOF to the rule I get the following error:

SL_COMMENT
    : '//'  ( options {greedy=false;} : . )*  '\r'? ('\n'|EOF) {$channel=HIDDEN;}
    ;

ANTLR Parser Generator  Version 3.1.1
error(201): SystemVerilogLexer.g:592:43: The following alternatives can never be matched: 1

This problem occurs with other rules as well.  Is it expected that files which end with no EOL are bad or should the lexer handle it?

Thanks.

Brent Yates


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090213/899bd176/attachment.html