[antlr-interest] Explanation about EOF needed
Jim Idle
jimi at temporal-wave.com
Sun Dec 7 09:07:45 PST 2008
On Sun, 2008-12-07 at 18:35 +0300, `VL wrote:
> Hello.
Having a newline at the end of a file defines a text file in UNIX, not
having one is basically creating a special condition that you have to
rejig your grammar to accommodate. I find that it generally isn't worth
it and just append a newline to the input source so that it will always
have one. This is much simpler.
Jim
>
> I'm trying to include comments information into my grammar and have
> some problems.
> Below is the minimal example i have:
>
> ========== Cut =================
> grammar simple;
>
> // the whole thing: all stuff till the end of file
> prog : commented_kw+ EOF;
>
> // Keyword with optional comment before it or just comment or empty line
> commented_kw: COMMENT* varspec* NL;
>
> // key=val
> varspec: TKN '=' (TKN);
>
> // token for keys and values
> TKN: ('a'..'z'|'A'..'Z'| '0'..'9')+;
>
> // newline separator
> NL : '\n' ;
>
> // whitespace are thrown away
> WS: (' '|'\t'| '\r')+ { $channel=HIDDEN; };
>
> // comment goes till end of line, including it
> COMMENT : (' '|'\t')* ( '#' (~'\n')* ) NL;
> ========== Cut =================
>
> Here is the input (generic configuration file):
>
> ========== Cut =================
>
>
> # This is one-line comment (global for file)
>
> # This is many one-line comments (global)
> # This is many one-line comments (global)
> # This is many one-line comments (global)
> # This is many one-line comments (global)
>
> # This is comment to var
> var=value
>
> # This is comment to many vars
> var=value var=value var=value
>
> var=value
> # After-var comment
>
> # Pre-Var 1
> # Pre-Var 2
> # Pre-Var 3
> var=value
> # After-Var 1
> # After-Var 2
>
>
> # LAST LINE - no newline at the END
> ========== Cut =================
>
> Everything goes fine, except i'm getting
> MismatchedTokenException(0!=0) on the last line.
>
> Well, using such grammar it is expected error, since my comment is
> defined to end with NL
> and it can't be found.
>
> I tried to modify COMMENT lexer rule to end with (NL|EOF). No effect.
> But i have one more
> required NL, in commented_kw rule, i changed it to (NL|EOF), and got
> error about unreachable
> alternative in 'prog' rule. I tried a lot of more variants, trying to
> move COMMENT rule into parser
> level and more, but i can't understand what's going on.
>
> I'm satisfied with first variant of grammar, but it gives me an error.
> How should i modify it to
> make it work? Where is my error when i tried to achive my goals?
>
> Thanks.
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20081207/71a224f4/attachment.html
More information about the antlr-interest
mailing list