[antlr-interest] Simple Grammar Question

[John Gardener]

    *Jim* hi

    Thank you for you answer; worked as you described;

    I will get the book today;

    Probably this needs to be the first thing on the "*5 minute intro*"
    antlr page:

    *"Lexer rules must be unambiguous or be just 'fragment' rules
    (basically called by other lexer rules).
    The lexer runs all the way through the text creating tokens, and
    when it is entirely complete, the parser is given the token set.
    The parser does not drive the lexer."
    *
    Especially: *The parser does not drive the lexer.*

    Thanks,

    John

-------- Original Message  --------
Subject: Re: [antlr-interest] Simple Grammar Question
From: Jim Idle <jimi at temporal-wave.com>
To: antlr-interest at antlr.org
Cc: antlr-interest at antlr.org
Date: Sat 17 Jan 2009 11:38:15 AM CST
> John Gardener wrote:
>>
>>     *Gerald*, hi;
>>
>>     Thank you very much for your response;
>>
>>     YES, you are right, I can see that: " Your NAME rule is consuming
>>     all tokens. "
>>
>>     What I can not see is how to *correct this properly*?
>>
>>     Thanks again,
>>
>>     John
>>
>>>>
>>>>     grammar Simple;  
>>>>
>>>>     options {
>>>>         language = Java;
>>>>     }
>>>>
>>>>     @parser::header {
>>>>         package simple;
>>>>         import static java.lang.System.out;
>>>>     }
>>>>
>>>>     @lexer::header{
>>>>       package simple;
>>>>       import static java.lang.System.out;
>>>>     }
>>>>
>>>>     // PARSER
>>>>
>>>>     record :
>>>>       digit name EOF
>>>>       { out.println( "+record: " +  $text );  };
>>>>
>>>>     digit : DIGIT
>>>>       { out.println( "+digit: " +  $text );  };
>>>>      
>>>>     name : (DIGIT|LETTER)
>>>>       ;
>>>>
> Change the name parser rule as above.
>>>>
>>>>
>>>>     // LEXER
>>>>
>>>>     DIGIT : '0'..'9'
>>>>       { out.println("+DIGIT: " + $text ); } ;
>>>>
>>>>     LETTER : 'A'..'Z'
>>>>       { out.println("+LETTER: " + $text ); } ;
>>>>
>>
> Remove the NAME lexer rule. Lexer rules must be unambiguous or be just
> 'fragment' rules (basically called by other lexer rules). The lexer
> runs all the way through the text creating tokens, and when it is
> entirely complete, the parser is given the token set. The parser does
> not drive the lexer, so the lexer cannot know whether to return you a
> NAME token or a LETTER token. Have a read of the getting started
> articles on the Wiki - most of these new user issues are covered and
> it will help you a lot. If you have the money, buy a copy of the book too.
>
> Jim
>> ------------------------------------------------------------------------
>>
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>   
>
> ------------------------------------------------------------------------
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090117/3464644d/attachment.html