[antlr-interest] simple grammar question

John B. Brodie jbb at acm.org
Fri Mar 2 18:55:44 PST 2012


Greetings!

On 03/02/2012 09:37 PM, Michael Roberts wrote:
> Hi folks.
>
> I'm in the process of learning antlr (I just ordered Terrance's books), but
> there something I am missing here.  The below compiles fine using
> AntlrWorks, but when I feed it:
>
> foo/aa/bb
>
> I get a a MissingTokenException (see attached bitmap), after it's matched
> an identifier and a /.

I do not use ANTLRWorks so this assessment may be bogus, but...

Do you see that the "MissingTokenException" node is a child of "identifier"?

I believe that ANTLRWorks is trying to tell you  that "identifier" was 
not found.


In any case. Regardless of how to analyze the bitmap.

Because your IdentifierPart and IdentifierStart Lexer rules are 
fragments, your Parser will never see them (that is the intent of 
fragment lexer rules; e.g. define a portion, fragment, of a token but 
keep it hidden from the parser). So your identifier rule, as written, is 
useless; e.g. will never recognize anything.

I would simply (with out any deep analysis of your example) suggest that 
you turn your `identifier` Parser rule into a Lexer rule by capitalizing 
it (identifier ==> Identifier).


As an aside, I would avoid the backtrack=true; option until and unless 
you **really** understand what that will do to your parser's performance 
(but that is, perhaps, just my own personal bias ;-).

Hope this helps...
    -jbb

>
> grammar JLG2;
>
> options {
>     backtrack=true;
>     memoize=true;
> }
>
> @header {
> package org.veve.reflect.interpreter.output;
> }
>
> @lexer::header {
> package org.veve.reflect.interpreter.output;
> }
>
> compilationUnit : relativePath;
>
> identifier : IdentifierStart IdentifierPart* ;
> relativePath : identifier (SLASH identifier)* ;
>
> SLASH : '/';
>
> fragment IdentifierPart
>      :
>      'a'..'z' | 'A'..'Z' | '_' | '0'..'9'
>      ;
>
> fragment IdentifierStart
>      :
>      'a'..'z' | 'A'..'Z' | '_'
>      ;
>
> My expectation there was that identifier would match an alphanumeric
> string, and that the rule for relativePath would let me string such
> identifiers together with / in between.  However, I'm clearly missing
> something obvious.  Thanks in advance.
>
> MR
>
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address



More information about the antlr-interest mailing list