[antlr-interest] whats wrong in this very very simple grammar ?

Gokulakannan Somasundaram gokul007 at gmail.com
Sun Mar 4 16:04:51 PST 2012


As Jim has pointed out, the better performing solution would be

 rule: stat? EOF ;
 stat: ID;
 fragment LETTER : ('a'..'z' | 'A'..'Z') ;
 ID: LETTER+ ;
 WS: (' ' | '\t' | '\n' | '\r' | '\f')+ {$channel = HIDDEN;} ;

This would reduce the number of tokens created.

Gokul.

On Mon, Mar 5, 2012 at 6:57 AM, Jim Idle <jimi at temporal-wave.com> wrote:

> Refer to my original reply. This will tokenism every letter. You just need
> + and not * in ID.
>
> Jim
>
> On Mar 4, 2012, at 13:13, Gokulakannan Somasundaram <gokul007 at gmail.com>
> wrote:
>
> > Keep it as a rule of thumb to make the Lexer simple. This should work.
> >
> > rule: stat EOF ;
> > stat: ID*;
> > fragment LETTER : ('a'..'z' | 'A'..'Z') ;
> > ID: LETTER ;
> > WS: (' ' | '\t' | '\n' | '\r' | '\f')+ {$channel = HIDDEN;} ;
> >
> > Thanks,
> > Gokul.
> >
> > On Sun, Mar 4, 2012 at 6:26 AM, Borat Borat <nukethenuke at yahoo.com>
> wrote:
> >
> >> Hi,
> >>
> >> I have very simple grammar, and I'm using antlr 3.3:
> >>
> >> rule: stat EOF ;
> >> stat: ID;
> >> fragment LETTER : ('a'..'z' | 'A'..'Z') ;
> >> ID: LETTER* ;
> >> WS: (' ' | '\t' | '\n' | '\r' | '\f')+ {$channel = HIDDEN;} ;
> >>
> >> And test rig is :
> >> ANTLRInputStream input = new ANTLRInputStream(System.in);
> >> TestLexer lexer = new TestLexer(input);
> >> CommonTokenStream tokens = new CommonTokenStream(lexer);
> >> TestParser parser = new TestParser(tokens);
> >> parser.rule();
> >>
> >>
> >> Now, the problem, when I input, for example, abc and hit Ctrl + Z, I get
> >> error:
> >> line 1:0 mismatched input '<EOF>' expecting ID
> >>
> >> However, if I type abc for input and then hit enter and then press
> >> Ctrl + Z everything executes as expected.
> >>
> >> I was wondering if someone could explain me what is exactly happening
> here,
> >> I fail to see why first scenario fails ?
> >>
> >> Thanks
> >>
> >>
> >> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> >> Unsubscribe:
> >> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
> >>
> >
> > List: http://www.antlr.org/mailman/listinfo/antlr-interest
> > Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>


More information about the antlr-interest mailing list