To Sinan --> RE: [antlr-interest] Simple questions
mzukowski at bco.com
mzukowski at bco.com
Mon Mar 4 06:26:03 PST 2002
You don't have a '!' in your ID rule, so you would never match connect!
because you never match the !. The way literals work is they are a table of
words and token numbers. At the end of any rule with testLiterals=true on
the table is consulted and the token type is changed if there was one in
the table. You don't need k=8, you need your ID rule to have all the
characters that are possible in the literals table.
ID
optons { testLiterals = true; }
: (LETTER|UNDERSCORE)
(LETTER|UNDERSCORE|MINUS|DOT|DIGIT|'!')*
;
Monty
> -----Original Message-----
> From: Eli Mesika [mailto:elim at tti-telecom.com]
> Sent: Sunday, March 03, 2002 8:11 AM
> To: 'antlr-interest at yahoogroups.com'
> Subject: To Sinan --> RE: [antlr-interest] Simple questions
>
>
> Thanks for your quick respond.
>
> Since I have another lexer rule :
>
> ID
> : (LETTER|UNDERSCORE)
> (LETTER|UNDERSCORE|MINUS|DOT|DIGIT)*
> ;
>
> and I have added the following :
>
> CATCH_LITERAL_TOKENS: "connect!"
>
> I have to use k=8 in the lexer options.
>
> Isn't that expensive ???
> I used till now k=2 for my grammar, is there any other
> solution that doesn't
> force me to increase k ???
>
> Thanks
>
>
> -----Original Message-----
> From: Sinan [mailto:sinan.karasu at boeing.com]
> Sent: Thursday, February 28, 2002 7:53 PM
> To: antlr-interest at yahoogroups.com
> Subject: Re: [antlr-interest] Simple questions
>
>
> Eli Mesika wrote:
> >
> > HI folks
> >
> > I have a real problem in using '!' in the Antlr Tokens section.
> > I have tried Matthew's solution, it doesn't work.
> >
> > Attached please find connect.g which describes the
> "connect" language.
> > The language can recognize the following statement:
> > (connect a b)
> > However, the exact command is
> > (connect! a b)
> > if you are generating java files from it compiling and
> running Main it
> will
> > accept (connect a b)
> > However if you change the CONNECT token in the .g file to
> "connect!" and
> > test (connect! a b) you
> > will get an error.
> >
> > Am I missing something ???
> >
> > Please HELP.
> > Thanks
>
>
> The way nextToken works in lexer, you have to have a rule that accepts
> the token before a hash
> table is consulted for keywords.
>
> so do in the lexer
>
> CATCH_LITERAL_TOKENS: ( "connect!" | "whatever_else_i_want!@#") ;
>
> so on...
>
> you HAVE TO HAVE testLiterals=true; for this to work.....
>
>
> I think nextToken should check the keyword table after consuming
> until a special token ( such as WS ) is seen. But both Ter and Ric are
> pretty busy these days.....
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
>
>
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/
>
>
>
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list