[antlr-interest] Problems with string literals as tokens in parser rules

[Monty Zukowski]

Look at the documentation for literals.  You need to have a rule that 
will match identifiers and then test the literals table for this scheme 
to work.

ANTLR & Java Consultant -- http://www.codetransform.com
ANSI C/GCC transformation toolkit -- 
http://www.codetransform.com/gcc.html
Embrace the Decay -- http://www.codetransform.com/EmbraceDecay.html

On Oct 12, 2004, at 8:18 AM, chr_mathis wrote:

>
>
>
> Hi there.
>
> I've got the following problem: I would like to use a 'direct' token
> in a parser rule. The documentation says:
>
> "In parser rules, strings represent tokens, and each unique string is
> assigned a token type. However, ANTLR does not create lexer rules to
> match the strings. Instead, ANTLR enters the strings into a literals
> table in the associated lexer. ANTLR will generate code to test the
> text of each token against the literals table, and change the token
> type when a match is encountered before handing the token off to the
> parser..."
>
> So far so good.
> If I got this right, I may write a grammar like this:
>
> ================ grammar ==========================
>
> class MyParser extends Parser;
> options {
>    k=1;
> }
> {
>   //for the access to the lexer inside the parser
>   private MyLexer lexer = null;
>
>   public MyParser(MyLexer lexer, boolean overloadDummy) {
>     this((TokenStream)lexer);
>      this.lexer = lexer;
>   }
> }
>
> entry :
>     ("abc" | DUMMY)
>   ;
>
> class MyLexer extends Lexer;
> options {
>   k=3;
> }
> DUMMY
> :
>   "ugh"
>   ;
> ======================== end ===========================
>
> So 'abc' as well as 'ugh' should be matched, right?
> In a main class I call the method for the 'entry'-Rule...
> If I enter 'abc' I get the follwoing message:
>
> line 1:1: unexpected char: 'a'
>         at MyLexer.nextToken(MyLexer.java:71)
>         at antlr.TokenBuffer.fill(TokenBuffer.java:69)
>         at antlr.TokenBuffer.LA(TokenBuffer.java:80)
>         at antlr.LLkParser.LA(LLkParser.java:52)
>         at MyParser.entry(MyParser.java:54)
>         at Main.main(Main.java:14)
>
> Am I making a terrible mistake?
>
> Thanks in advance,
> Christian.
>
> Ps: Used antlr 2.7.4
> And here is the Main-Class:
>
> ===================== Main =============================
> import java.io.*;
>
> public class Main {
>   public static void main(String argv[]) {
>     try {
>       MyLexer lexer = new MyLexer(new BufferedInputStream(System.in));
>       MyParser parser = new MyParser(lexer, true);
>       parser.entry();
>     }
>     catch(Exception ex) {
>       ex.printStackTrace();
>     }
>   }
> }
> ===================== end ==============================
>
>
>
>
>
>
>
>
>
> Yahoo! Groups Links
>
>
>
>
>
>
>
>
>



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/