[antlr-interest] Problems with string literals as tokens in parser rules

chr_mathis chriss007 at gmx.de
Tue Oct 12 08:18:58 PDT 2004




Hi there.

I've got the following problem: I would like to use a 'direct' token
in a parser rule. The documentation says:

"In parser rules, strings represent tokens, and each unique string is
assigned a token type. However, ANTLR does not create lexer rules to
match the strings. Instead, ANTLR enters the strings into a literals
table in the associated lexer. ANTLR will generate code to test the
text of each token against the literals table, and change the token
type when a match is encountered before handing the token off to the
parser..."

So far so good.
If I got this right, I may write a grammar like this:

================ grammar ==========================

class MyParser extends Parser;
options {
   k=1;
}
{
  //for the access to the lexer inside the parser
  private MyLexer lexer = null;
      
  public MyParser(MyLexer lexer, boolean overloadDummy) {
    this((TokenStream)lexer);
     this.lexer = lexer;
  } 
}
                  
entry :
    ("abc" | DUMMY)
  ;

class MyLexer extends Lexer;
options {
  k=3;
}
DUMMY 
:
  "ugh"
  ;
======================== end ===========================

So 'abc' as well as 'ugh' should be matched, right?
In a main class I call the method for the 'entry'-Rule... 
If I enter 'abc' I get the follwoing message:

line 1:1: unexpected char: 'a'
        at MyLexer.nextToken(MyLexer.java:71)
        at antlr.TokenBuffer.fill(TokenBuffer.java:69)
        at antlr.TokenBuffer.LA(TokenBuffer.java:80)
        at antlr.LLkParser.LA(LLkParser.java:52)
        at MyParser.entry(MyParser.java:54)
        at Main.main(Main.java:14)

Am I making a terrible mistake?

Thanks in advance,
Christian.

Ps: Used antlr 2.7.4
And here is the Main-Class:

===================== Main =============================
import java.io.*;

public class Main {
  public static void main(String argv[]) {
    try {
      MyLexer lexer = new MyLexer(new BufferedInputStream(System.in));
      MyParser parser = new MyParser(lexer, true);
      parser.entry();
    }
    catch(Exception ex) {
      ex.printStackTrace();
    }
  }
}
===================== end ==============================








 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
    antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 





More information about the antlr-interest mailing list