[antlr-interest] How do I use a custom token object? Wiki FAQ sugestion does not seem to work ...

Uwe Lammers Uwe.Lammers at sciops.esa.int
Tue Aug 26 08:51:08 PDT 2008


Hi,

I want the lexer to return a custom token object with a special field 'x'.
I tried the Wiki suggestion

http://www.antlr.org/wiki/pages/viewpage.action?pageId=1844

but this fails with a compilation problem in

public Token emit() {
    MyToken t =
          new MyToken(input, type, channel,
                    tokenStartCharIndex, getCharIndex()-1);

       ...

the compiler complains that 'type', 'channel' etc are undefined ...

I fixed this in the following manner:

public Token emit() {
      Token t0 = super.emit();
      int type = t0.getType();
      int channel = t0.getChannel();
      int tokenStartLine = t0.getLine();
      String text = t0.getText();
      int tokenStartCharPositionInLine = t0.getCharPositionInLine();
      int tokenStartCharIndex = t0.getTokenIndex();
    MyToken t =
        new MyToken(input, type, channel,
                    tokenStartCharIndex, getCharIndex()-1);
    ...

which then compiles and runs ok. Can somebody confirm that this is 
ok/and or suggest
a better way to do this? If yes, I can update the Wiki.


So, that's one problem solved. The next question is, how can I access my 
custom
field in the parser rules?

 From something like

arith: t=NLITERAL { here I want to access t.x };

antlr (3.1) generates

t=(Token)match(input,NLITERAL,FOLLOW_NLITERAL_in_arith615);

and Token does not know anything about the x field.
Solution is probably obvious but I cannot see it.

Thanks
Uwe


================================================================================================
This message and any attachments are intended for the use of the addressee or addressees only. The
unauthorised disclosure, use, dissemination or copying (either in whole or in part) of its content
is prohibited. If you received this message in error, please delete it from your system and notify
the sender. E-mails can be altered and their integrity cannot be guaranteed. ESA shall not be liable
for any e-mail if modified.
=================================================================================================



More information about the antlr-interest mailing list