[antlr-interest] inversion problems: unreachable statements & lex failures

[girard_chandler]

I've tried to scan the archives and web for similar problems but
I haven't found any solutions.  Apologies if these are known problems.

1st, the following produces code which can't compile:

        class P extends Parser;
        startRule :  TEXT ( COMMA TEXT )* ;

        class L extends Lexer;
        TEXT : (~',')* ;
        COMMA : ",";

Gives:

+ java antlr.Tool -trace t.g
ANTLR Parser Generator   Version 2.7.2   1989-2003 jGuru.com
warning: public lexical rule TEXT is optional (can match "nothing")
+ javac L.java Main.java P.java PTokenTypes.java
L.java:100: unreachable statement
                        if ( _createToken && _token==null &&
_ttype!=Token.SKIP ) {
                        ^
1 error


This error can be eliminated by lexing a bogus keyword which overlaps
the TEXT token.  However, the result fails to run:

        class P extends Parser;
        options { buildAST=true; }
        startRule :  a:TEXT
                { System.out.println("a:" + a.getText()); }
                ( COMMA b:TEXT
                { System.out.println("b:" + b.getText()); }
                )*
                ;

        class L extends Lexer;
        options { k=2; }
        TEXT : (~',')+ ;
        COMMA : ",";
        HACK: ",==";

When run, with input "a,b,c", I get:

java Main
 > startRule; a,b,c
exception: line 1:1: unexpected char: 'a'
line 1:1: unexpected char: 'a'
        at L.nextToken(L.java:78)
        at antlr.TokenBuffer.fill(TokenBuffer.java:69)
        at antlr.TokenBuffer.LT(TokenBuffer.java:86)
        at antlr.LLkParser.LT(LLkParser.java:56)
        at antlr.LLkParser.trace(LLkParser.java:66)
        at antlr.LLkParser.traceIn(LLkParser.java:78)
        at P.startRule(P.java:56)
        at Main.main(Main.java:8)

This can be partially fixed by not using an inverse rule, and
substituting the following for TEXT:

TEXT : ('a'..'z'|'A'..'Z')+;

Then the above example works fine.  However, I want to read all
possible
characters, except for ','.

Can this be done with Antlr?

Thanks,

Girard

p.s.
Partial java output for the 1st example:
L.java:
        public final void mTEXT(boolean _createToken) throws
RecognitionException, CharStreamException, TokenStreamException {
                int _ttype; Token _token=null; int
_begin=text.length();
                traceIn("mTEXT");
                _ttype = TEXT;
                int _saveIndex;
                try { // debugging

                        {
                        _loop6:
                        do {
                                {
                                        matchNot(',');
                                }

                        } while (true);
                        }
                        if ( _createToken && _token==null &&
_ttype!=Token.SKIP ) {
                                _token = makeToken(_ttype);
                                _token.setText(new
String(text.getBuffer(), _begin, text.length()-_begin));
                        }
                        _returnToken = _token;
                } finally { // debugging
                        traceOut("mTEXT");
                }
        }

p.p.s:
My Main.java:
import java.io.*;
class Main {
    public static void main(String[] args) {
        try {
            L lexer = new L(new DataInputStream(System.in));
            P parser = new P(lexer);
            parser.startRule();
        } catch(Exception e) {
            System.err.println("exception: "+e);
            e.printStackTrace();
        }
    }
}


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/