[antlr-interest] inversion problems: unreachable statements & lex failures
girard_chandler
girard_chandler at yahoo.com
Tue Sep 9 13:42:36 PDT 2003
I've tried to scan the archives and web for similar problems but
I haven't found any solutions. Apologies if these are known problems.
1st, the following produces code which can't compile:
class P extends Parser;
startRule : TEXT ( COMMA TEXT )* ;
class L extends Lexer;
TEXT : (~',')* ;
COMMA : ",";
Gives:
+ java antlr.Tool -trace t.g
ANTLR Parser Generator Version 2.7.2 1989-2003 jGuru.com
warning: public lexical rule TEXT is optional (can match "nothing")
+ javac L.java Main.java P.java PTokenTypes.java
L.java:100: unreachable statement
if ( _createToken && _token==null &&
_ttype!=Token.SKIP ) {
^
1 error
This error can be eliminated by lexing a bogus keyword which overlaps
the TEXT token. However, the result fails to run:
class P extends Parser;
options { buildAST=true; }
startRule : a:TEXT
{ System.out.println("a:" + a.getText()); }
( COMMA b:TEXT
{ System.out.println("b:" + b.getText()); }
)*
;
class L extends Lexer;
options { k=2; }
TEXT : (~',')+ ;
COMMA : ",";
HACK: ",==";
When run, with input "a,b,c", I get:
java Main
> startRule; a,b,c
exception: line 1:1: unexpected char: 'a'
line 1:1: unexpected char: 'a'
at L.nextToken(L.java:78)
at antlr.TokenBuffer.fill(TokenBuffer.java:69)
at antlr.TokenBuffer.LT(TokenBuffer.java:86)
at antlr.LLkParser.LT(LLkParser.java:56)
at antlr.LLkParser.trace(LLkParser.java:66)
at antlr.LLkParser.traceIn(LLkParser.java:78)
at P.startRule(P.java:56)
at Main.main(Main.java:8)
This can be partially fixed by not using an inverse rule, and
substituting the following for TEXT:
TEXT : ('a'..'z'|'A'..'Z')+;
Then the above example works fine. However, I want to read all
possible
characters, except for ','.
Can this be done with Antlr?
Thanks,
Girard
p.s.
Partial java output for the 1st example:
L.java:
public final void mTEXT(boolean _createToken) throws
RecognitionException, CharStreamException, TokenStreamException {
int _ttype; Token _token=null; int
_begin=text.length();
traceIn("mTEXT");
_ttype = TEXT;
int _saveIndex;
try { // debugging
{
_loop6:
do {
{
matchNot(',');
}
} while (true);
}
if ( _createToken && _token==null &&
_ttype!=Token.SKIP ) {
_token = makeToken(_ttype);
_token.setText(new
String(text.getBuffer(), _begin, text.length()-_begin));
}
_returnToken = _token;
} finally { // debugging
traceOut("mTEXT");
}
}
p.p.s:
My Main.java:
import java.io.*;
class Main {
public static void main(String[] args) {
try {
L lexer = new L(new DataInputStream(System.in));
P parser = new P(lexer);
parser.startRule();
} catch(Exception e) {
System.err.println("exception: "+e);
e.printStackTrace();
}
}
}
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list