[antlr-interest] exception unexpected char in simple parser
earlsinclair8888
earlsinclair8888 at yahoo.com
Tue Mar 23 11:25:40 PST 2004
Hello,
I am encountering a problem while constructing a simple test parser,
for an gSQL likeh language. I would appreciate any help
that I could
get solving this. I created the parser/lexer using antlr.tool and
used a test file for input which looks like this:
select field1, field2, field3
from table1;
The trace output appears to being what I would expect the parser to
be doing, except that once it reaches eof it produces the following
message:
exception: E:\SQLJava\Test1.txt:6:1: unexpected char: 0x?F
The last line of input is line 5. I assume this is an EOF problem on
my part (?). Being an amateur at this, perhaps someone can point me
in the right direction.
Thanks for any help.
John Parke
Pertinent Info:
class SqlParser extends Parser;
options
{
exportVocab = Sql;
k = 2;
buildAST = false;
defaultErrorHandler = false;
}
start_rule: select_command (SEMI)? EOF! ;
select_command: select_statement;
select_statement:
( OPEN_PAREN select_command CLOSE_PAREN ) => OPEN_PAREN
select_command CLOSE_PAREN
| select_expression ;
select_expression:
select_clause from_clause (where_clause)?;
select_clause: SELECT ( ALL )? select_list;
from_clause: FROM table_name ;
where_clause: WHERE where_condition;
select_list: field_name_list
| ASTERISK;
field_name_list: field_name ( COMMA field_name )* ;
table_name: IDENTIFIER;
where_condition: condition ;
field_name: IDENTIFIER ;
c rest omittedc
class SqlLexer extends Lexer;
options {
exportVocab = Sql;
testLiterals = false;
k = 2;
caseSensitive = false;
caseSensitiveLiterals = false;
charVocabulary = '\u0000'..'\uFFFE';
}
tokens
{
SELECT = "select";
ALL = "all";
FROM = "from";
WHERE = "where";
OR = "or";
AND = "and";
NOT = "not";
}
IDENTIFIER options {testLiterals=true;} :
'a' .. 'z' ( 'a' .. 'z' | '0' .. '9' | '_' | '$' )* ;
// quoted_string
// ::= "'" { "any_character" } "'"
QUOTED_STRING: '\'' ( ~'\'' )* '\'' ;
SEMI: ';' ;
DOT: '.' ;
COMMA: ',' ;
ASTERISK: '*' ;
AT_SIGN: '@' ;
OPEN_PAREN: '(';
CLOSE_PAREN: ')';
EQ: '=' ;
LE: "<=";
NOT_EQ: "!=" | "^=";
GT: '>';
GE: ">=";
QUOTE: '\'';
NUMBER: (SIGN UINTEGER) => INTEGER
|(UINTEGER '.' UINTEGER) => DECIMAL
|(SIGN UINTEGER '.' UINTEGER) => DECIMAL
|('.' (DIGIT)+ ) => DECIMAL
| UINTEGER;
HEXLITERAL: 'x' QUOTE (HEXDIGIT HEXDIGIT)+ QUOTE;
protected
DIGIT: '0'..'9';
protected
HEXDIGIT: (DIGIT | 'a'..'f' );
protected
INTEGER:;
protected
UINTEGER: (DIGIT)+ ;
PLUS: '+' ;
MINUS: '-' ;
protected
SIGN: PLUS | MINUS;
protected
DECIMAL:;
// Whitespace -- ignored
WS: ( ' ' | '\t' | '\f'
| ( "\r\n" | '\r' | '\n' ) { newline(); } ) {
_ttype = Token.SKIP; } ;
ML_COMMENT: "/*" (options { generateAmbigWarnings=false;} :
{ LA(2)!='/' }? '*'
| ~('*') )*
"*/"
{$setType(Token.SKIP);} ;
Execution Output with Trace
Parsing: E:\SQLJava\Test1.txt
> start_rule; > lexer mIDENTIFIER; c==s
< lexer mIDENTIFIER; c==
LA(1)==select, > lexer mWS; c==
< lexer mWS; c==f
> lexer mIDENTIFIER; c==f
< lexer mIDENTIFIER; c==,
LA(2)==field1
> select_command; LA(1)==select, LA(2)==field1
> select_statement; LA(1)==select, LA(2)==field1
> select_expression; LA(1)==select, LA(2)==field1
> select_clause; LA(1)==select, LA(2)==field1
> select_list; LA(1)==field1, > lexer mCOMMA; c==,
< lexer mCOMMA; c==
LA(2)==,
> field_name_list; LA(1)==field1, LA(2)==,
> field_name; LA(1)==field1, LA(2)==,
< field_name; LA(1)==,, > lexer mWS; c==
< lexer mWS; c==f
> lexer mIDENTIFIER; c==f
< lexer mIDENTIFIER; c==,
LA(2)==field2
> field_name; LA(1)==field2, > lexer mCOMMA; c==,
< lexer mCOMMA; c==
LA(2)==,
< field_name; LA(1)==,, > lexer mWS; c==
< lexer mWS; c==f
> lexer mIDENTIFIER; c==f
< lexer mIDENTIFIER; c==
LA(2)==field3
> field_name; LA(1)==field3, > lexer mWS; c==
< lexer mWS; c==f
> lexer mIDENTIFIER; c==f
< lexer mIDENTIFIER; c==
LA(2)==from
< field_name; LA(1)==from, > lexer mWS; c==
< lexer mWS; c==t
> lexer mIDENTIFIER; c==t
< lexer mIDENTIFIER; c==;
LA(2)==table1
< field_name_list; LA(1)==from, LA(2)==table1
< select_list; LA(1)==from, LA(2)==table1
< select_clause; LA(1)==from, LA(2)==table1
> from_clause; LA(1)==from, LA(2)==table1
> table_name; LA(1)==table1, > lexer mSEMI; c==;
< lexer mSEMI; c==
LA(2)==;
< table_name; LA(1)==;, > lexer mWS; c==
< lexer mWS; c==
> lexer mWS; c==
< from_clause; LA(1)==;, > lexer mNUMBER; c==?
> lexer mSIGN; c==?
> lexer mSIGN; c==?
< lexer mSIGN; c==?
< lexer mNUMBER; c==?
< select_expression; LA(1)==;, > lexer mNUMBER; c==?
> lexer mSIGN; c==?
< lexer mSIGN; c==?
< lexer mNUMBER; c==?
< select_statement; LA(1)==;, > lexer mNUMBER; c==?
> lexer mSIGN; c==?
< lexer mNUMBER; c==?
< select_command; LA(1)==;, > lexer mNUMBER; c==?
> lexer mSIGN; c==?
< lexer mNUMBER; c==?
< start_rule; LA(1)==;, > lexer mNUMBER; c==?
> lexer mSIGN; c==?
< lexer mSIGN; c==?
< lexer mNUMBER; c==?
exception: E:\SQLJava\Test1.txt:6:1: unexpected char: 0x?F
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/antlr-interest/
<*> To unsubscribe from this group, send an email to:
antlr-interest-unsubscribe at yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list