[antlr-interest] exception unexpected char in simple parser

earlsinclair8888 earlsinclair8888 at yahoo.com
Tue Mar 23 11:25:40 PST 2004


Hello,

I am encountering a problem while constructing a simple test parser, 
for an gSQL likeh language. I would appreciate any help
that I could 
get solving this. I created the parser/lexer using antlr.tool and 
used a test file for input which looks like this:

select field1, field2, field3
from table1;     

The trace output appears to being what I would expect the parser to 
be doing, except that once it reaches eof it produces the following 
message:

exception: E:\SQLJava\Test1.txt:6:1: unexpected char: 0x?F


The last line of input is line 5. I assume this is an EOF problem on 
my part (?). Being an amateur at this, perhaps someone can point me 
in the right direction.

Thanks for any help.

John Parke



Pertinent Info:



class SqlParser extends Parser;

options
{
    exportVocab                = Sql;
    k                          = 2;
    buildAST                   = false;
    defaultErrorHandler        = false;
}



start_rule:        select_command (SEMI)?  EOF! ;

select_command:    select_statement;

select_statement:
          ( OPEN_PAREN select_command CLOSE_PAREN ) => OPEN_PAREN 
select_command CLOSE_PAREN
          | select_expression ;


select_expression:
                select_clause from_clause (where_clause)?;
         
         
select_clause:  SELECT ( ALL )? select_list;
         
from_clause:    FROM table_name ;

where_clause:   WHERE where_condition;


select_list:    field_name_list
                | ASTERISK;

field_name_list:   field_name ( COMMA field_name )* ;


table_name:        IDENTIFIER;

where_condition:   condition ;


field_name:        IDENTIFIER ;

c rest omittedc



class SqlLexer extends Lexer;

options {
    exportVocab   = Sql;
    testLiterals  = false;
    k             = 2;
    caseSensitive = false;
    caseSensitiveLiterals = false;
    charVocabulary = '\u0000'..'\uFFFE';
}

tokens
{
 SELECT   =   "select";
 ALL      =   "all";
 FROM     =   "from";
 WHERE    =   "where";
 OR       =   "or";
 AND      =   "and";
 NOT      =   "not";
}


IDENTIFIER options {testLiterals=true;} :

           'a' .. 'z' ( 'a' .. 'z' | '0' .. '9' | '_' | '$' )* ;


// quoted_string 
//    ::= "'" { "any_character" } "'" 

QUOTED_STRING:  '\'' ( ~'\'' )* '\''  ;


SEMI:        ';' ;
DOT:         '.' ;
COMMA:       ',' ;
ASTERISK:    '*' ;
AT_SIGN:     '@' ;

OPEN_PAREN:  '(';
CLOSE_PAREN: ')';


EQ:          '=' ;
LE:          "<="; 
NOT_EQ:      "!=" | "^=";
GT:          '>';
GE:          ">=";

QUOTE: '\'';

NUMBER:        (SIGN UINTEGER)                 => INTEGER
              |(UINTEGER  '.' UINTEGER)        => DECIMAL
              |(SIGN UINTEGER '.' UINTEGER)    => DECIMAL
              |('.' (DIGIT)+ )                 => DECIMAL
              | UINTEGER;                      


HEXLITERAL:   'x' QUOTE (HEXDIGIT HEXDIGIT)+ QUOTE; 
    
protected
DIGIT:        '0'..'9';

protected
HEXDIGIT:     (DIGIT | 'a'..'f' ); 

protected                 
INTEGER:;  

protected
UINTEGER:     (DIGIT)+ ;     

PLUS:         '+' ;
MINUS:        '-' ;

protected
SIGN:         PLUS | MINUS;

protected
DECIMAL:;

// Whitespace -- ignored
WS:         ( ' ' | '\t' | '\f'       
        |   (   "\r\n"  |   '\r' |   '\n' ) { newline(); }  ) { 
_ttype = Token.SKIP; } ;
   

ML_COMMENT: "/*" (options { generateAmbigWarnings=false;} :
              { LA(2)!='/' }? '*'
              | ~('*') )*
              "*/"
             {$setType(Token.SKIP);} ;


Execution Output with Trace

Parsing: E:\SQLJava\Test1.txt
 > start_rule;  > lexer mIDENTIFIER; c==s
 < lexer mIDENTIFIER; c== 
LA(1)==select,  > lexer mWS; c== 
 < lexer mWS; c==f
 > lexer mIDENTIFIER; c==f
 < lexer mIDENTIFIER; c==,
LA(2)==field1
  > select_command; LA(1)==select, LA(2)==field1
   > select_statement; LA(1)==select, LA(2)==field1
    > select_expression; LA(1)==select, LA(2)==field1
     > select_clause; LA(1)==select, LA(2)==field1
      > select_list; LA(1)==field1,  > lexer mCOMMA; c==,
 < lexer mCOMMA; c== 
LA(2)==,
       > field_name_list; LA(1)==field1, LA(2)==,
        > field_name; LA(1)==field1, LA(2)==,
        < field_name; LA(1)==,,  > lexer mWS; c== 
 < lexer mWS; c==f
 > lexer mIDENTIFIER; c==f
 < lexer mIDENTIFIER; c==,
LA(2)==field2
        > field_name; LA(1)==field2,  > lexer mCOMMA; c==,
 < lexer mCOMMA; c== 
LA(2)==,
        < field_name; LA(1)==,,  > lexer mWS; c== 
 < lexer mWS; c==f
 > lexer mIDENTIFIER; c==f
 < lexer mIDENTIFIER; c==

LA(2)==field3
        > field_name; LA(1)==field3,  > lexer mWS; c==

 < lexer mWS; c==f
 > lexer mIDENTIFIER; c==f
 < lexer mIDENTIFIER; c== 
LA(2)==from
        < field_name; LA(1)==from,  > lexer mWS; c== 
 < lexer mWS; c==t
 > lexer mIDENTIFIER; c==t
 < lexer mIDENTIFIER; c==;
LA(2)==table1
       < field_name_list; LA(1)==from, LA(2)==table1
      < select_list; LA(1)==from, LA(2)==table1
     < select_clause; LA(1)==from, LA(2)==table1
     > from_clause; LA(1)==from, LA(2)==table1
      > table_name; LA(1)==table1,  > lexer mSEMI; c==;
 < lexer mSEMI; c== 
LA(2)==;
      < table_name; LA(1)==;,  > lexer mWS; c== 
 < lexer mWS; c== 
 > lexer mWS; c== 
 
 < from_clause; LA(1)==;,  > lexer mNUMBER; c==?
  > lexer mSIGN; c==?

  > lexer mSIGN; c==?
  < lexer mSIGN; c==?
 < lexer mNUMBER; c==?
      < select_expression; LA(1)==;,  > lexer mNUMBER; c==?
  > lexer mSIGN; c==?
  < lexer mSIGN; c==?

 < lexer mNUMBER; c==?
      < select_statement; LA(1)==;,  > lexer mNUMBER; c==?
  > lexer mSIGN; c==?

 < lexer mNUMBER; c==?
      < select_command; LA(1)==;,  > lexer mNUMBER; c==?
  > lexer mSIGN; c==?

 < lexer mNUMBER; c==?
      < start_rule; LA(1)==;,  > lexer mNUMBER; c==?
  > lexer mSIGN; c==?
  < lexer mSIGN; c==?

 < lexer mNUMBER; c==?
exception: E:\SQLJava\Test1.txt:6:1: unexpected char: 0x?F





 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/antlr-interest/

<*> To unsubscribe from this group, send an email to:
     antlr-interest-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/
 



More information about the antlr-interest mailing list