[antlr-interest] Why No Error?

genericised trigonometric at softhome.net
Thu Aug 15 03:16:54 PDT 2002


I created the following parser, as an example of how to
parse comma separated variable (CSV) files:

class CSVParser extends Parser;
file : (line)+ ;
line : (rec)+ NEWLINE ;
rec  : (r:RECORD) (COMMA)?
       {System.out.println(r.getText());}
     ;

The corresponding Lexer is:

class CSVLexer extends Lexer;
options { charVocabulary='\3'..'\377'; }
RECORD  : (~(','|'\r'|'\n'|' '|'\t'))+ ;
COMMA   : ',' ;
NEWLINE : ('\r''\n')=> '\r''\n' //DOS
        | '\r'                  //MAC
        | '\n'                  //UNIX
        { newline(); }
        ;
WS      : (' '|'\t') { $setType(Token.SKIP); } ;

Pretty straightforward, but, when I run this on a
CSV it produces no error.

The last line of a CSV is:

blah, blah, blah

so the line does not consist of

rec+ NEWLINE

but

rec+

When 

match(NEWLINE)

is called from the parser, why does it not throw
a mismatchedTokenException?

Or does it throw some kind of exception that is
caught and causes the parsing of the inputstream
to terminate gracefully?

The parser is invoked from some main file like this:

csvParser.file();

I have spent a couple of hours investigating this,
looking through the ANTLR source and stuff but I
have not yet found where this is dealt with?

I might do a bit of weekend investigation into this
because of what I will learn in the process of
determining this but at the moment I am supposed to
be writing this ANTLR tutorial and then got side
tracked trying to explain why it is OK that the
parser does not match the final NEWLINE.

Well actually, is it ok, or should the rule for file
be defined something like:

file : (line)+ EOFCHAR;

Regards

A Person


 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/ 



More information about the antlr-interest mailing list