[antlr-interest] Stopping parser and lexer at first error

Jim Idle jimi at temporal-wave.com
Fri Apr 2 08:35:56 PDT 2010


You should program your lexer such that it does not throw any errors. Program for the common mistakes (such as un-terminated "string) and have a catch all rule for unknown characters.

Jim



> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Corrado Campisano
> Sent: Friday, April 02, 2010 7:59 AM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Stopping parser and lexer at first error
> 
> Hi all,
> 
> I setup an ANTLR-maven archetype with a grammar providing the override
> for
> the 'always resume' behaviour.
> 
> You can find details on my
> website<http://www.servicemix.eu/index.php?option=com_content&view=arti
> cle&id=14>,
> maybe it's worth checking it and adding a notice on this archetype to
> this
> ANTLR wiki
> page<http://www.antlr.org/wiki/display/ANTLR3/Building+ANTLR+Projects+w
> ith+Maven>and/or
> to the ANTLR
> v3 Maven plugin page <http://www.antlr.org/antlr3-maven-
> plugin/index.html>.
> 
> 
> Regards,
> Corrado.
> 
> 
> 2010/3/10 Corrado Campisano <corrado.campisano at gmail.com>
> 
> > Hi all,
> >
> > I needed to catch any syntax error (letting the lexer insert/delete
> chars
> > or the parser keeping parsing with the sys.err message only could be
> very
> > dangerous to my application), so I took a look on the reference
> (which
> > reports information not valid anymore) and on the internet and I
> found
> > several hints and articles:
> >
> > Why the generated parser code tolerates illegal
> expression?<http://www.antlr.org/wiki/pages/viewpage.action?pageId=4554
> 943>
> > How can I make the lexer exit upon first lexical
> error?<http://www.antlr.org/wiki/pages/viewpage.action?pageId=5341217>
> > http://www.antlr.org/wiki/display/ANTLR3/Custom+Syntax+Error+Recovery
> > [antlr-interest] I want to throw an exception and stop parse, please!
> > <http://www.antlr.org/pipermail/antlr-interest/2009-May/034605.html>
> >
> > It looks to me I found a way to do this, maybe it's worth to publish
> that
> > on the wiki, once validated.
> >
> >
> > I just added the following overrides to my grammar (attached):
> >
> > @parser::members
> > {
> >     public class ParserException extends RuntimeException {
> >             Object objCurrentInputSymbol = null;
> >
> >             public ParserException(Object oCurrentInputSymbol) {
> >                 this.objCurrentInputSymbol = oCurrentInputSymbol;
> >             }
> >         }
> >
> >         protected Object recoverFromMismatchedToken(IntStream input,
> int
> > ttype, BitSet follow) throws RecognitionException {
> >             System.out.println("PARSER :
> > this.getCurrentInputSymbol(input).toString() : " +
> > this.getCurrentInputSymbol(input).toString());
> >             System.out.println("PARSER : this.failed() : " +
> > this.failed());
> >             System.out.println("PARSER :
> this.getNumberOfSyntaxErrors() : "
> > + this.getNumberOfSyntaxErrors());
> >             throw new
> ParserException(this.getCurrentInputSymbol(input));
> >         }
> > }
> >
> > @lexer::members
> > {
> >     public class LexerException extends RuntimeException {
> >             RecognitionException recognitionException = null;
> >             String strErrorHeader = null;
> >             String strErrorMessage = null;
> >
> >             public LexerException(RecognitionException recExc, String
> > sHead, String sMsg) {
> >                 this.recognitionException = recExc;
> >                 this.strErrorHeader = sHead;
> >                 this.strErrorMessage = sMsg;
> >
> >                 System.out.println("LEXER : ErrorHeader : " + sHead);
> >                 System.out.println("LEXER : ErrorMessage : " + sMsg);
> >                 System.out.println("LEXER : RecognitionException : "
> +
> > this.recognitionException.toString());
> >             }
> >         }
> >
> >
> >         public void reportError(RecognitionException recExc) {
> >         throw new LexerException(recExc, this.getErrorHeader(recExc),
> > getErrorMessage(recExc, this.getTokenNames()));
> >     }
> > }
> >
> >
> > Then I tested it with a simple class:
> >     public static void main(String[] args) {
> >         testLexerError();
> >         testParserError();
> >     }
> >     private static void testLexerError() {
> >         String strDlToParse = "{CORRADO PIPPO ;feee}";
> >         System.out.println("TESTING LEXER with : " + strDlToParse);
> >         testError(strDlToParse);
> >     }
> >     private static void testParserError() {
> >         String strDlToParse = "{CORRADO PIPPO feee} dhert";
> >         System.out.println("TESTING PARSER with : " + strDlToParse);
> >         testError(strDlToParse);
> >     }
> >     private static void testError(String strDlToParse) {
> >         CommonTree tree=null;
> >         String strError = null;
> >
> >         ANTLRStringStream input = new
> > org.antlr.runtime.ANTLRStringStream(strDlToParse);
> >         Dl2OwlJavaBLexer lexer = new Dl2OwlJavaBLexer(input);
> >         TokenStream tokens = new
> > org.antlr.runtime.CommonTokenStream(lexer);
> >         Dl2OwlJavaBParser parser = new Dl2OwlJavaBParser(tokens);
> >
> >         try {
> >             // this may rise an exception
> >             // TODO : check why NO EXCEPTION is risen with error
> "line 1:9
> > no viable alternative at character ';'" on inputs like "{CORRADO ;}"
> >             eu.servicemix.dl2owl.Dl2OwlJavaBParser.axiom_return ret =
> > parser.axiom();
> >
> >             // TODO : check if this will be executed if no exception
> rises
> >             tree = (CommonTree) ret.getTree();
> >
> >             printTreeHelper(tree);
> >
> >         } catch (RecognitionException e) {
> >
> >             System.out.println(e.toString());
> >             e.printStackTrace();
> >
> >         } catch (RuntimeException e) {
> >
> >             System.out.println(e.toString());
> >             e.printStackTrace();
> >         }
> >     }
> >
> >
> > The output looks ok, I wonder whether the whole 'trick' is too...
> >
> > TESTING LEXER with : {CORRADO PIPPO *;*feee}
> > LEXER : ErrorHeader : line 1:15
> > LEXER : ErrorMessage : no viable alternative at character ';'
> > LEXER : RecognitionException : NoViableAltException(';'@[1:1: Tokens
> : (
> > T__37 | T__38 | T__39 | T__40 | HAS_VALUE | ALL_VALUES | SOME_VALUES
> | DOT |
> > HAS_CARD | MIN_CARD | MAX_CARD | NOT | AND | OR | URI_REF | INT_VALUE
> | WS |
> > CTRL_CHAR );])
> > eu.servicemix.dl2owl.Dl2OwlJavaBLexer$LexerException
> > eu.servicemix.dl2owl.Dl2OwlJavaBLexer$LexerException
> >     at
> >
> eu.servicemix.dl2owl.Dl2OwlJavaBLexer.reportError(Dl2OwlJavaBLexer.java
> :69)
> >     at org.antlr.runtime.Lexer.nextToken(Lexer.java:94)
> >     at
> >
> org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:1
> 19)
> >     at
> org.antlr.runtime.CommonTokenStream.LT<http://org.antlr.runtime.commont
> okenstream.lt/>
> > (CommonTokenStream.java:238)
> >     at
> >
> eu.servicemix.dl2owl.Dl2OwlJavaBParser.axiom(Dl2OwlJavaBParser.java:110
> )
> >     at
> >
> eu.servicemix.dl2owl.CommonTreeHelper.testError(CommonTreeHelper.java:1
> 40)
> >     at
> >
> eu.servicemix.dl2owl.CommonTreeHelper.testLexerError(CommonTreeHelper.j
> ava:121)
> >     at
> > eu.servicemix.dl2owl.CommonTreeHelper.main(CommonTreeHelper.java:113)
> >
> > TESTING PARSER with : {CORRADO PIPPO feee} *dhert*
> > PARSER : this.getCurrentInputSymbol(input).toString() :
> > [@8,21:25='dhert',<7>,1:21]
> > PARSER : this.failed() : false
> > PARSER : this.getNumberOfSyntaxErrors() : 0
> > eu.servicemix.dl2owl.Dl2OwlJavaBParser$ParserException
> > eu.servicemix.dl2owl.Dl2OwlJavaBParser$ParserException
> >     at
> >
> eu.servicemix.dl2owl.Dl2OwlJavaBParser.recoverFromMismatchedToken(Dl2Ow
> lJavaBParser.java:97)
> >     at
> org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
> >     at
> >
> eu.servicemix.dl2owl.Dl2OwlJavaBParser.axiom(Dl2OwlJavaBParser.java:232
> )
> >     at
> >
> eu.servicemix.dl2owl.CommonTreeHelper.testError(CommonTreeHelper.java:1
> 40)
> >     at
> >
> eu.servicemix.dl2owl.CommonTreeHelper.testParserError(CommonTreeHelper.
> java:126)
> >     at
> > eu.servicemix.dl2owl.CommonTreeHelper.main(CommonTreeHelper.java:114)
> >
> >
> > Any comment really appreciated!!
> >
> > Corrado
> >
> >
> 
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address





More information about the antlr-interest mailing list