[antlr-interest] Failure of parser on invalid input

[Jim Idle]

Adding the EOF should force the parser to consume all the tokens, unless
that has changed in v4. However, does your lexer actually consume all the
characters? Your last lexer rule should be a catch all:

ERROR : . ;

You can ask the v3 parsers how many errors they logged, not sure about v4.
Generally though you will want to use your own error handling routine,
which would accumulate errors in a collection of some sort, which you can
then examine.

If you are getting different behavior in Java to Clojure then I would use
Java for the moment and leave Clojure for later.

Jim

-----Original Message-----
From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Tomas Svarovsky
Sent: Friday, October 12, 2012 4:25 PM
To: antlr-interest at antlr.org
Subject: Re: [antlr-interest] Failure of parser on invalid input

Hey Jim

Thanks for the link to search engine this is very helpful.

I tried to add the EOF to grammar as you suggested and this helps to force
the parser to stop immediately but still does not fail. To try to further
minimize the possible effect of clojure. I tried to use the TestRig/grun
which is mentioned in the book.

This prints some messages that are useful but still does not seem to fail
on invalid input. The return code is always 0. Reading the source code for
the class helped to discover couple of useful methods for printing token
stream etc but still nothing that would indicate that the exceptions are
consumed silently. Is there a way how to programatically tell it failed
and what is a problem?

As for the language it is actually not SQL. It is true that in full extent
it is quite complicated as well (even though it is still simpler than SQL)
but I need only very small portion so I think it fits the bill.

Thanks again for your help

Tomas

On Thu, Oct 11, 2012 at 11:59 PM, Jim Idle <jimi at temporal-wave.com> wrote:
> Please make use of http://antlr.markmail.org as you will find it is a
> good search engine for the email list.
>
> Unless you tell ANTLR to keep parsing while there is any input left,
> it will just stop when there is input that will not match any parser
> rule. So you want;
>
> query: 'SELECT' EOF ;
>
> Also, I would not try to write a SQL grammar as your first project as
> it is far too complicated to learn while trying to do that. Start with
> expression trees and simple languages until you have the basics down.
>
> Jim
>
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Tomas Svarovsky
> Sent: Friday, October 12, 2012 2:20 PM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] Failure of parser on invalid input
>
> Hi list
>
> I am trying to create simple application and I decided I will try to
> use ANTLR so I bought the book and I am working through it (thanks for
> it) while trying to create something.
>
> I am little confused with the behavior of the generated parser while I
> feed it with input that does not conform to the grammar. I would
> expect it to fail loud with an exception or something like that but
> instead it seems to recover and sort of finish parsing.
>
> For example. This is my grammar
>
> grammar Gooddata;
>
> @header {
> package org.fluke.tools;
> }
>
> query : 'SELECT';
>
> The app looks roughly like this It is not Java but Clojure hopefully
> it is not influencing things.
>
> (let [input (new ANTLRFileStream "test.txt")
>         lexer (new GooddataLexer input)
>         tokens (new CommonTokenStream lexer)
>         parser (new GooddataParser tokens)]
>     (. (.  parser query) toStringTree parser)))
>
> If I feed it SELECT it prints "(query SELECT)" and that is expected.
> If I feed it SLCT It prints "query" but it does not fail.
>
> Is there something I need to do to be able to tell that it did not
> parse correctly and be able to present present some reasonable error
> message? Is there a place in the book where this is discuss ed? I am
> using antlr-4.0b1
>
> Thanks in advance
>
> Tomas
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address

List: http://www.antlr.org/mailman/listinfo/antlr-interest
Unsubscribe:
http://www.antlr.org/mailman/options/antlr-interest/your-email-address