[antlr-interest] Failure of parser on invalid input

[Tomas Svarovsky]

Jim,

Thanks again for your help.

Regarding EOF I need to dig around since I thing I do not understand
it correctly.

Regarding the errors I found a chapter in the book that has some
information on this topic so along with your comments I have a place
to start. Clojure seems to give same results I was even able to get
the "error messages" out of it. The problem was the configuration of
REPL in my Emacs installation.

You are right that I will probably have to roll out my own error handler.

Tomas

On Fri, Oct 12, 2012 at 1:31 AM, Jim Idle <jimi at temporal-wave.com> wrote:
> Adding the EOF should force the parser to consume all the tokens, unless
> that has changed in v4. However, does your lexer actually consume all the
> characters? Your last lexer rule should be a catch all:
>
> ERROR : . ;
>
> You can ask the v3 parsers how many errors they logged, not sure about v4.
> Generally though you will want to use your own error handling routine,
> which would accumulate errors in a collection of some sort, which you can
> then examine.
>
> If you are getting different behavior in Java to Clojure then I would use
> Java for the moment and leave Clojure for later.
>
> Jim
>
> -----Original Message-----
> From: antlr-interest-bounces at antlr.org
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Tomas Svarovsky
> Sent: Friday, October 12, 2012 4:25 PM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Failure of parser on invalid input
>
> Hey Jim
>
> Thanks for the link to search engine this is very helpful.
>
> I tried to add the EOF to grammar as you suggested and this helps to force
> the parser to stop immediately but still does not fail. To try to further
> minimize the possible effect of clojure. I tried to use the TestRig/grun
> which is mentioned in the book.
>
> This prints some messages that are useful but still does not seem to fail
> on invalid input. The return code is always 0. Reading the source code for
> the class helped to discover couple of useful methods for printing token
> stream etc but still nothing that would indicate that the exceptions are
> consumed silently. Is there a way how to programatically tell it failed
> and what is a problem?
>
> As for the language it is actually not SQL. It is true that in full extent
> it is quite complicated as well (even though it is still simpler than SQL)
> but I need only very small portion so I think it fits the bill.
>
> Thanks again for your help
>
> Tomas
>
> On Thu, Oct 11, 2012 at 11:59 PM, Jim Idle <jimi at temporal-wave.com> wrote:
>> Please make use of http://antlr.markmail.org as you will find it is a
>> good search engine for the email list.
>>
>> Unless you tell ANTLR to keep parsing while there is any input left,
>> it will just stop when there is input that will not match any parser
>> rule. So you want;
>>
>> query: 'SELECT' EOF ;
>>
>> Also, I would not try to write a SQL grammar as your first project as
>> it is far too complicated to learn while trying to do that. Start with
>> expression trees and simple languages until you have the basics down.
>>
>> Jim
>>
>> -----Original Message-----
>> From: antlr-interest-bounces at antlr.org
>> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Tomas Svarovsky
>> Sent: Friday, October 12, 2012 2:20 PM
>> To: antlr-interest at antlr.org
>> Subject: [antlr-interest] Failure of parser on invalid input
>>
>> Hi list
>>
>> I am trying to create simple application and I decided I will try to
>> use ANTLR so I bought the book and I am working through it (thanks for
>> it) while trying to create something.
>>
>> I am little confused with the behavior of the generated parser while I
>> feed it with input that does not conform to the grammar. I would
>> expect it to fail loud with an exception or something like that but
>> instead it seems to recover and sort of finish parsing.
>>
>> For example. This is my grammar
>>
>> grammar Gooddata;
>>
>> @header {
>> package org.fluke.tools;
>> }
>>
>> query : 'SELECT';
>>
>> The app looks roughly like this It is not Java but Clojure hopefully
>> it is not influencing things.
>>
>> (let [input (new ANTLRFileStream "test.txt")
>>         lexer (new GooddataLexer input)
>>         tokens (new CommonTokenStream lexer)
>>         parser (new GooddataParser tokens)]
>>     (. (.  parser query) toStringTree parser)))
>>
>> If I feed it SELECT it prints "(query SELECT)" and that is expected.
>> If I feed it SLCT It prints "query" but it does not fail.
>>
>> Is there something I need to do to be able to tell that it did not
>> parse correctly and be able to present present some reasonable error
>> message? Is there a place in the book where this is discuss ed? I am
>> using antlr-4.0b1
>>
>> Thanks in advance
>>
>> Tomas
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe:
>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe:
> http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address