[antlr-interest] Bug with nested syntactic predicates?ReducedC#2.0 test grammar.

[Robin Davies]

I looked at the sources. There don't appear to be disambiguating predicates
of any kind in the Java sources.

The issue may be more severe in C#, where there's a bona-fide language
ambiguity, that actually does require apredicate to disambiguate.

The ambiguity is:

    f(a<b,c>(7))  // function with one argument or two?  Generic member
call, or two boolean arguments?

and the full predicate test, required to disambiguate a<b,c>(7) -- per the
language standard -- depends on the character that follows (7).

I wasn't able to get the Java grammar to build in antlrworks (it locks up),
so I can't run specific tests.

fwiw, I was running the grammar-in-progress using "backtrack=true;" (but not
"k=2"), and I was getting inconsistent parses in this case. For example,

    f(a<b,c>d)

(legal c# 2.0) with the full grammar, with backtrack=true, no predicates,
would not parse. Should it? If so, maybe I'll see if I can isolate
production rules for that case.

Part of my personal challenge is that I'm running in a C# production
environment, and debugging through java code is .... um... challenging. Next
step for me was to debug through the generated code to see if I could
understand what's happening. Is there a way to obtain sources for the C#
runtime? This would greatly help with debugging. I do have the source, but I
haven't actually built it. Are the .cs files for the runtime auto-generated?



----- Original Message ----- 
From: "Micheal J" <open.zone at virgin.net>
To: "'ANTR Interest'" <antlr-interest at antlr.org>
Sent: Wednesday, June 13, 2007 9:45 AM
Subject: Re: [antlr-interest] Bug with nested syntactic
predicates?ReducedC#2.0 test grammar.


> Have you looked at how the Java equivalent handles this same issue?
> http://www.antlr.org/grammar/1152141644268/Java.g
>
> Micheal
>
> -----------------------
> The best way to contact me is via the list/forum. My time is very limited.
>
>> -----Original Message-----
>> From: antlr-interest-bounces at antlr.org
>> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Robin Davies
>> Sent: 13 June 2007 14:03
>> To: ANTR Interest
>> Subject: [antlr-interest] Bug with nested syntactic
>> predicates? ReducedC#2.0 test grammar.
>>
>>
>> The following grammar is a reduced test case to simulate
>> resolution of the
>> generic/expression ambiguity in C# 2.0 via syntactic
>> predicates. I can't get
>> it to work. I'm pretty sure it's a bug; but I'm still at the
>> "a little
>> knowledge is a dangerous thing" stage. It is  possible that I'm still
>> missing an essential concept wrt/ syntactic predicates. I've
>> been over
>> chapters 12-14 backwards and forwards, and I firmly believe
>> that the grammar
>> is correct.
>>
>> The conflict in C# is of the form
>>         methodCall(a<b,c>.StaticMethod())   // is a<b a
>> boolean expression
>> or the start of a generic type name)?
>>         methodCall(a<b,c>d)                         // is a<b
>> a boolean
>> expression or the start of a generic type name)?
>>
>> The intended resolution is: if it's a generic name (e.g.
>> a<b,c>) then it's a
>> generic name. The second case should be treated as an error.
>> (The actual
>> rule in the standard is a bit more unpleasant, requiring an
>> ad-hoc pre-scan
>> of incoming tokens as a semantic predicate to locate a
>> probably-matching '>'
>> token).
>>
>> Test input is:
>>     List<int,int>
>>
>> In ANTLRWorks, this succesfully and correctly parses as an
>> expr3, and a
>> generic_type_expr, but fails to parse as an expr. The
>> question is: why isn't
>> the first option of the expr production taken for the given
>> test input,
>> since it is a valid generic_type_expr?
>>
>> expr
>>     :    (generic_type_expr)=>   // e.g. generic   List<int>
>>          expr3    // bypass LT/GT expressions
>>     |     (expr3 LT  expr3)=> (expr3 LT expr3)  //
>> (non-assoc, for clarity)
>>     |     (expr3 GT expr3)=> (expr3 GT expr3)
>>     |     (expr3)=> expr3
>>     ;
>>
>> I've seen various behaviors depending on the state of my test
>> grammar. In
>> the supplied grammar -- if AntlrWorks is displaying syntactic
>> predicate
>> evaluation properly -- the (generic_type_expr) syntactic
>> predicate doesn't
>> seem to get run at all. But I've also seen cases where
>> (generic_type_expr)=>
>> succeeds, and expr3 fails for this input. And cases where an apparent
>> pre-scan for generic_type_expr accepts "List<int,int", and
>> then fails,
>> despite the fact that the next token is the expected ">" token.
>>
>> If I remove either of the LT or GT rules, the parse succeeds
>> (probably
>> through option 4). But shouldn't the semantic predicate in
>> the first option
>> prevent options 2 and 3 from running in the intact grammar?
>>
>> The full grammar is attached. I've done my best to simplify
>> it as much as
>> possible.
>>
>> As an aside,  "!=>" would be a very useful construct in
>> ANTLR, allowing the
>> following rule to cut off the '<' binary operator, in
>> left-associative
>> comparison expressions.
>>
>> comparison_expr:   expr_next
>>                                     (
>>                                         ( (generic_arg_list)!=> '<'
>> expr_next)
>>                                     |   ('>' expr_next)
>>                                     )*
>>                             ;
>>
>> The nightmarish input case for this:   a < b <
>> G<int,int>.StaticBoolMember()
>> (a valid, but not very useful C# expression, I think; == != are
>> non-associative (good thing), but <, > are right-assoc, for
>> some bizarre
>> reason).
>>
>> Any help appreciated.
>>
>>
>>
>>
>