[antlr-interest] To Sam Harwell

chris king kingces95 at gmail.com
Tue Aug 9 19:55:40 PDT 2011


Sam, hey. Sorry to shower you with e-mails today. :) I got your new version
setup up and building but now one of my grammar file is causing a stack
overflow. I'm not sure if this is a problem with the CSharp port or a bug in
the Java ANTLR logic. Anyway, the attached grammer file of mine worked with
v3.3.1.7705 but with v3.4.1.9004 is causing a stack overflow in antlr3.exe.
To repro include the attached file in a .csproj as an Antlr3 include per
your instructions. Then uncomment either one of these rules and save to
execute Antlr3.


pp_conditional_section
  : //{ !SkipSection }? => input_section
  | //{ SkipSection }? => pp_skipped_section_part*
  ;


Here's how I'm include the file in the project:

    <Antlr3 Include="CSharp.g">
      <Generator>MSBuild:Compile</Generator>
    </Antlr3>

Here's the top of the function signature that I'm recursing through:

        // Factor so (a || b) == (result || a || b)
        public static SemanticContext FactorOr(ref SemanticContext a, ref
SemanticContext b)
        {
            HashSet<SemanticContext> opsA = new
HashSet<SemanticContext>(GetOrOperands(a));
            HashSet<SemanticContext> opsB = new
HashSet<SemanticContext>(GetOrOperands(b));

Here's a stack trace of the recursion I'm seeing. The line numbers should
match the source in v3.4.1.9004. I used that source to build the Antlr3.exe
that's having the stack overflow.

  Antlr3.EXE!Antlr3.Analysis.SemanticContext.And(a, b) Line 681 C#
  Antlr3.EXE!Antlr3.Analysis.SemanticContext.And(a, b) Line 681 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 809 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 730 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 823 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 740 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 740 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 823 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 730 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 813 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 730 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 823 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 823 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Closure(p, alt, context,
semanticContext, d, collectPredicates) Line 760 C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.ComputeStartState() Line 176
C#
  Antlr3.EXE!Antlr3.Analysis.NFAToDFAConverter.Convert() Line 90 C#
  Antlr3.EXE!Antlr3.Analysis.DFA.CreateFromNfa(decisionNumber,
decisionStartState) Line 243 C#
  Antlr3.EXE!Antlr3.Tool.Grammar.CreateLookaheadDFA(decision,
wackTempStructures) Line 1858 C#
  Antlr3.EXE!Antlr3.Tool.Grammar.CreateLookaheadDFAs(wackTempStructures)
Line 1586 C#
  Antlr3.EXE!Antlr3.Tool.Grammar.CreateLookaheadDFAs() Line 1526 C#
  Antlr3.EXE!Antlr3.Codegen.Target.PerformGrammarAnalysis(generator,
grammar) Line 121 C#
  Antlr3.EXE!Antlr3.Codegen.CodeGenerator.GenRecognizer() Line 670 C#
  Antlr3.EXE!Antlr3.AntlrTool.GenerateRecognizer(grammar) Line 791 C#
  Antlr3.EXE!Antlr3.AntlrTool.Process() Line 586 C#
  AntlrBuildTask.DLL!Antlr3.Build.Tasks.AntlrClassGenerationTaskInternal.Execute()
Line 135 C#

Thanks,
Chris


On Tue, Aug 9, 2011 at 6:39 PM, chris king <kingces95 at gmail.com> wrote:

> Also, in Antlr3.csproj I think LeftRecursiveRules.stg was pointing to the
> wrong place. I changed it to this and then the solution compiled:
>
>     <EmbeddedResource
> Include="..\..\..\antlr\antlr3-main\tool\src\main\resources\org\antlr\codegen\templates\LeftRecursiveRules.stg">
>
> Thanks,
> Chris
>
> On Tue, Aug 9, 2011 at 6:24 PM, chris king <kingces95 at gmail.com> wrote:
>
>> Sam, thanks! FYI The compilation of your source drop fails because the key
>> file is not included. You may want to disabled signing in all your projects
>> before you release the source drop or provide a dummy key file or find some
>> work around so folks can just open up the source and compile it. That would
>> help upgrading to the next version.
>>
>> Thanks,
>> Chris
>>
>> On Tue, Aug 9, 2011 at 5:02 PM, Sam Harwell <sharwell at pixelminegames.com>wrote:
>>
>>> Hi Chris,****
>>>
>>> ** **
>>>
>>> I got rid of the use of Stack<T> and the call to ElementAt() quite a
>>> while back. Instead I use a ListStack<T> which derives from List<T> and adds
>>> Push and Pop methods. I recommend updating to ANTLR 3.4.1 to correct this
>>> issue.****
>>>
>>> ** **
>>>
>>> Sam****
>>>
>>> ** **
>>>
>>> *From:* chris king [mailto:kingces95 at gmail.com]
>>> *Sent:* Tuesday, August 09, 2011 6:29 PM
>>> *To:* Sam Harwell; antlr-interest at antlr.org
>>>
>>> *Subject:* Re: [antlr-interest] To Sam Harwell****
>>>
>>> ** **
>>>
>>> Sam, hey, so I might have bumped into a code gen bug. I'm trying to
>>> access a variable "skipSection" on the parent production of
>>> "pp_conditional". It looks like the generated code is doing some computation
>>> involving the stack count when really it should just pass my index without
>>> any calculation. For example, below, shouldn't the highlighted code
>>> simply be my index? Or am I missing something? Looks like the
>>> highlighted code is assuming TOS is Count - 1 (like it would be with a stack
>>> implemented with a List<T>) but I don't think that's the case. The stack is
>>> a System.Collections.Generic.Stack<T> which maintains that TOS is always
>>> index 0. ****
>>>
>>> ** **
>>>
>>> $pp_conditional[1]::skipSection****
>>>
>>> ** **
>>>
>>> The code above is transformed to:****
>>>
>>> ** **
>>>
>>> System.Linq.Enumerable.ElementAt(pp_conditional_stack,
>>> pp_conditional_stack.Count-1-1).skipSection;****
>>>
>>> ** **
>>>
>>> Also, you may want to ensure that all user expressions are enclosed in
>>> parens. As I was trying to work around this issue I used "count -2" which
>>> got translated to stack.Count-count -2 -1. I had to add the parens to get it
>>> to be stack.Count-(count -2) -1.****
>>>
>>> ** **
>>>
>>> Thanks,****
>>>
>>> Chris****
>>>
>>> ** **
>>>
>>> On Thu, Aug 4, 2011 at 6:00 PM, chris king <kingces95 at gmail.com> wrote:*
>>> ***
>>>
>>> Ok. Thanks for looking into it. ****
>>>
>>> ** **
>>>
>>> On Thu, Aug 4, 2011 at 5:56 PM, Sam Harwell <sharwell at pixelminegames.com>
>>> wrote:****
>>>
>>> I did, and I can repro the issue but I haven’t resolved it yet.****
>>>
>>>  ****
>>>
>>> Sam****
>>>
>>>  ****
>>>
>>> *From:* chris king [mailto:kingces95 at gmail.com]
>>> *Sent:* Thursday, August 04, 2011 6:48 PM
>>> *To:* Sam Harwell
>>> *Cc:* antlr-interest at antlr.org
>>> *Subject:* Re: [antlr-interest] To Sam Harwell****
>>>
>>>  ****
>>>
>>> Awesome! I'll give it try. Did you see my follow up email that the parse
>>> string is 5 characters (there is a trailing space)? "/**/ " ****
>>>
>>>  ****
>>>
>>> And again, love the tool chain. I only really started to get traction on
>>> my project after I installed it. It's working great.****
>>>
>>>  ****
>>>
>>> Thanks,
>>> Chris****
>>>
>>>  ****
>>>
>>> On Thu, Aug 4, 2011 at 4:34 PM, Sam Harwell <sharwell at pixelminegames.com>
>>> wrote:****
>>>
>>> Hi Chris,
>>>
>>> In build 3.4.1.9004 that I released today, I switched all the projects to
>>> using $(ProjectDir) with relative paths.
>>>
>>> http://www.antlr.org/wiki/display/ANTLR3/Antlr3CSharpReleases****
>>>
>>>
>>> Sam
>>>
>>> -----Original Message-----
>>> From: antlr-interest-bounces at antlr.org
>>> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of chris king
>>> Sent: Monday, July 25, 2011 6:39 PM
>>> To: antlr-interest at antlr.org
>>> Subject: [antlr-interest] To Sam Harwell****
>>>
>>> Sam, hey, hope this finds you. Very small suggestion follows :). In
>>> Antlr3.StringTemplate.csproj could you use
>>>
>>>
>>>
>>> <AntlrBuildTaskPath>$(MSBuildProjectDirectory)\..\bin\Bootstrap</AntlrBuildT
>>> askPath>
>>>
>>> to reference the bootstrap directory? Originally it used the solutionDir
>>> and
>>> that prevented me from including a subset of the projects in my project
>>> (so
>>> I could simply reference them and have all the debugging, pdb, source,
>>> etc
>>> just work).
>>>
>>> Thanks,
>>> Chris****
>>>
>>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>>> Unsubscribe:
>>> http://www.antlr.org/mailman/options/antlr-interest/your-email-address**
>>> **
>>>
>>>  ****
>>>
>>> ** **
>>>
>>> ** **
>>>
>>
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CSharp.cs
Type: application/octet-stream
Size: 21831 bytes
Desc: not available
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20110809/14ad6512/attachment.obj 


More information about the antlr-interest mailing list