[antlr-interest] ANTLR Parser file different on different machines - method to exceed 65535 characters

[Jim Idle]

Switch to 3.4. The timeout is temporal based, not CPU based and so the
'speed' of your CPU will influence how far the alogorithm gets. In 3.4 the
timeout is not needed so you should get the same result. However, a large
conversion timeout, say 25000 should achieve the same.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Luke Tucker
> Sent: Tuesday, August 23, 2011 3:28 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] ANTLR Parser file different on different
> machines - method to exceed 65535 characters
>
> Hi, first of all, a little bit of background that you might find
> interesting in terms of what ANTLR is being used for...
>
> I have been using ANTLR to convert a language called STL into
> JavaScript. This STL language is used to define control procedures for
> commanding European Space Agency ground station monitoring and control
> equipment. This is a custom language only used by ESA and is quite
> quirky in terms of grammar. The existing system is around 15 years old
> and is being replaced with a new system. These STL procedures are being
> converted into equivalent in JavaScript which will run in a Rhino
> engine. My grammar for doing this conversion is more or less complete
> and the new JavaScripts are successfully commanding ground station
> equipment running in a simulator. I should also mention that the target
> language for the grammar is Java and that I am using antlr-3.2.
>
>
> Now for the problem and the reason for my message...
>
> As part of the build process for the application doing the conversion,
> ANTLR is run to generate the lexer and parser files from ant before
> building the Java application. Previously I had been generating the
> Parser and Lexer in our environment and committing the resulting java
> files to the repository rather than performing the generation as part
> of the build. This works perfectly well here on our development
> machines on a number of different machines and even a virtual machine
> with a small amount of memory.
>
>
> Typically however, when we deliver the software to ESA and they try to
> run the build process on their machines, the Parser file produced is
> different and the build fails. The reason for the build failure is that
> "static final String DFA53_specialS" and "static final String
> DFA53_transitionS" arrays are being produced with a huge number
> elements, together with a massive switch statement in a
> "specialStateTransition()" method that is causing the method to exceed
> 65535 characters.
> I have read that this can occur with complex and/or unoptimised grammar
> and I will be the first to admit that the grammar I have written might
> not be 100% optimised. Since the parser generation works on our
> machines and not on ESAs, my limited ANTLR-foo is not the root cause of
> this problem. I have also confirmed that the version of ANTLR being
> used is exactly the same.
>
> I have done a lot of searching through the mailing list archives and
> found a suggestion that using -Xconversiontimeout 100000 as an input to
> ANTLR might help solve this issue, but that doesn't seem to be helping.
> Just in case I did this wrong, this is how I used this option to
> generate the file:
>
> java -Xms1024M -Xmx1024M -jar antlr-3.2.jar -Xconversiontimeout 100000
>
> Is there anything else that might cause a large switch block in
> specialStateTransition() on one machine and not another with the same
> grammar/same ANTLR version? Should I get them to try with even higher
> values for Xconversiontimeout or am I barking up the wrong tree with
> this?
>
> If necessary I can post the grammar I am using (minus the inline code)
> and an example of the STL language being parsed, but I don't think
> that's necessary at this stage. In any case, it will probably hurt your
> eyes.
>
> Thanks very much in advance for any help.
>
> Luke
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-
> email-address