[antlr-interest] Antlr3.4 Python bugs, workarounds

Benjamin S Wolf jokeserver at gmail.com
Mon Dec 26 17:20:39 PST 2011

On Fri, Oct 28, 2011 at 8:56 AM, Benjamin Niemann <pink at odahoda.de> wrote:
> On Fri, Oct 7, 2011 at 11:17 PM, Benjamin S Wolf <jokeserver at gmail.com> wrote:
>> I've discovered two bugs in working with the Python target to generate lexers.
>> The first is in that when supplying k, the generated code for special
>> state transitions is invalid: the "elif" keyword is split across lines
>> as "el\nif", so the python interpreter crashes upon reading "el".
> That's been reported before, but I have problem reproducing it. Are
> you using antlr-3.4-complete.jar (I can only see this problem with
> that build) or did you build it yourself from antlr-3.4.tar.gz (or
> something else completely)?

Hi Benjamin,

I was meddling around with the stg templates for Python in trying to
fix some other bugs I reported in another thread, and after updating
the files in antlr-3.4-complete.jar this problem was alleviated.

I narrowed down the diff and discovered that the stg templates in the
original jar all had DOS line endings (that is, \r\n instead of just
\n), and that removing all the carriage returns in
org/antlr/codegen/templates/Python/Python.stg solved the issue of the
elif being split across a newline.

That certainly explains why it only showed in antlr-3.4-complete.jar,
since the templates included with antlr-3.4.tar.gz did not have the
carriage returns. :)

More information about the antlr-interest mailing list