[antlr-interest] Antlr3.4 Python bugs, workarounds

[Terence Parr]

On Dec 27, 2011, at 5:08 PM, Benjamin Niemann wrote:
>> Hi Benjamin,
>> 
>> I was meddling around with the stg templates for Python in trying to
>> fix some other bugs I reported in another thread, and after updating
>> the files in antlr-3.4-complete.jar this problem was alleviated.
>> 
>> I narrowed down the diff and discovered that the stg templates in the
>> original jar all had DOS line endings (that is, \r\n instead of just
>> \n), and that removing all the carriage returns in
>> org/antlr/codegen/templates/Python/Python.stg solved the issue of the
>> elif being split across a newline.
>> 
>> That certainly explains why it only showed in antlr-3.4-complete.jar,
>> since the templates included with antlr-3.4.tar.gz did not have the
>> carriage returns. :)
> 
> Good catch, thanks a lot for figuring that out.
> That seems like a bug in stringtemplate to me - I thought it was
> smarter about dealing with line endings.
> 
> Ter:
> Was the jar built on a windows box?

nope. mac os x.

> I assume perforce adds the CRLFs
> when checking out the files under windows - the files are stored as
> "text", i.e. line endings are converted to the native system.

that's correct i think.

> Unless ST can be taught to deal with that properly, we could store the
> templates as binary in the repository - but that could be messy when
> someone actually wants to edit them under windows and it's hard to
> notice when CRs creep back in.
> Or avoid building jars on windows ;)
> This probably affects other targets as well, but those are probably
> less picky about some extra whitespace here and there. Could lead to
> some obscure bugs though.

ST emits proper newlines per platform and should read \r\n just like \n.  BUT…apparently not ;)  I'm adding to list to check ST v4 to see how it works.

Ter