[antlr-interest] normalize \n to \n in ST

Terence Parr parrt at cs.usfca.edu
Wed Jun 4 13:32:41 PDT 2008


Here are 3 of my unit tests. Basically, no matter which you used in  
templates or attributes or options, ST normalizes everything to be the  
newline field of the output writer AutoIndentWriter and its subclass  
NoIndentWriter. Pushing. I guess I need to push 3.2 ST soon :)

Ter

	public void testNewlineNormalizationInTemplateString() throws  
Exception {
		StringTemplate st = new StringTemplate(
				"Foo\r\n"+
				"Bar\n",
				AngleBracketTemplateLexer.class
				);
		StringWriter sw = new StringWriter();
		st.write(new AutoIndentWriter(sw,"\n")); // force \n as newline
		String result = st.toString();
		String expecting ="Foo\nBar\n";     // expect \n in output
		assertEquals(expecting, result);
	}

	public void testNewlineNormalizationInTemplateStringPC() throws  
Exception {
		StringTemplate st = new StringTemplate(
				"Foo\r\n"+
				"Bar\n",
				AngleBracketTemplateLexer.class
				);
		StringWriter sw = new StringWriter();
		st.write(new AutoIndentWriter(sw,"\r\n")); // force \r\n as newline
		String result = st.toString();
		String expecting ="Foo\r\nBar\r\n";     // expect \r\n in output
		assertEquals(expecting, result);
	}

	public void testNewlineNormalizationInAttribute() throws Exception {
		StringTemplate st = new StringTemplate(
				"Foo\r\n"+
				"<name>\n",
				AngleBracketTemplateLexer.class
				);
		st.setAttribute("name", "a\nb\r\nc");
		StringWriter sw = new StringWriter();
		st.write(new AutoIndentWriter(sw,"\n")); // force \n as newline
		String result = st.toString();
		String expecting ="Foo\na\nb\nc\n";     // expect \n in output
		assertEquals(expecting, result);
	}


On Jun 4, 2008, at 1:13 PM, Doucette, Charles wrote:

> I'm glad you are looking to fix this, as this is affecting me know  
> (as I reported previously).
>
> We prefer to run ANTLR manually only when our grammar changes, and  
> then commit the latest generated parsers and lexers
> to our source control system (to be compiled with the rest of the  
> sources). The problem is that
> a) our grammar was developed on Windows and has \r\n as line endings
> b) the rest of the lexer/parser generated code (from StringTemplate  
> I assume) has only \n as line endings
> c) our source control system (SVN) will not accept a file with  
> inconsistent line endings
>
> I can work around this in at least these 2 ways:
> 1)      Permanently change the line endings in the grammar file to  
> just use \n
>        (so it will be consistent with the rest of the generated code)
> 2)      Normalize the line endings of the parser/lexer each time  
> they are generated
>        (this is what I am doing now)
>
> It could be fixed permanently if I could configure StringTemplate  
> and/or ANTLR to
> generate lexer/parser code which matches the line endings of my  
> grammar.
>
> FYI, here's a suggested way to configure this.
> Subversion has a property called "svn:eol-style" whose value can be:
> a) native (automatically determined based on the platform)
> b) CRLF
> c) LF
> d) CR
>
> Chuck
>
>> -----Original Message-----
>> From: antlr-interest-bounces at antlr.org
>> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Terence Parr
>> Sent: Wednesday, June 04, 2008 3:52 PM
>> To: stringtemplate-interest
>> Cc: Antlr Interest
>> Subject: [antlr-interest] normalize \n to \n in ST
>>
>> Hi,
>>
>> I am proposing to normalize the use of \n in templates. If
>> you say \n you get \n or \r\n depending on locale.  Actually,
>> maybe we should make it a parameter you send to the template
>> group? You might have a Web server that should be sensitive
>> to the client locale not the server locale, right?
>>
>> When you say wrap="\n" it would actually generate \r\n on a PC.
>>
>> When you have a UNIX file with \n line endings, you would
>> still get \r \n on a PC.
>>
>> The problem is that I really should fix this inside ST, but I
>> need to send out ANTLR v3.1 shortly. given all of the recent
>> bug fixes, should I add this to ST and then release ST 3.2
>> for use with ANTLR 3.1?
>>
>> Ter
>>



More information about the antlr-interest mailing list