[antlr-interest] normalize \n to \n in ST

[Doucette, Charles]

By the way, will this change ultimately resolve the issue with consistent line endings in files generated by ANTLR
(for those of us not using StringTemplate directly)?

Right now I am forced to do this:

> > 2)      Normalize the line endings of the parser/lexer each time
> > they are generated
> >        (this is what I am doing now)

Since this didn't work:

> > 1)      Permanently change the line endings in the grammar file to
> > just use \n
> >        (so it will be consistent with the rest of the
> generated code)

Thanks,
Chuck

> -----Original Message-----
> From: Terence Parr [mailto:parrt at cs.usfca.edu]
> Sent: Wednesday, June 04, 2008 4:33 PM
> To: Doucette, Charles
> Cc: stringtemplate-interest; Antlr Interest
> Subject: Re: [antlr-interest] normalize \n to \n in ST
>
> Here are 3 of my unit tests. Basically, no matter which you
> used in templates or attributes or options, ST normalizes
> everything to be the newline field of the output writer
> AutoIndentWriter and its subclass NoIndentWriter. Pushing. I
> guess I need to push 3.2 ST soon :)
>
> Ter
>
>         public void
> testNewlineNormalizationInTemplateString() throws Exception {
>                 StringTemplate st = new StringTemplate(
>                                 "Foo\r\n"+
>                                 "Bar\n",
>                                 AngleBracketTemplateLexer.class
>                                 );
>                 StringWriter sw = new StringWriter();
>                 st.write(new AutoIndentWriter(sw,"\n")); //
> force \n as newline
>                 String result = st.toString();
>                 String expecting ="Foo\nBar\n";     // expect
> \n in output
>                 assertEquals(expecting, result);
>         }
>
>         public void
> testNewlineNormalizationInTemplateStringPC() throws Exception {
>                 StringTemplate st = new StringTemplate(
>                                 "Foo\r\n"+
>                                 "Bar\n",
>                                 AngleBracketTemplateLexer.class
>                                 );
>                 StringWriter sw = new StringWriter();
>                 st.write(new AutoIndentWriter(sw,"\r\n")); //
> force \r\n as newline
>                 String result = st.toString();
>                 String expecting ="Foo\r\nBar\r\n";     //
> expect \r\n in output
>                 assertEquals(expecting, result);
>         }
>
>         public void testNewlineNormalizationInAttribute()
> throws Exception {
>                 StringTemplate st = new StringTemplate(
>                                 "Foo\r\n"+
>                                 "<name>\n",
>                                 AngleBracketTemplateLexer.class
>                                 );
>                 st.setAttribute("name", "a\nb\r\nc");
>                 StringWriter sw = new StringWriter();
>                 st.write(new AutoIndentWriter(sw,"\n")); //
> force \n as newline
>                 String result = st.toString();
>                 String expecting ="Foo\na\nb\nc\n";     //
> expect \n in output
>                 assertEquals(expecting, result);
>         }
>
>
> On Jun 4, 2008, at 1:13 PM, Doucette, Charles wrote:
>
> > I'm glad you are looking to fix this, as this is affecting
> me know (as
> > I reported previously).
> >
> > We prefer to run ANTLR manually only when our grammar changes, and
> > then commit the latest generated parsers and lexers to our source
> > control system (to be compiled with the rest of the sources). The
> > problem is that
> > a) our grammar was developed on Windows and has \r\n as line endings
> > b) the rest of the lexer/parser generated code (from
> StringTemplate I
> > assume) has only \n as line endings
> > c) our source control system (SVN) will not accept a file with
> > inconsistent line endings
> >
> > I can work around this in at least these 2 ways:
> > 1)      Permanently change the line endings in the grammar file to
> > just use \n
> >        (so it will be consistent with the rest of the
> generated code)
> > 2)      Normalize the line endings of the parser/lexer each time
> > they are generated
> >        (this is what I am doing now)
> >
> > It could be fixed permanently if I could configure StringTemplate
> > and/or ANTLR to generate lexer/parser code which matches the line
> > endings of my grammar.
> >
> > FYI, here's a suggested way to configure this.
> > Subversion has a property called "svn:eol-style" whose value can be:
> > a) native (automatically determined based on the platform)
> > b) CRLF
> > c) LF
> > d) CR
> >
> > Chuck
> >
> >> -----Original Message-----
> >> From: antlr-interest-bounces at antlr.org
> >> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Terence Parr
> >> Sent: Wednesday, June 04, 2008 3:52 PM
> >> To: stringtemplate-interest
> >> Cc: Antlr Interest
> >> Subject: [antlr-interest] normalize \n to \n in ST
> >>
> >> Hi,
> >>
> >> I am proposing to normalize the use of \n in templates. If
> you say \n
> >> you get \n or \r\n depending on locale.  Actually, maybe we should
> >> make it a parameter you send to the template group? You
> might have a
> >> Web server that should be sensitive to the client locale not the
> >> server locale, right?
> >>
> >> When you say wrap="\n" it would actually generate \r\n on a PC.
> >>
> >> When you have a UNIX file with \n line endings, you would
> still get
> >> \r \n on a PC.
> >>
> >> The problem is that I really should fix this inside ST,
> but I need to
> >> send out ANTLR v3.1 shortly. given all of the recent bug fixes,
> >> should I add this to ST and then release ST 3.2 for use with ANTLR
> >> 3.1?
> >>
> >> Ter
> >>
>
>