[antlr-interest] Preserving Unix end-of-line on Windows

greg94301 at comcast.net greg94301 at comcast.net
Sat Aug 29 15:50:31 PDT 2009



I'm rewriting files with mixed Unix and Windows end-of-line conventions on a Windows system using the Java code-generation target and Antlr 3.1.3. I'd like to preserve the eol as they appear in the source files, but it appears that the template processor is converting Unix eol to Windows eol. This is a rewriting application, and it looks as if only lines that pass through "-> template" are affected. 

Can I prevent this? 

A stripped-down driver looks like this: 
ANTLRInputStream input = new ANTLRInputStream(new FileInputStream(args[0])); 
TestLexer lexer = new TestLexer(input); 
CommonTokenStream tokens = new TokenRewriteStream(lexer); 
TestParser parser = new TestParser(tokens); 

parser.prog(); 

System.out.print(tokens.toString().length()); 

The test grammar looks like this: 
grammar Test; 
options { 
output=template; rewrite=true; 
backtrack=true; 
memoize=true; 
k=2; 
} 
prog 
: 
IDENTIFIER* -> template(r={$text}) "<r>" 
; 

IDENTIFIER 
: LETTER LETTER* 
; 

fragment 
LETTER 
: 'A'..'Z' 
; 

WS : (' '|'\r'|'\t'|'\u000C'|'\n') {$channel=HIDDEN;} 

Sample test file with Unix eol: 
A 
B 
C 

turns into 
A^M 
B^M 
C 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20090829/e674a426/attachment.html 


More information about the antlr-interest mailing list