[antlr-interest] "new CommonToken" issued in Java target in spiteof setting TokenLabelType to something else

[Hardy, Stephen]

OK, don't bother answering.  For the record, here is how to modify a
string template.  Replace / with \ if on windows.

Suppose you have installed in c:/antlr-3.0.

Go to c:/antlr-3.0/src/org/antlr/codegen/templates
and make a new directory (e.g. JavaFix).  Copy all .stg files from the
broken set (say, Java) and make modifications as desired.

Go to c:/antlr-3.0/src and issue the command
  jar cf ../lib/JavaFix.jar org/antlr/codegen/templates/JavaFix

The 'jar' command is found in your JDK distribution e.g. c:/program
files/java/jdk1.6.0_02/bin.

The ../lib directory is where all the antlr jar files are kept.  Add
JavaFix.jar to the classpath when running Antlr.

Modify your .g file to add the language=JavaFix option to the main
options section.

Most of the above is explained in TDAR and other places, but I have
never found a step-by-step guide for dummies like me.

Regards,
SJH

PS: unfortunately, replacing "CommonToken" in Java.stg with <LabelType>
fixed the original error, but now I have another puzzler in that
imaginary tokens seem to be created with a base Token type, not MyToken.
Sigh...


> -----Original Message-----
> From: antlr-interest-bounces at antlr.org 
> [mailto:antlr-interest-bounces at antlr.org] On Behalf Of Hardy, Stephen
> Sent: Thursday, August 02, 2007 11:07 AM
> To: antlr-interest at antlr.org
> Subject: [antlr-interest] "new CommonToken" issued in Java 
> target in spiteof setting TokenLabelType to something else
> 
> There seems to be a bug in the Java target string template.  
> Even though
> I set TokenLabelType = MyToken in the parser/lexer options, the lexer
> template is generating code like
> 
>             MyToken DECIMAL_LITERAL1=null;
> 		...
>             DECIMAL_LITERAL1 = new CommonToken(input,
> Token.INVALID_TOKEN_TYPE, 
>                  Token.DEFAULT_CHANNEL, 
>                  DECIMAL_LITERAL1Start1480, 
> getCharIndex()-1);		
> 
> (which causes a type mismatch compile error) for .g file input like
> 
> LINE_COMMAND 
> 	: '#' WSNONL+ 
> 		DECIMAL_LITERAL 
> 		{
> 			line_offset =
> Integer.parseInt($DECIMAL_LITERAL.text) - tokenStartLine;
> 		}
> 		WSNONL+ 
> 		STRING_LITERAL 
> 		{
> 			currentFile = $STRING_LITERAL.text;
> 			file_changed = true;
> 		}
> 		~('\n'|'\r')* '\r'? '\n' {skip();}
>     ;
> 
> The error seems to be fixable by modifying the Java.stg (string
> template) file in the source, however I have not had previous 
> success in
> modifying the templates owing to my abysmal lack of understanding of
> Java etc.
> 
> Can anyone tell me (preferably in words of 1 syllable or less) how to
> get Antlr to actually read a modified template file?  Where 
> do I have to
> stick it to get it to be read?  Somebody mentioned putting it in the
> classpath, but the classpath is not really a path but a list of jar
> files.  How do I add the template files to a jar file??!!??
> 
> Regards,
> SJH
>