[antlr-interest] How do I get rid of "unexpected token:null" error

[Greg Lindholm]

I believe the problem is all your lexer rules expect IMG are protected.
Protected lexer rules do not return tokens.

Your parser statement rule wants to see WORD and STRING tokens so you
need to make these rules un-protected.

Greg
--- sachin_sawant10 <sachin_sawant10 at yahoo.com> wrote:
> I am using following grammer to parse JSP file. I want to read 
> <i18n:img tags from the jsp page.
> 
> My grammer ---
> 
> class JSPParser extends Parser;
> 
> file:   (statement)+ 
> 	;
> 
> statement: (   WORD
>              | STRING
> 	     | i:IMG {System.out.println("IMG Tag " + i.getText());}
>       )
> 	;
> 
> 
> class JSPLexer extends Lexer;
> options {
> 	caseSensitive=false;
> 	k=7;
> 	filter=SCARF;
> 	charVocabulary='\3'..'\177';
> }
> 
> 
> IMG    
>         :      	"<i18n:img" 
> 		(WS (ATTR)*)?       
> 		"/>"		
> 	;
> 
> 
> protected
> ATTR
> options {
> 	ignore=WS;
> }
> 	:	w:WORD '='
> 		(	s:STRING
> 		|	v:WORD
> 		)
> 	;
> 
> /** Match until next whitespace; can be file, int, etc... */
> protected
> WORD:	(
> 			options {
> 				generateAmbigWarnings=false;
> 			}
> 		:	'a'..'z' | '0'..'9' | '/' | '.' | '#' | '_'
> 		)+
> 	;
> 
> protected
> STRING
> 	:	'"' (~'"')* '"'
> 	|	'\'' (~'\'')* '\''
> 	;
> 
> 
> protected
> WS	:	(	' '
> 		|	'\t'
> 		|	'\f'
> 		|	(	"\r\n"  // DOS
> 			|	'\r'    // Macintosh
> 			|	'\n'    // Unix (the right way)
> 			)
> 			{ newline(); }
> 		)
> 		
> 	;
> 
> protected
> SCARF
> 	:	WS	// track line numbers while you scarf
> 	|	.
> 	;
> -----------------------end of grammer
> --------------------------------
> 
> When I used the above grammer with any jsp files which does not 
> have "<i18n:img" tag I get "unexpected token:null" error.  I also get
> 
> the error when I use empty file as an input.
> 
> How do I get rid of it?
> 
> Regards
> 
> Sachin 
> 
> 
> 
> 
>  
> 
> Your use of Yahoo! Groups is subject to
> http://docs.yahoo.com/info/terms/ 
> 
> 


__________________________________________________
Do You Yahoo!?
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com

 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/