[antlr-interest] comments, newlines, and emit()

Jim Idle jimi at temporal-wave.com
Fri Aug 3 14:17:07 PDT 2007


I should mention for the C wallas, that this is built in to the input
stream. If you spec your tokens all in upper case, then you can do this:

 

 

input = antlr3NewAsciiStringInPlaceStream(charptr, len, "name");

input->setUcaseLA(input, ANTLR3_TRUE);

 

and it all happens magically for you. The LA function can also be
overridden of course in case your upper case conversion is not defined
by the C library toupper() function.

 

Jim

 

From: antlr-interest-bounces at antlr.org
[mailto:antlr-interest-bounces at antlr.org] On Behalf Of Andy Tripp
Sent: Friday, August 03, 2007 2:03 PM
Cc: antlr-interest
Subject: Re: [antlr-interest] comments, newlines, and emit()

 

Yes, that does it.
Thanks, Jim. Thanks for the other answer on case sensitivity, too!

Andy

Jim Idle wrote: 

Andy,
 
If you need the new line, then why not remove it from the match?
 
LINE_COMMENT
    : '\'' ~('\n'|'\r')* {$channel=HIDDEN;}
    ;
 
The normal rule for newline should then pick it up.
 
Jim
 
  

	-----Original Message-----
	From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
	bounces at antlr.org] On Behalf Of Andy Tripp
	Sent: Friday, August 03, 2007 1:06 PM
	To: antlr-interest; Andy Tripp
	Subject: [antlr-interest] comments, newlines, and emit()
	 
	I'm parsing a language (visual basic), in which newlines are
	significant,
	and it also allows single line comments. My comment rule is the
	standard
	one that matches a single quote up to and including the next
newline:
	 
	LINE_COMMENT
	    : '\'' ~('\n'|'\r')* '\r'? '\n' {$channel=HIDDEN;}
	    ;
	 
	The problem is that need to keep the '\n' on the DEFAULT
channel, and
	put the rest on the HIDDEN channel.
	 
	According to the "Emitting More Than One Token per Lexer Rule"
section
	of the book,
	it seems like I should be able to emit the newline on the
DEFAULT
	channel like this:
	 
	LINE_COMMENT
	    : '\'' ~('\n'|'\r')* '\r'? newline='\n' {emit(newline);
	$channel=HIDDEN;}
	    ;
	 
	Yet that generates an emit() call that takes an int, which
doesn't
	compile.
	 
	There must be an easy way to either emit an extra newline token
here.
	Anyone?
	 
	Andy
	    

 
 
  

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.antlr.org/pipermail/antlr-interest/attachments/20070803/c577cfb6/attachment-0001.html 


More information about the antlr-interest mailing list