[antlr-interest] null pointer to ADAPTOR->setTokenBoundaries

[Alan Condit]

I can put two of the productions in separate rules but the first two productions are really one split for simplicity of writing the rewrite rules.

Without the rewrite rules it is this
line	:	line_number? segment* K_NEWLINE
	| 	oword_stmt
	;

With the rewrite rules you can get to this
line	:	line_number? ((segment+)? -> ^(STMT segment+)?) K_NEWLINE
	| 	oword_stmt
		-> ^(STMT oword_stmt)
	;

You can split those two productions into two separate rules but they ultimately have to be combined. Like shown below:
program	:	stmt
	;

stmt	:	line+
	;

line	:	line_number? ((segment+)? -> ^(STMT segment+)?) K_NEWLINE
	| 	oword_stmt
		-> ^(STMT oword_stmt)
	;

So by splitting them you would get something like this:
program	:	stmt
	;

stmt	:	line+
		    ;

line	:	sline
	| 	oline
	;

		// a segment line can have 0 to several segments
		// but segment lines with 0 segments should not be in the AST tree
sline	:	line_number? ((segment+)? -> ^(STMT segment+)?) K_NEWLINE
	;

oline
	| 	oword_stmt
		-> ^(STMT oword_stmt)
	;

The line_number and the K_NEWLINE token are never in the tree. Bottom line is you still have to deal with an empty rewrite rule.

Alan

jim wrote:

I think you will have to put those three productions in separate rules, but I will look into it more.

Jim

> -----Original Message-----
> From: antlr-interest-bounces at antlr.org [mailto:antlr-interest-
> bounces at antlr.org] On Behalf Of Alan Condit
> Sent: Wednesday, May 19, 2010 2:06 PM
> To: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] null pointer to ADAPTOR-
>> setTokenBoundaries
> 
> Jim,
> 
> Here is what I have set in options:
> options {
> 	backtrack 	= 	true;
> 	memoize		= 	true;
> 	language	=	C;
> 	output		=	AST;
> 	ASTLabelType	=	pANTLR3_BASE_TREE;
> 	}
> 
> The null is inside 'ctx' inside 'adaptor' at 'setTokenBoundaries'.
> 
> It is inside a function
> /**
> * $ANTLR start line
> *
> /Users/acondit/source/GCCnv/LatheBranch/trunk/Parser/RS274ngc.g:184:1:
> line : ( ( line_number )? ( segment )+ K_NEWLINE -> ^( STMT ( segment
> )+ ) | ( line_number )? K_NEWLINE -> | oword_stmt -> ^( STMT oword_stmt
> ) );
> */
> static RS274ngcParser_line_return
> line(pRS274ngcParser ctx)
> {
> ...
> }
> 
> which I assume, based on the comment, is generated from this rule:
> line	:	line_number? segment+ K_NEWLINE
> 		-> ^(STMT segment+)
> 	|	line_number? K_NEWLINE
> 		->
> 	|	oword_stmt
> 		-> ^(STMT oword_stmt)
> 	;
> 
> The grammar is for parsing an existing language not one of my
> invention, and grammatically the newlines delineate a semantic block
> therefore must be known by the parser, but empty lines are discarded
> and therefore should not be in the tree.
> 
> Alan
> ---
> 
> Alan's MachineWorks
> 1085 Tierra Ct.
> Woodburn, OR 97071
> 
> Email -- acondit at alansmachineworks.com
> www.alansmachineworks.com
> 
> Jim wrote--
> Please post more information about your grammar, what the null pointer
> is, etc. It is hard to interpolate, but the common mistake is not
> adding output=AST; to the options, so you do not get a tree adaptor
> created.
> 
> Jim
>> -----Original Message-----
>> From: antlr-interest-bounces at antlr.org
> [mailto:antlr-interest-
>> bounces at antlr.org
> ] On Behalf Of Alan Condit
> 
>> Sent: Wednesday, May 19, 2010 11:25 AM
>> To: antlr-interest at antlr.org
>> Subject: [antlr-interest] null pointer to ADAPTOR->setTokenBoundaries
>> 
>> Help!!!
>> 
>> I am getting a null pointer to setTokenBoundaries in the following
> line
>> of generated code.
>> "ADAPTOR->setTokenBoundaries(ADAPTOR, retval.tree, retval.start,
>> retval.stop);"
>> 
>> The grammar works under Java.  In moving it back to 'C', I changed
> the
>> language option to 'C', added option ASTLabelType=pANTLR3_BASE_TREE;
>> and added the necessary includes to compile and link under Objective-
> C.
>> 
>> Is there anything obvious that I am doing wrong?
>> 
>> Thanks,
>> Alan
>>