[antlr-interest] Expr.text null?

[Foust]

> From: Terence Parr [mailto:parrt at cs.usfca.edu]
> Sent: Tuesday, August 12, 2008 3:32 PM
> To: Foust
> Cc: antlr-interest at antlr.org
> Subject: Re: [antlr-interest] Expr.text null?
> 
> should work.  java target right?  did you setTokenStream on the node
> stream?
> Ter

Yes, Java target, ASTLabelType set like this:

  tree grammar RunModl;
  options {
	tokenVocab=Modl;
	ASTLabelType=CommonTree;
  }

And the rule is just a little more complicated, in that
	expr: a? b;

is actually more like:
	
	expr: ^(EXPR_TOKEN a? b)
		{ $a.text is also null here }

	a   options {backtrack=true; memoize=true;}	
	   : ^(PART1_TOKEN IDENT ...)
		{ $IDENT.text is correct here }


I'm wondering if the backtrack on that single rule in the tree grammar might
be causing the strangeness?

Anyway, the generated calls to get the text both return -1:

	input.getTreeAdaptor().getTokenStartIndex(a.start), and
	input.getTreeAdaptor().getTokenStopIndex(a.start)

Where "a" is the actual Java variable generated for expression "a".

I think I'm going to just have to use a return parameter on the subrule to
get the proper text.

Note:
	1. $b.text works fine in rule "expr", while:

	2. $a.text returns null

	3. Both rule "a" and "b" are actually matching identical subtrees
	   (using a different parent node token)

		subRuleA : (^SUB_TOKENA identifier)
		subRuleB : (^SUB_TOKENB identifier)

	4. differences:
		"a" is optional in "expr" (uses a closure)
		rule "a" uses a backtrack option (the only one in the file)

	5. if subrule "a" returns its subrule's text as a return parameter,
then "expr" sees it just fine.

	   subRuleA : returns [String name]
			  (^SUB_TOKENA identifier)
			  { $subRuleA.name = $identifier.text; } // this
works!

	  Now, although "expr" cannot see $subRuleA.text (null), it can see
$subRuleA.name.

Brent


> On Aug 12, 2008, at 3:23 PM, Foust wrote:
> 
> > Does it make sense that a matched sub-rule returns null for its text
> > attribute?
> >
> > That is, the following rule matches the input AST just fine.
> >
> > tree grammar;
> > expr : a? b;
> >
> > a : ^(SOME_TOKEN someargs)
> > b: ^(etc.)
> >
> >
> > But when the text of a is desired, it always returns null, whether
> > in the rule action, or in a sub-rule action, as below:
> >
> > expr : (a {System.out.println($a.text); } )?  b  ;
> >
> > prints "null".
> >
> >