[antlr-interest] Semantic predicate losing token/char position on error

[Karim Chichakly]

Hi,

Thank you again for your previous help.  I now know about
antlr.markmail.org(perhaps a link from
www.antlr.org would help others) and have searched for this problem to no
avail.  A year ago, someone asked what may be a related question, but no one
answered.

If you use the grammar I posted last week, and type in "a ab", it will give
you an error and by looking at (token->start - parser->input_start) and
(token->stop - parser->input_start), it is possible to extract the character
position of the erroneous text in the input (giving 2 and 3, respectively,
corresponding to "ab").  [Note token is picked up from
recognizer->state->exception->token and parser is picked up from
base_parser->super, where base_parser = recognizer->super.  The recognizer
is passed by ANTLR to my error handler, which overrides
displayRecognitionError.]

If, however, I add a semantic predicate to that grammar (enclosed) to
distinguish between X as a function call and X as a variable (which is
described starting on page 297 of the Definitive ANTLR Reference), I no
longer get a character position.  All four of the variables involved in the
position calculation are set to 1, and the start and stop then become zero.
These values are, by the way, a bit peculiar as these fields usually hold
pointers into the text.  I also note that token->input is now NULL.

I am guessing that perhaps the erroneous text has been consumed before the
error has been triggered, but I do not really know.  I do need the character
position of the text causing the error and would like to know how to get it
when using a semantic predicate in this manner.  Using a semantic predicate
at the end of the rule works just fine.  It is just this disambiguation
usage that fails to give the token causing the error.

Thank you,

Karim

PS is_function() takes an pANTLR3_STRING and can unconditionally return
false to drive this problem.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Test.g
Type: application/octet-stream
Size: 2112 bytes
Desc: not available
Url : http://www.antlr.org/pipermail/antlr-interest/attachments/20100607/55763903/attachment.obj