[antlr-interest] ANTLR 3.0.1 C Runtime - Bug and possible correction in displayRecognitionError()

[Florian Pasco]

I've had invalid memory access errors in the C runtime of ANTLR 3.0.1
during tests where the parser was fed invalid inputs. These errors
occurred in displayRecognitionError() in antlr3baserecognizer.c (line
822) :
		fprintf(stderr, " : expected %s ...\n", tokenNames[ex->expecting]);
when ex->expecting was 0xFFFFFFFF, which is the token for EOF. The
error would then be an array access with an index of -1.

To correct this, I've then set my own error display handler via
@apifuncs and used an exact copy of displayRecognitionError() with
just the offending line replaced by these 2 lines (C++ instead of C
used here) :
            ANTLR3_UINT8* tokenName = ex->expecting ==
ANTLR3_TOKEN_EOF ? (ANTLR3_UINT8*)"<EOF>" : tokenNames[ex->expecting];
            fprintf(stderr, " : expected %s ...\n", tokenName);
and everything runs just fine.

I'm not sure this is the best way to handle this since if there are
accesses to the table pointed to by tokenNames elsewhere in the code,
the same bug will happen. The cleanest way would be to prevent any
token from having a negative value but the EOF token seems to be
hardcoded to 0xFFFFFFFF for a reason.