[antlr-interest] syntax predicate strange behavior

[Loring Craymer]

--- Jim Idle <jimi at temporal-wave.com> wrote:
...
> The fact that backtracking mode works makes me guess
> [pun intended] that
> your syntactic predicates are not quite enough. You

synpreds have been broken in ANTLR 3 from the start;
backtrack synpreds are implemented differently.  This
was supposed to be fixed in b7/8 but has not been,
although Benjamin Niemann's report made it to the
buglist.

The problem derives from Ter's initial investigations
that indicated that synpreds were unnecessary with LL*
and could be "optimized away".  Synpreds are
instantiated within the DFAs, but not at the point
where they are needed.  The simplest example I know of
is (this works with ANTLR 2):

=========================================
lexer grammar TestFilter;
options {
	k = 2;
}

CONTENT
	:
	( ~('\r' | '\n') )+
	;


fragment
SEPARATOR
	:
	"/****"! ('*')*! (' ' | '\t')*! "TEST"
	(	~('*' | '\r' | '\n')
	)+
	('*')+! '/'!
	;

NEWLINE
	:
	('\r')? '\n'
	( (SEPARATOR)=>	SEPARATOR { setType: type =
SEPARATOR; }
	|
	)
	;
================================================
The SEPARATOR predicate within NEWLINE is removed
during code generation.  (The above syntax is not
exactly ANTLR 3--the setType action would be
different).

Backtracking was introduced after Ter ran into non-LL*
grammars; unfortunately, that did not trigger a
re-examination of the synpred implementation. 
Instead, an alternate implementation was added in.  I
expect that this will be fixed in a release or so.  
Fortunately, Ilia's report that backtracking works
gives me a place to start hunting for the problem.

--Loring


> can probably try:
> 
> : (LCURL) => LCURL ( (WORD)=>command_arguments) ?
> RCURL
> |
> ;
> 
> For the shortest predicate that indicates
> command_argument is present.
> If that does not work then you will have to
> predicate the whole of
> command_arguments perhaps, but in that case you are
> effectively doing
> what backtracking mode does.
> 
> The reason you need to predicate command_arguments
> is unclear from the
> context you gave us, but is it possible that that
> rule itself has an
> 'empty' option so that you are confusing things by
> both making it option
> in command_user_body AND optional in and of itself?
> 
> Jim
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com