[antlr-interest] Selective ignoring of whitespace

[Michael]

Am Monday 18 May 2009 16:00:03 schrieb Ratul Bhadury:
> Thanks for the tips guys.
>
> I've made some progress but seeing one peculiarity which I would be
> grateful if someone could explain.
>
> Single line input like 'sec=      sec,    dfd    dfdf' is accepted as I
> intend, i.e sec & dfd    dfdf are regarded as search_terms. However if I
> put a space before the '=', the line cannot be parsed. I tried putted a WS*
> element in the definition of keyword_field, and also tried putting it in
> keyword_clause, but they made no difference.
>
> Can someone explain to me why this is the case?  Grammar is below.

if you write a space after your keyword_field then the lexer sees that the 
KEYWORD rule gives a longer match than keyword_field and so matches to the 
KEYWORD rule (it includes the spaces):

> KEYWORD        : CHAR(CHAR|' ')*;

which later fails since a wrong token is returned.

 Michael