[antlr-interest] Lexer matching non-matching rule

[Jesper Larsson]

On Sun, 2009-05-17 at 12:48 +0200, Micha wrote:
> On Sunday 17 May 2009 11:01:36 Jesper Larsson wrote:
> > Not always so easy, however. My original example was, even more
> > simplified, something like this:
> >
> > FOO:    'foo';
> > BAR:    'bar';
> > FOOZ:   'foo'* 'z';
> >
> 
> that works with the method mentioned earlier:

Hm, as far as I can tell your variant has exactly the same problems as
mine. The following grammar, incorporating the trick from yours, can
match "foobar" as FOO BAR but fails on "foofoobar":

============================================
grammar Y;
options { output=AST; }

file: (FOO|BAR|FOOZ)* EOF;

fragment FOO:   ;
fragment FOOZ:  ;
BAR:            'bar';
WS:             (' ' | '\n')+ { skip(); };

FOO_OR_FOOZ:    (
    'foo' { $type = FOO; }
    ('foo'* 'z' { $type = FOOZ; })?
)
| 'z' { $type = FOOZ;};
============================================

J'