[antlr-interest] Matching compound keywords in the lexer
Franck Mangin
franck102 at yahoo.com
Fri Nov 25 21:33:43 PST 2011
Thanks all for the replies.
As far as I know just the tokens won't work, the parser complains that "no *rule* defines token xxx" - which is why I hadn't tried fragments, I
assumed I would get the same complaint, but that actually works
perfectly.
Franck
________________________________
From: Benjamin S Wolf <jokeserver at gmail.com>
To: Jim Idle <jimi at temporal-wave.com>
Cc: Bart Kiers <bkiers at gmail.com>; franck102 <franck102 at yahoo.com>; "antlr-interest at antlr.org" <antlr-interest at antlr.org>
Sent: Friday, November 25, 2011 11:16 PM
Subject: Re: [antlr-interest] Matching compound keywords in the lexer
You can do it either way, I guess. I use just the tokens def myself.
On Fri, Nov 25, 2011 at 2:07 PM, Jim Idle <jimi at temporal-wave.com> wrote:
> You don't need the tokens def. just the lexer fragment.
>
> Jim
>
> On Nov 25, 2011, at 12:54, Bart Kiers <bkiers at gmail.com> wrote:
>
>> Hi Franck,
>>
>> On Fri, Nov 25, 2011 at 9:47 PM, franck102 <franck102 at yahoo.com> wrote:
>>
>>> ...
>>
>> containOperator : CONTAINS_TEXT | CONTAINS_MATCH
>>>
>>> CONTAINS_TEXT
>>> : 'contains' WS+ ( 'match' { $type=CONTAINS_MATCH }
>>> | 'text' )
>>> ;
>>>
>>> // CONTAINS_MATCH:; // causes "token definitions can never be matched"
>>> error
>>>
>>>
>> Add CONTAINS_MATCH to your @tokens{...} and create an empty fragment rule
>> called CONTAINS_MATCH to silence the warning:
>>
>> -----------------------------
>>
>> tokens {
>> CONTAINS_MATCH;
>> }
>>
>> ...
>>
>> CONTAINS_TEXT
>> : 'contains' WS+ ( 'match' CONTAINS_MATCH
>> | 'text' )
>> ;
>>
>> ...
>>
>> fragment CONTAINS_MATCH : ;
>>
>> -----------------------------
>>
>> Regards,
>>
>> Bart.
>>
>> List: http://www.antlr.org/mailman/listinfo/antlr-interest
>> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
> List: http://www.antlr.org/mailman/listinfo/antlr-interest
> Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address
>
More information about the antlr-interest
mailing list