[antlr-interest] Understanding Lexer rules
Johannes Luber
jaluber at gmx.de
Wed Feb 20 07:05:21 PST 2008
Mark Volkmann schrieb:
> On Feb 20, 2008 8:50 AM, Johannes Luber <jaluber at gmx.de> wrote:
>> No. Rereading the text, I suppose one could be confused about that, as
>> it isn't as clear as it could be. Is adding "Longer matches are
>> preferred over shorter matches. If one has two tokens KEY='key'; and
>> KEYWORD='keyword';, then the input 'keyword' will match KEYWORD, even if
>> KEY comes first." enough?
>
> Does the same rule about matching the maximum length apply when the
> lexer rules use the cardinality operators (?, * and +)? I recommend
> adding something about that and not just showing examples with fixed
> literal values.
>
> In case of a tie in the number of characters matched by multiple lexer
> rules, does the first one win?
I honestly never thought about that, but I believe that the first one
has to win, or the "keyword vs. identifier" disambiguation wouldn't
work. Can you whip up some tests to confirm this with a bigger test set?
I am short of time. :(
Johannes
More information about the antlr-interest
mailing list