[antlr-interest] chat message parser rule?
Gavin Lambert
antlr at mirality.co.nz
Mon Jan 26 12:34:01 PST 2009
At 09:16 27/01/2009, ian eyberg wrote:
>Running this through antlrworks I am hitting my 'chatrule'
>every time but it will start chopping up the
>player's message into other tokens that I have defined.
>
>My question: why wouldn't the ALPHA token always be
>chosen over the other tokens?
Lexing is done independently of parsing (and before parsing
begins).
Essentially, when the lexer is "between" tokens it will look at
the input and decide which of the lexer rules best fits the
input. Then it will consume input using that rule until the rule
is no longer satisfied, and then it will exit out and look for
another lexer rule to satisfy.
When you have overlapping definitions (such as ALPHA and LAMB),
this can lead to differences in behaviour depending on the
ordering of the input. (eg. "rack" => LAMB, "racket" => LAMB
ALPHA, "track" => ALPHA)
More information about the antlr-interest
mailing list