[antlr-interest] Token parsing speed

[Richard Druce]

I have a question on the general best practice and speed between of
using  tokens vs rules to construct parts of the grammar. Our language
has many phrases that share words, a simplified sample being 'first'
and 'first second'.  Would I be better off putting them in as tokens
or rules from a speed of parsing perspective. Some of my tokens also
contain whitespace.

i.e
rule1: FIRST;

rule2: FIRST WS SECOND;

FIRST: 'first';
SECOND: 'second'

WS: ' ';

or

rule1: RULE1;

rule2: RULE2;

RULE1: 'first second';

RULE2: 'first';

Thanks,

Richard