[antlr-interest] (follow up) setting, altering text in lexer rules
Terence Parr
parrt at cs.usfca.edu
Mon Jun 12 12:30:39 PDT 2006
ok, so lexing time when from 1340ms to 2095ms when I added all this
crap in there. That is more than 1.5x the cost in time. I can't
justify that 50% increase in lexer time. Also the parser is taking
more time...weird...a GC issue?
Wow. the
text.setLength(0);
in the nextToken() method costs 200ms out of that 2095ms. When I
remove all this machinery it goes back to what it was in my notes
time-wise so it's not a fluke. Rats!
Ok, I propose that we take a big step back and say "you can set the
text for the token manually". You get a setText() method and the
auto mechanism will see your altered text if nonnull. If you want to
build up a token piecemeal you must do so manually. So you'd do this:
ESC : '\\' 'n' {setText("\n");} ;
I still need to spend time inc/dec the rule level though so I know
when to emit a token. It seems to cost a wee bit but that is ok I
guess.
Ter
More information about the antlr-interest
mailing list