[antlr-interest] Short circuit of the lexer
Xavier Decoret
decoret at graphics.lcs.mit.edu
Fri Jan 17 14:24:03 PST 2003
I have a grammar with arrays of int defined as follow:
[ 12, 12, 4, 5, 6 ]
The number of numbers in an array can be quite large (several thousands)
and I found that antlr (though it is *really* a cool stuff that
simplified my life so much) is pretty slow at this. What I would like to
do is entering a special loop when the parser figures out it has started
a list. It would then consume by itself the int to read using fast code
reading directly the flow of char until it finds a closing bracket which
has to be put back in the flow.
My idea is to overide the lexer::nextToken class and use a state
variable to decide in that function wether to use the nexttoken build by
ANTLR or use a local loop that directly reads the input stream until it
finds a closing bracket in which case it returns that Token, and
everything continues as if nothing happens.
But I am not sure how this would fit with the LA() functions if I remove
things of the input stream of the lexer without telling him.
Is what I want to do something known?
Are they any caveheat I will run into?
Is it the correct way to speed up this part of the parsing?
Thanks for any hint.
Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
More information about the antlr-interest
mailing list