[antlr-interest] streaming parser?
Jim Idle
jimi at temporal-wave.com
Sat Sep 19 16:01:09 PDT 2009
On 09/19/2009 12:12 PM, Karl Meissner wrote:
> Are there any good article about using antlr in the context of a
> steaming parser?
Search the list for prior discussions, but all you really need is to
write your own input stream that gives EOF back to the caller when a
complete "unit" is detected, then keep setting up the input stream
again. Presumably there is something in the incoming packets that
indicates a safe place to call something a "unit" of work?
>
> I have text files which I need to filter and search as they download,
> packet by packet. The packets may fragment the text tokens so I need
> some sort of stateful stream....
>
>
> It would be really nice if I could arrange for the parse to run
> deterministically. I am hoping to use Antlr in a highly optimized, low
> latency application. boost regex was not up to the task...
Depends what you mean by deterministically? You mean that after nnms of
cpu or elapsed time that it gives up control to an embedded system
control loop? Or that it always takes the same time? You will want to
increase the value of -Xmaxdfaedges to a large number to avoid DFAs
being generated as tables (inline Ifs are much faster because of branch
prediction and avoiding memory cache misses).
BTW - I presume that you are using the C target here as you have no
chance of achieving the kind of performance you seem to be asking for
from the other targets.
Jim
More information about the antlr-interest
mailing list