[antlr-interest] multi-core usage

[Gavin Lambert]

At 04:57 13/06/2008, Loring Craymer wrote:
>Language processing tends to be single threaded--token sequence 
>matters--and ANTLR generates single thread recognizers.  To take 
>advantage of a multicore/multithread environment, the usual 
>approach (excluding server apps) would be to use the recognizer 
>as a task dispatcher that spawns threads according to recognized 
>(from the token stream) semantics.  There are server applications 
>where multiple recognizers are spawned, one per connection.

Another way to use multiple cores (if you're parsing multiple 
files) would be to create one worker thread per core and have each 
of them run a single-threaded parse on a single file.  Set it up 
so there's a queue of files to be processed and whichever thread 
finishes first pulls off the next file from the queue, so you 
don't get duplication.

This is a much simpler setup to pull off, but it's only really 
useful if you've got a lot of files to process, and they're all 
mostly independent.