Pool
is faster than ThreadPool
in some situations
#57
Labels
documentation
Improvements or additions to documentation
Pool
is faster than ThreadPool
in some situations
#57
Not really a bug, but I'm posting this here to make it easier to find. We talked about it over email before, but someone brought it up to me again. Figured it'd be good to make this information searchable.
Pyrodigal's CLI command uses
multiprocessing.pool.ThreadPool
for parallel gene calling. The documentation suggests usingThreadPool
too. But in my benchmarks,multiprocessing.pool.Pool
was way faster. I mentioned this to @althonos, who did some tests and found thatPool
is faster thanThreadPool
when there are lots of CPUs available (though we didn't do super in-depth tests to figure out why).If Pyrodigal is running slow for you, try using
multiprocessing.pool.Pool
instead. There's an example you can check out here.The text was updated successfully, but these errors were encountered: