Parallel library registration

[marcobICR]

Hi everyone,

I'm trying out lwreg to register an ultra-large library.
Right now it's working great on a susbset of the full library, but the registration process would be too slow to apply to the full library (estimated time: ~180 days!).
Do you know if there's a way for parallelising the registration over multiple CPUs (and I guess writing threads?).

Best,

Marco

[greglandrum]

Hi @marcobICR If you are using postgresql as the backend, it's no problem at all to have multiple processes using lwreg to register compounds simultaneously. I don't think this will work using sqlite as the backend.

As a general warning: if by "ultra-large library" you mean something with billions of compounds then I suspect that you are going to reach the limits of what can be managed by either postgresql or sqlite without putting a lot of thought into how the machine running the database is configured. I've only gone up to a couple of million rows (ChEMBL size) and am pretty sure that everything would be ok with 100 million (PubChem size), but I think billions are likely to be a problem. I would be happy to learn that isn't the case though! :-)