Multithreading Issue with CLP Linear Solver

[keerthi-ravichandran]

Hello, we are currently working on a scheduling problem using a CLP linear solver in or-tools version 7.4.7247. We are unable to upgrade to a newer version.

The solver is run on each element within our dataset (e.g. if there are 1000 elements, then 1000 different LPs will be created).

We wrote a C++ function to parallelize the variable, constraint, & solver creation. The solver is executed and reset in this loop as well. Our program runs as expected on datasets with sizes less than 150000 elements, but we run into problems with larger ones.

We tested our program on a very large dataset (~3 million elements) on the following 2 machines:

Machine A specs:
CPU: 11th Gen Intel(R) Core(TM) i7-11850H @2.50 GHz
RAM: 32 GB

Machine B specs:
CPU: Intel(R) Xeon(R) E5-2690 0 @2.90 GHz (2 processors)
RAM: 128 GB

We used the psutil library to record CPU and RAM usage throughout the run of our program, and found that the multithreading executes as expected on Machine A (CPU usage peaks at 96%), but does not at all on Machine B (CPU usage peaks at 12%).

We retested with a simpler LP model for each element (number of constraints vary per LP, but does not exceed 30) on Machine B, and found that the multithreading occurs (CPU usage peaks at 96%). However if we add any more constraints, the multithreading does not occur (CPU usage never exceeds 15%).

Does anyone have an idea on why multithreading is not occurring on Machine B under the above circumstances? Has anyone else experienced any issues multithreading with or-tools?

Let me know if you need more information on our program or our machines. Any help is greatly appreciated. Thank you in advance.

[lperron]

it seems you are memory bandwidth bound.