Wandb.init latency problems

[jdai1]

Hello,

I've been trying Weave out, and I think it's good so far. However, I have a problem with the startup times of wandb.init. It's adding about 3-4 seconds of latency which is a big problem for deploying the tracing in production. I've narrowed down the problem (I believe) to the autopatching of many different types of LLMs. I'm wondering if there could be an option added such that autopatching is turned off, or the user can specify which autopatchings are desired, such that ones that are not needed do not pile onto the latency.

Any other ideas to improve latency would also be welcome.

Thanks!
Julian

[jamie-rasmussen]

In the PR above I've created a small benchmarking script for weave.init, and added an option to disable the autopatching. I'll follow up with our team about whether this is an approach we want to ship. In the small amount of testing I did on my machine, disabling autopatch brings the init time from roughly 1.5 seconds down to 0.5 seconds. (Since these numbers are different than the 3-4s you experienced it could be helpful if you were willing to run that script on your own machine.)

I timed a few other things that happen in weave.init, e.g. our version check takes about 0.1s. Besides the autopatching I didn't see any immediately obvious quick performance wins.

[jdai1]

Thank you so much for the quick turn around! I think the feature is useful and would be good for users to ship. Quick question: in your PR, you mentioned there already is a way to disable single integrations. Could you point me to something or provide a snippet that shows how that can be done please? Couldn't find anything in the documentation so far.