New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Expose warmup thread pool to model server #3660

Open

corona10 wants to merge 2 commits into tensorflow:master from corona10:warmup-thread-pool

+12 −4

corona10 commented Jan 3, 2025

Please let me know if I misunderstood the original intention.

Parallel warm-up options were exposed a very long time ago, but AFAIK, there is no way to set it on the model server side.
- 79ac1fd
- 79f9d28
So it would be great if we can set this, and even if we use one thread, it would be beneficial to separate the specific thread pool ("Warmup_ThreadPool") if the user specify the number of thread explicitly.

Please let me know if I missed something or if I understand wrongly.
I just try to understand TensorFlow serving internally.

corona10 added 2 commits

January 4, 2025 00:22


          feat: Expose warmup thread pool to model server

60ad1fc

fix

de30142

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet