-
|
First of all, thank you to the creaters and maintainers of Hera. This has been such a godsend. Was wondering if its possible to use argo for model serving as opposed to training/ batch jobs. It is possible to deploy a k8s app that will host a dockerised fastai endpoint (that can autoscale according to requests). I'm hoping that given the underlying k8s architecture there is a way to make a persistent (say fastapi) endpoint with Hera. If so how would I do that? TIA. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
For reference, this was x-posted to You can also use Workflows to create In short, there are purpose built tool stacks for each of these things, although you can certainly mix some parts together.
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks @agilgur5! +1 on KServe, and using Workflows to update deployments worked for us in the past. |
Beta Was this translation helpful? Give feedback.
For reference, this was x-posted to
#argo-workflowsSlack, where I responded and said that while Argo can quite easily fit a batch serving model, for API-driven real-time serving, KServe/Seldon/etc are a better fit and I have used them respectively for batch vs real-time inference.You can also use Workflows to create
DeploymentsorInferenceServices(i.e. your MLOps pipelines), but CD may suffice for that too.In short, there are purpose built tool stacks for each of these things, although you can certainly mix some parts together.
Also this sounds like it should've been a Discussion rather than an issue.EDIT: This has now been converted into a Discussion