diff --git a/docs/concepts/mlops/kserve.md b/docs/concepts/mlops/serving.md similarity index 54% rename from docs/concepts/mlops/kserve.md rename to docs/concepts/mlops/serving.md index 18130de90..6a49a49c2 100644 --- a/docs/concepts/mlops/kserve.md +++ b/docs/concepts/mlops/serving.md @@ -1,4 +1,4 @@ -In Hopsworks, you can easily deploy models from the model registry in KServe or in Docker containers (for Hopsworks Community). KServe is an open-source framework for model serving on Kubernetes. You can deploy models in either programs, using the HSML library, or in the UI. A KServe model deployment can include the following components: +In Hopsworks, you can easily deploy models from the model registry in KServe or in Docker containers (for Hopsworks Community). KServe is the defacto open-source framework for model serving on Kubernetes. You can deploy models in either programs, using the HSML library, or in the UI. A KServe model deployment can include the following components: **`Transformer`** @@ -15,16 +15,15 @@ In Hopsworks, you can easily deploy models from the model registry in KServe or **`Inference Batcher`** -: Inference requests can be batched in different ways to adjust the trade-off between throughput and latencies of the model predictions. +: Inference requests can be batched to improve throughput (at the cost of slightly higher latency). **`Istio Model Endpoint`** -: You can publish a model via a ^^REST or gRPC Endpoint^^ using Istio and access it over HTTP using a Hopsworks API key (with serving scope). Secure and authorized access is guaranteed by Hopsworks. +: You can publish a model over ^^REST(HTTP)^^ or ^^gRPC^^ using a Hopsworks API key. API keys have scopes to ensure the principle of least privilege access control to resources managed by Hopsworks. - -Models deployed on KServe in Hopsworks can be easily integrated with the Hopsworks Feature Store using a Transformer Python script, that builds the predictor's input feature vector using the application input and pre-computed features from the Feature Store. +Models deployed on KServe in Hopsworks can be easily integrated with the Hopsworks Feature Store using either a Transformer or Predictor Python script, that builds the predictor's input feature vector using the application input and pre-computed features from the Feature Store. !!! info "Model Serving Guide" - More information can be found in the [Model Serving guide](../../../user_guides/mlops/serving/index.md). \ No newline at end of file + More information can be found in the [Model Serving guide](../../user_guides/mlops/serving/index.md). \ No newline at end of file diff --git a/docs/concepts/mlops/training.md b/docs/concepts/mlops/training.md index f03ed6fed..517cda91f 100644 --- a/docs/concepts/mlops/training.md +++ b/docs/concepts/mlops/training.md @@ -6,4 +6,4 @@ If you train models with Hopsworks, you can setup CI/CD pipelines as shown below -Hopsworks [Model Registry](registry.md) and [Model Serving](kserve.md) capabilities can then be used to build a batch or online prediction service using the model. +Hopsworks [Model Registry](registry.md) and [Model Serving](serving.md) capabilities can then be used to build a batch or online prediction service using the model. diff --git a/docs/index.md b/docs/index.md index 91e563f4c..9d5ec6575 100644 --- a/docs/index.md +++ b/docs/index.md @@ -219,7 +219,7 @@ pointer-events: initial;
-
Model Serving
+
Model Serving
diff --git a/docs/user_guides/mlops/serving/api-protocol.md b/docs/user_guides/mlops/serving/api-protocol.md index 266a0af1f..886bdf3b1 100644 --- a/docs/user_guides/mlops/serving/api-protocol.md +++ b/docs/user_guides/mlops/serving/api-protocol.md @@ -5,11 +5,11 @@ Hopsworks supports both REST and gRPC as API protocols for sending inference requests to model deployments. While REST API protocol is supported in all types of model deployments, support for gRPC is only available for models served with [KServe](predictor.md#serving-tool). !!! warning - At the moment, gRPC API protocol is only supported for **Python model deployments** (e.g., scikit-learn, xgboost, ...). Support for Tensorflow model deployments is coming soon. + At the moment, the gRPC API protocol is only supported for **Python model deployments** (e.g., scikit-learn, xgboost). Support for Tensorflow model deployments is coming soon. ## GUI -### Step 1: Create new deployment +### Step 1: Create a new deployment If you have at least one model already trained and saved in the Model Registry, navigate to the deployments page by clicking on the `Deployments` tab on the navigation menu on the left. @@ -20,11 +20,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Go to advanced options -A simplified creation form will appear including the most common deployment fields among all the configuration possible. Resource allocation is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`. +A simplified creation form will appear including the most common deployment fields from all available configurations. Resource allocation is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`.

@@ -46,8 +46,8 @@ Enabling gRPC as the API protocol for a model deployment requires KServe as the Then, you can select the API protocol to be enabled in your model deployment. -!!! note "Only one API protocol can be enabled simultaneously" - Currently, KServe model deployments are limited to one API protocol at a time. Therefore, only one of REST or gRPC API protocols can be enabled at the same time on the same model deployment. +!!! info "Only one API protocol can be enabled for a model (they cannot support both gRPC and REST)" + Currently, KServe model deployments are limited to one API protocol at a time. Therefore, only one of REST or gRPC API protocols can be enabled at the same time on the same model deployment. You can change the API protocol of existing deployments.

diff --git a/docs/user_guides/mlops/serving/deployment.md b/docs/user_guides/mlops/serving/deployment.md index 9d696345d..2a9978c75 100644 --- a/docs/user_guides/mlops/serving/deployment.md +++ b/docs/user_guides/mlops/serving/deployment.md @@ -27,11 +27,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Basic deployment configuration -A simplified creation form will appear including the most common deployment fields among all the configuration possible. We provide default values for the rest of the fields, adjusted to the type of deployment you want to create. +A simplified creation form will appear including the most common deployment fields from all available configurations. We provide default values for the rest of the fields, adjusted to the type of deployment you want to create. In the simplified form, select the model framework used to train your model (i.e., _TensorFlow Serving_ or _Python_). Then, select the model you want to deploy from the list of available models under `pick a model`. @@ -195,7 +195,7 @@ Resources include the number of replicas for the deployment as well as the resou ## API protocol -Hopsworks supports both REST and gRPC as the API protocols to send inference requests to model deployments. To learn about REST and gRPC API protocols for model deployments, see the [API Protocol Guide](api-protocol.md). +Hopsworks supports both REST and gRPC as the API protocols to send inference requests to model deployments. In general, you use gRPC when you need lower latency inference requests. To learn more about the REST and gRPC API protocols for model deployments, see the [API Protocol Guide](api-protocol.md). ## Conclusion diff --git a/docs/user_guides/mlops/serving/inference-batcher.md b/docs/user_guides/mlops/serving/inference-batcher.md index d7ec29f38..243c491e4 100644 --- a/docs/user_guides/mlops/serving/inference-batcher.md +++ b/docs/user_guides/mlops/serving/inference-batcher.md @@ -17,11 +17,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Go to advanced options -A simplified creation form will appear including the most common deployment fields among all the configuration possible. Inference batching is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`. +A simplified creation form will appear including the most common deployment fields from all available configurations. Inference batching is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`.

diff --git a/docs/user_guides/mlops/serving/inference-logger.md b/docs/user_guides/mlops/serving/inference-logger.md index 8af482073..e767b21d8 100644 --- a/docs/user_guides/mlops/serving/inference-logger.md +++ b/docs/user_guides/mlops/serving/inference-logger.md @@ -21,11 +21,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Go to advanced options -A simplified creation form will appear including the most common deployment fields among all the configuration possible. Inference logging is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`. +A simplified creation form will appear including the most common deployment fields from all available configurations. Inference logging is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`.

diff --git a/docs/user_guides/mlops/serving/predictor.md b/docs/user_guides/mlops/serving/predictor.md index 21b94b42c..85c657630 100644 --- a/docs/user_guides/mlops/serving/predictor.md +++ b/docs/user_guides/mlops/serving/predictor.md @@ -32,11 +32,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Choose a model server -A simplified creation form will appear, including the most common deployment fields among all the configuration possible. These fields include the [model server](#model-server) and [custom script](#custom-script) (for python models). +A simplified creation form will appear, including the most common deployment fields from all available configurations. These fields include the [model server](#model-server) and [custom script](#custom-script) (for python models).

@@ -250,7 +250,7 @@ Resources include the number of replicas for the deployment as well as the resou ## API protocol -Hopsworks supports both REST and gRPC as the API protocols to send inference requests to model deployments. To learn about REST and gRPC API protocols for model deployments, see the [API Protocol Guide](api-protocol.md). +Hopsworks supports both REST and gRPC as the API protocols to send inference requests to model deployments. In general, you use gRPC when you need lower latency inference requests. To learn more about the REST and gRPC API protocols for model deployments, see the [API Protocol Guide](api-protocol.md). ## Conclusion diff --git a/docs/user_guides/mlops/serving/resources.md b/docs/user_guides/mlops/serving/resources.md index afa9f04a3..87286349c 100644 --- a/docs/user_guides/mlops/serving/resources.md +++ b/docs/user_guides/mlops/serving/resources.md @@ -17,11 +17,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Go to advanced options -A simplified creation form will appear including the most common deployment fields among all the configuration possible. Resource allocation is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`. +A simplified creation form will appear including the most common deployment fields from all available configurations. Resource allocation is part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`.

diff --git a/docs/user_guides/mlops/serving/transformer.md b/docs/user_guides/mlops/serving/transformer.md index 954d6c068..45ccd8a36 100644 --- a/docs/user_guides/mlops/serving/transformer.md +++ b/docs/user_guides/mlops/serving/transformer.md @@ -30,11 +30,11 @@ If you have at least one model already trained and saved in the Model Registry,

-Once in the deployments page, click on `New deployment` if there are not existing deployments or on `Create new deployment` at the top-right corner to open the deployment creation form. +Once in the deployments page, you can create a new deployment by either clicking on `New deployment` (if there are no existing deployments) or on `Create new deployment` it the top-right corner. Both options will open the deployment creation form. ### Step 2: Go to advanced options -A simplified creation form will appear including the most common deployment fields among all the configuration possible. Transformers are part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`. +A simplified creation form will appear including the most common deployment fields from all available configurations. Transformers are part of the advanced options of a deployment. To navigate to the advanced creation form, click on `Advanced options`.

diff --git a/mkdocs.yml b/mkdocs.yml index eb1a73887..7b29c039f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -47,7 +47,7 @@ nav: - Prediction Services: concepts/mlops/prediction_services.md - Model Training: concepts/mlops/training.md - Model Registry: concepts/mlops/registry.md - - Model Serving: concepts/mlops/kserve.md # create, read, version + - Model Serving: concepts/mlops/serving.md - Vector Database: concepts/mlops/opensearch.md - BI Tools: concepts/mlops/bi_tools.md - Development: