added snippets for Connection, Model Registry #160

rktraz · 2022-11-14T20:21:43Z

No description provided.

jimdowling

I think Javier should review this.

jimdowling · 2022-11-16T07:06:01Z

docs/index.md

+
+HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models.
+
+The library automatically configures itself based on the environment it is run.


This sentence doesn't mean anything.
configures itself how?
Technically, it should be
"The library automatically configures itself based on the environment it is run in."
Do you mean?
The library automatically configures itself based on the environment it is run in, either Python or Spark.
If you mean - it configures itself based on whether it is run inside Hopsworks or outside Hopsworks - in this case, just remove the sentence. Users will expect it should work inside/outside Hopsworks. We shouldn't need to tell them.

jimdowling · 2022-11-16T07:09:23Z

docs/index.md

+HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models.
+
+The library automatically configures itself based on the environment it is run.
+However, to connect from an external Python environment additional connection information, such as host and port, is required. For more information about the setup from external environments, see the setup section.


If you connect to Hopsworks from an external Python environment to your own managed Hopsworks cluster, you need to provide additional connection information - the Hopsworks hostname (or IP address) and port (if it is not port 443).

jimdowling · 2022-11-16T07:11:52Z

docs/index.md

+import hsml
+
+# Create a connection
+connection = hsml.connection()


put the hostname and project in the example here:

connection = hsml.connection(host="my.cluster.com", project="fraud")

jimdowling · 2022-11-16T07:14:13Z

docs/index.md

+                                   version=1,
+                                   metrics={"accuracy": 0.94},
+                                   description="mnist model description")
+model.save("/tmp/model_directory") # or /tmp/model_file


Add a line of code before this where you export the tensorflow model to the local directory:
"/tmp/model_directory"

Otherwise this code snippet doesn't actually upload a tensorflow model

docs/index.md

javierdlrm · 2022-12-21T08:50:04Z

python/hsml/connection.py

@@ -119,6 +119,12 @@ def get_model_registry(self, project: str = None):
        """Get a reference to a model registry to perform operations on, defaulting to the project's default model registry.
        Shared model registries can be retrieved by passing the `project` argument.

+        !!! example
+            ```python
+            mr = conn.get_model_registry()          # Get the project's default model registry


Suggested change

mr = conn.get_model_registry() # Get the project's default model registry

mr = conn.get_model_registry()

javierdlrm · 2022-12-21T08:55:09Z

python/hsml/model.py

+
+        !!! example
+            ```python
+            model.save('model.pkl')


Suggested change

model.save('model.pkl')

model.save('model.pkl') # save a single model file, or

model.save('path/to/model_directory') # save a model with multiple files

javierdlrm · 2022-12-21T08:59:14Z

python/hsml/model.py

+            # get model object
+            model = mr.get_model("citibike_mlp_model", version=1)
+
+            # delete this model version
+            model.delete()


Suggested change

# get model object

model = mr.get_model("citibike_mlp_model", version=1)

# delete this model version

model.delete()

# get model object of a specific version

model = mr.get_model("mnist", version=1)

# delete the model version

model.delete()

javierdlrm · 2022-12-21T09:01:34Z

python/hsml/model_registry.py

@@ -94,6 +108,11 @@ def get_models(self, name: str):
        Getting all models from the Model Registry for a given name returns a list of model entities, one for each version registered under
        the specified model name.

+        !!! example
+            ```python
+            models = mr.get_models("churnmodel")


Suggested change

models = mr.get_models("churnmodel")

models = mr.get_models("mnist")

javierdlrm · 2022-12-21T09:01:47Z

python/hsml/model_registry.py

+            SORT_METRICS_BY = "max" # your sorting criteria
+
+            # get best model based on custom metrics
+            best_model = mr.get_best_model("citibike_mlp_model",


Suggested change

best_model = mr.get_best_model("citibike_mlp_model",

best_model = mr.get_best_model("mnist",

javierdlrm · 2022-12-21T09:27:12Z

python/hsml/tensorflow/signature.py

+        model = mr.tensorflow.create_model(
+                    name="bitcoin_price_model",
+                    metrics=metrics,
+                    description="bitcoin daily price detection model.",
+                    input_example=[1613512800000]
+                    )


Suggested change

model = mr.tensorflow.create_model(

name="bitcoin_price_model",

metrics=metrics,

description="bitcoin daily price detection model.",

input_example=[1613512800000]

)

model = mr.tensorflow.create_model(

name="mnist_classifier",

metrics=metrics,

model_schema=model_schema,

input_example=input_example,

description="MNIST Classifier"

)

javierdlrm · 2022-12-21T09:28:50Z

python/hsml/torch/signature.py

+        model = mr.torch.create_model(
+                    name="electricity_price_prediction_model",
+                    metrics=metrics,
+                    description="Daily electricity price prediction model.",
+                    input_example=n_step_window.example[0].numpy()
+                    )


Suggested change

model = mr.torch.create_model(

name="electricity_price_prediction_model",

metrics=metrics,

description="Daily electricity price prediction model.",

input_example=n_step_window.example[0].numpy()

)

model = mr.torch.create_model(

name="mnist_classifier",

metrics=metrics,

model_schema=model_schema,

input_example=input_example,

description="MNIST Classifier"

)

javierdlrm · 2022-12-21T10:10:54Z

docs/index.md

+
+HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models.
+
+However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section.


I think we can keep the first sentence as

Suggested change

However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section.

The library automatically configures itself based on the environment it is run. However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section.

javierdlrm · 2022-12-21T10:16:33Z

docs/index.md

+import hsml
+
+# Create a connection
+connection = hsml.connection(host="my.cluster.com", project="fraud")


Suggested change

connection = hsml.connection(host="my.cluster.com", project="fraud")

connection = hsml.connection()

# or connection = hsml.connection(host="my.hopsworks.cluster", project="my_project")

javierdlrm · 2022-12-21T10:20:48Z

docs/index.md

+import tensorflow as tf
+
+
+model = mr.tensorflow.create_model(name="mnist",
+                                   version=1,
+                                   metrics={"accuracy": 0.94},
+                                   description="mnist model description")
+
+export_path = "/tmp/model_directory" # or "/tmp/model_file"
+tf.saved_model.save(model, export_path)                                    
+model.save(export_path)


Suggested change

import tensorflow as tf

model = mr.tensorflow.create_model(name="mnist",

version=1,

metrics={"accuracy": 0.94},

description="mnist model description")

export_path = "/tmp/model_directory" # or "/tmp/model_file"

tf.saved_model.save(model, export_path)

model.save(export_path)

import tensorflow as tf

export_path = "/tmp/model_directory"

tf.saved_model.save(model, export_path)

model = mr.tensorflow.create_model(name="mnist",

version=1,

metrics={"accuracy": 0.94},

description="mnist model description")

model.save(export_path) # "/tmp/model_directory" or "/tmp/model_file"

added snippets for Connection, Model Registry

5a69ccf

davitbzh requested review from javierdlrm, davitbzh and jimdowling November 15, 2022 10:30

jimdowling requested changes Nov 16, 2022

View reviewed changes

made fixes, suggested by Jim

d4da535

rktraz requested review from jimdowling and removed request for javierdlrm and davitbzh November 16, 2022 17:05

jimdowling requested a review from javierdlrm December 9, 2022 11:35

javierdlrm requested changes Dec 21, 2022

View reviewed changes

resolved comments for PR

60f3aab

rktraz requested review from javierdlrm and removed request for jimdowling December 21, 2022 18:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

added snippets for Connection, Model Registry #160

added snippets for Connection, Model Registry #160

rktraz commented Nov 14, 2022

jimdowling left a comment

jimdowling Nov 16, 2022

jimdowling Nov 16, 2022

jimdowling Nov 16, 2022

jimdowling Nov 16, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022

javierdlrm Dec 21, 2022


		HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models.

		The library automatically configures itself based on the environment it is run.

	mr = conn.get_model_registry() # Get the project's default model registry
	mr = conn.get_model_registry()

	model.save('model.pkl')
	model.save('model.pkl') # save a single model file, or
	model.save('path/to/model_directory') # save a model with multiple files

	models = mr.get_models("churnmodel")
	models = mr.get_models("mnist")

	best_model = mr.get_best_model("citibike_mlp_model",
	best_model = mr.get_best_model("mnist",


		HSML is the library to interact with the Hopsworks Model Registry and Model Serving. The library makes it easy to export, manage and deploy models.

		However, to connect from an external Python environment additional connection information, such as the Hopsworks hostname (or IP address) and port (if it is not port 443), is required. For more information about the setup from external environments, see the setup section.

	connection = hsml.connection(host="my.cluster.com", project="fraud")
	connection = hsml.connection()
	# or connection = hsml.connection(host="my.hopsworks.cluster", project="my_project")

added snippets for Connection, Model Registry #160

Are you sure you want to change the base?

added snippets for Connection, Model Registry #160

Conversation

rktraz commented Nov 14, 2022

jimdowling left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment