Skip to content

Commit

Permalink
Add attributes to MLmodel
Browse files Browse the repository at this point in the history
  • Loading branch information
@sguequierre
sguequierre committed Mar 11, 2024
1 parent 4ab9ba9 commit f77ecd9
Show file tree
Hide file tree
Showing 2 changed files with 63 additions and 38 deletions.
2 changes: 1 addition & 1 deletion docs/ml/vision/color_detector.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ Add the vision service object to the services array in your raw JSON configurati
{{% /tab %}}
{{< /tabs >}}

The following parameters are available for a `color_detector`.
The following parameters are available for a `color_detector`:

<!-- prettier-ignore -->
| Parameter | Inclusion | Description |
Expand Down
99 changes: 62 additions & 37 deletions docs/ml/vision/mlmodel.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ You can [add an existing model](/ml/upload-model/) or [train your own models](/m
<h4>Deploy your model</h4>

To make use of ML models with your machine, use the built-in [ML model service](/ml/) to deploy and run the model.

s
{{% /manualcard %}}
{{< /cards >}}

Expand Down Expand Up @@ -117,6 +117,18 @@ Add the vision service object to the services array in your raw JSON configurati

Click **Save config**.

The following attributes are available for an `mlmodel` detector or classifier:

<!-- prettier-ignore -->
| Parameter | Type | Inclusion | Description |
| --------- | ---- | --------- | ----------- |
| `mlmodel_name` | string | **Required** | The name of the [ML model service](/ml/deploy) you want to use the model from. |
| `remap_output_names` | object | Optional | The names of your output tensors, mapped to the service requirements. See [Tensor names](#tensor-names) for more information. |
| `remap_input_names` | object | Optional | The name of your input tensor, mapped to the service requirements. See [Tensor names](#tensor-names) for more information. |
| `input_image_bgr` | bool | Optional | Set this to `true` if the ML model service expect the input image to have BGR pixels, rather than RGB pixels. |
| `input_image_mean_value` | array | Optional | The standard deviation of the RGB (or BGR) values. Only required if the ML model service expects the input image to be normalized. |
| `input_image_std_dev` | array | Optional | The standard deviation of the RGB (or BGR) values. Only required if the ML model service expects the input image to be normalized. |

## Tensor names

Both the `mlmodel` detector and classifier require that the input and output tensors defined by your ML model are named according to the following:
Expand All @@ -128,55 +140,68 @@ Both the `mlmodel` detector and classifier require that the input and output ten
- The _input tensor_ must be named `image`
- The _output tensor_ must be named `probability`

If you [trained your ML model using the Viam app](/ml/train-model/), your `mlmodel` tensors is already named in this fashion, and you can proceed to [test your detector or classifier](#test-your-detector-or-classifier).
If you [trained your ML model using the Viam app](/ml/train-model/), your `mlmodel` tensors are already named in this fashion, and you can proceed to [test your detector or classifier](#test-your-detector-or-classifier).
However, if you [uploaded your own ML model](/ml/upload-model/), or are using one from the [Viam registry](https://app.viam.com/registry), you may need to remap your tensor names to meet this requirement, and should follow the instructions to [remap tensor names](#remap-tensor-names).

{{%expand "Click for details on how to remap tensor names." %}}

### Remap tensor names

If you need to remap the tensor names defined by your ML model to meet the tensor name requirements of the `mlmodel` detector or classifier, you can use the `remap_input_names` and `remap_output_names` attributes:

- To remap your model's tensor names to work with an `mlmodel` detector, add the following to your `mlmodel` vision service configuration, replacing the `my_model` input and output tensor names with the names from your model:
{{< tabs >}}
{{% tab name="Detector" %}}

```json {class="line-numbers linkable-line-numbers"}
{
"type": "vision",
"model": "mlmodel",
"attributes": {
"mlmodel_name": "my_model",
"remap_output_names": {
"my_model_output_tensor1": "category",
"my_model_output_tensor2": "location",
"my_model_output_tensor3": "score"
},
"remap_input_names": {
"my_model_input_tensor1": "image"
}
To remap your model's tensor names to work with an `mlmodel` detector, add the following to your `mlmodel` vision service configuration, replacing the `my_model` input and output tensor names with the names from your model:

```json {class="line-numbers linkable-line-numbers"}
{
"type": "vision",
"model": "mlmodel",
"attributes": {
"mlmodel_name": "my_model",
"remap_output_names": {
"my_model_output_tensor1": "category",
"my_model_output_tensor2": "location",
"my_model_output_tensor3": "score"
},
"name": "my-vision-service"
}
```
"remap_input_names": {
"my_model_input_tensor1": "image"
}
},
"name": "my-vision-service"
}
```

- To remap your model's tensor names to work with an `mlmodel` classifier, add the following to your `mlmodel` vision service configuration, replacing the `my_model` input and output tensor names with the names from your model:
{{% /tab %}}
{{% tab name="Classifier" %}}

```json {class="line-numbers linkable-line-numbers"}
{
"type": "vision",
"model": "mlmodel",
"attributes": {
"mlmodel_name": "my_model",
"remap_output_names": {
"my_model_output_tensor1": "probability"
},
"remap_input_names": {
"my_model_input_tensor1": "image"
}
To remap your model's tensor names to work with an `mlmodel` classifier, add the following to your `mlmodel` vision service configuration, replacing the `my_model` input and output tensor names with the names from your model:

```json {class="line-numbers linkable-line-numbers"}
{
"type": "vision",
"model": "mlmodel",
"attributes": {
"mlmodel_name": "my_model",
"remap_output_names": {
"my_model_output_tensor1": "probability"
},
"name": "my-vision-service"
}
```
"remap_input_names": {
"my_model_input_tensor1": "image"
}
},
"name": "my-vision-service"
}
```

{{% /tab %}}
{{< /tabs >}}

When done, click **Save config**, then proceed to [test your detector or classifier](#test-your-detector-or-classifier).

{{% /expand%}}

## Test your detector or classifier

You can test your detector or classifier with [existing images in the Viam app](#existing-images-in-the-cloud) or [live camera footage](#live-camera-footage).
Expand All @@ -186,7 +211,7 @@ You can also test classifiers with [existing images on a computer](#existing-ima

{{< alert title="Note" color="note" >}}

The feature is only available for classifiers that were uploaded after September 19, 2023.
The feature is only available for classifiers that are uploaded after September 19, 2023.

{{< /alert >}}

Expand Down

0 comments on commit f77ecd9

Please sign in to comment.