DOCS-982: Document `obstacles_depth` model of segmenter in vision service #1721

sguequierre · 2023-08-30T15:24:44Z

Make a new section in segmenters docs for obstacles_depth

sguequierre · 2023-08-30T15:35:58Z

docs/services/vision/segmentation.md

+| `h_min_m` | Optional | The minumum vertical height in meters acceptable so that any obstacle strictly lower than is not deemed an obstacle by the segmenter. <br> Default: `0.0` </br> |
+| `h_max_m` | Optional | The maximum vertical height in meters acceptable so that any obstacle strictly higher than is not deemed an obstacle by the segmenter. <br> Default: `1.0` </br> |
+| `theta_max_deg` | Optional | The largest slope in degrees acceptable for non-obstacle surface ramps. <br> Default: `45` </br> |
+| `with_geometries` | Optional | If you have configured the geometries, or, spatial orientation of the components of your robot, within the frame system. `true` is necessary to `return_pcds` properly, if `"return_pcds": "true"`  <br> Example: `false` </br> |


@kharijarrett Any suggestions here on the tables are very welcome, I'm not sure if I'm phrasing properly esp. with_geometries

docs/services/vision/segmentation.md

npentrel · 2023-08-31T20:19:34Z

docs/services/vision/segmentation.md

+| `return_pcds` | Required | If you would like point clouds or a single point returned within the `GeometryInFrame` object captured by this segmenter. `true` indicates wanting point clouds. <br> Example: `"false"` </br> |
+| `h_min_m` | Optional | The minimum vertical height in meters acceptable so that any obstacle strictly lower than is not deemed an obstacle by the segmenter. <br> Default: `0.0` </br> |
+| `h_max_m` | Optional | The maximum vertical height in meters acceptable so that any obstacle strictly higher than is not deemed an obstacle by the segmenter. <br> Default: `1.0` </br> |
+| `theta_max_deg` | Optional | The largest slope in degrees acceptable for non-obstacle surface ramps. <br> Default: `45` </br> |


giving it a crack below

Ok, edited w khari suggestion!

docs/services/vision/segmentation.md

kharijarrett

Nice! This looks mostly good I appreciate the added clarity about the frame system and intrinsics. The main thing left is a lil confusion about with_geometries vs. return_pcds. I hope what I said makes sense.

kharijarrett · 2023-09-01T14:27:11Z

docs/services/vision/segmentation.md

@@ -19,6 +19,7 @@ The types of segmenters supported are:

 - [**Obstacles point cloud (`obstacles_pointcloud`)**](#configure-an-obstacles_pointcloud-segmenter): A segmenter that identifies well-separated objects above a flat plane.
 - [**Object detector (`detector_3d_segmenter`)**](#configure-a-detector_3d_segmenter): This model takes 2D bounding boxes from an object detector and projects the pixels in the bounding box to points in 3D space.
+- [**Obstacles depth (`obstacles_depth`)**](#configure-an-obstacles_depth-segmenter): A segmenter for depth cameras that returns a projection of a bounding box or boxes with a Pose as a vector from the depth camera lens to the center of the obstacle identified.


Can we change to "... that returns the perceived obstacles as a set of 3-dimensional bounding boxes, each with a Pose as a vector.... "

Done, perfect thank you!

Wait I'm sorry I messed up. I'm actually wrong because in some cases (no intrinsics) this isn't true. I think the most accurate thing is "A segmenter for depth cameras that returns the perceived obstacles as a set of one or more GeometriesInFrame, each with a Pose as a vector from the depth camera lens to the center of the obstacle identified."

This comment @sguequierre

kharijarrett · 2023-09-01T14:33:09Z

docs/services/vision/segmentation.md

+
+To configure an `obstacles_depth` segmenter, first decide if you want to use the segmenter with or without intrinsic parameters.
+
+This is determined by the value of the configuration attribute `return_pcds`:


Not true.. this is actually determined by with_geometries.

return_pcds is a separate optional parameter (default false) that determines whether pointclouds should be included within the GeometryInFrame object.

So if w_g and r_p were both true, you would get 10 GeometryInFrame objects, each with a geometry and a pointcloud inside. If w_g was false but r_p was true, you'd get a single GeometryInFrame object that would have a 1-point-pointcloud and a geometry (point). When r_p is false, you get the same things just without the pointcloud. Geometry only.

Ok tried to update!! Sorry for the confusion

docs/services/vision/segmentation.md

kharijarrett · 2023-09-01T14:36:47Z

docs/services/vision/segmentation.md

+
+| Parameter | Inclusion | Description |
+| --------- | --------- | ----------- |
+| `return_pcds` | Required | If you would like point clouds or a single point returned within the `GeometryInFrame` object captured by this segmenter. `true` indicates wanting point clouds. <br> Example: `"false"` </br> |


Again this is with_geometries. (See explanation above)

Co-authored-by: Naomi Pentrel <[email protected]>

docs/services/vision/segmentation.md

…metries` Co-authored-by: Naomi Pentrel <[email protected]>

sguequierre · 2023-09-01T15:47:41Z

docs/services/vision/segmentation.md

+| `h_min_m` | Optional | The minimum vertical height in meters for an object to be considered an obstacle. <br> Default: `0.0` </br> |
+| `h_max_m` | Optional | The maximum vertical height in meters at which an object is considered an obstacle. <br> Default: `1.0` </br> |
+| `theta_max_deg` | Optional | The largest slope in degrees acceptable for non-obstacle surface ramps. <br> Default: `45` </br> |
+| `return_pcds` | Optional | If you have configured the geometries or spatial orientation of the components of your robot, within the frame system.  <br> Example: `"false"` </br> |


@kharijarrett I flipped with_geometries and return_pcds re. your feedback-- is this accurate now or still flipped up?

"Whether you would like pointclouds to be included within the GeometryInFrame object captured by this segmenter."

Thank you, done/added!

docs/services/vision/segmentation.md

kharijarrett · 2023-09-01T16:18:41Z

docs/services/vision/segmentation.md

+
+| Parameter | Inclusion | Description |
+| --------- | --------- | ----------- |
+| `with_geometries` | Required | Whether you would like point clouds or a single point returned within the `GeometryInFrame` object captured by this segmenter. If `true`, return point clouds. <br> Example: `"false"` </br> |


"Whether you would like multiple boxes (true) or a single point (false) returned within the GeometryInFrame object captured by this segmenter."

TY, sorry for the confusion. Added!

sguequierre · 2023-09-11T15:56:51Z

docs/services/vision/segmentation.md

+This segmenter model is for depth cameras, and is best for motion planning with transient obstacles.
+
+Use the segmenter to identify well separated objects above a flat plane.
+Configure the `with_geometries` attribute according to whether you've [configured the frame system](/services/frame-system/#configuration) to provide intrinsic parameters:


@npentrel Edited this re. your review/suggestion to combine diff sections

I still don't get this - you're saying here "wheter you've configured the frame system" and on 232 "start by configuring your frame system". Either you have already configured it or you start configuring it in 232. Or do you just configure part of the frame system subsequently. Also how do I choose whether I want to use a frame system. I think this should say something like "If you need to receive a point cloud, ..." "If you only need a single point, then you do not need ..."

sguequierre · 2023-09-12T14:53:01Z

docs/services/vision/segmentation.md

+This segmenter model is for depth cameras, and is best for motion planning with transient obstacles.
+
+Use the segmenter to identify well separated objects above a flat plane.
+Configure the `with_geometries` attribute according to whether you've [configured the frame system](/services/frame-system/#configuration) to provide intrinsic parameters:


Suggested change

Configure the `with_geometries` attribute according to whether you've [configured the frame system](/services/frame-system/#configuration) to provide intrinsic parameters:

First, choose whether you want to configure the `with_geometries` attribute to provide intrinsic parameters from [configuring the frame system](/services/frame-system/#configuration).

The options are as follows:

@npentrel I think I see why you're confused-- removed so it's not directing to configure-- what i mean to say is that you have to choose whether you want to or not as the beginning of the configuration process, bc this attribute should be determined based on what you have done in the past -- configured the frame system or not. Does that make any sense?

I would just remove this full section, but I think it's important to have the options listed so people can see immediately that their choice to configure frame system or not/selection for this attribute will then have this effect on what your segmenter returns

npentrel · 2023-09-14T18:56:49Z

docs/services/vision/segmentation.md

+This segmenter model is for depth cameras, and is best for motion planning with transient obstacles.
+
+Use the segmenter to identify well separated objects above a flat plane.
+Configure the `with_geometries` attribute according to whether you've [configured the frame system](/services/frame-system/#configuration) to provide intrinsic parameters:
+
+- `true`: this segmenter will return point clouds within the `GeometryInFrame` object it captures.
+- `false`: this segmenter will return a single point within the `GeometryInFrame` object it captures.
+
+If `true`, start by [configuring your frame system](/services/frame-system/#configuration) to configure the relative spatial orientation of the components of your robot within Viam's [frame system service](/services/frame-system/).


I think what this needs to do is explain better what the two options are. You're not choosing whether to configure a parameter to true or false, the reader should choose what functionality they need and based on that set the parameter. Now I am not entirely certain I am capturing the functionality correctly but the way I'd structure it is like this:

Suggested change

This segmenter model is for depth cameras, and is best for motion planning with transient obstacles.

Use the segmenter to identify well separated objects above a flat plane.

Configure the `with_geometries` attribute according to whether you've [configured the frame system](/services/frame-system/#configuration) to provide intrinsic parameters:

- `true`: this segmenter will return point clouds within the `GeometryInFrame` object it captures.

- `false`: this segmenter will return a single point within the `GeometryInFrame` object it captures.

If `true`, start by [configuring your frame system](/services/frame-system/#configuration) to configure the relative spatial orientation of the components of your robot within Viam's [frame system service](/services/frame-system/).

This segmenter model is for depth cameras, and is best for motion planning with transient obstacles.

You can use the segmenter to identify well seprated objects above a flat plane or to identify a single object (?):

- To identify multiple well separated objects above a flat plane, set `with_geometries: true`.

Ensure that you have configured your camera's [`intrinsic_parameters`](https://docs.viam.com/components/camera/calibrate/).

Then, [configure your frame system](/services/frame-system/#configuration) to configure the relative spatial orientation of the components of your robot within Viam's [frame system service](/services/frame-system/).

After configuring your frame system, your camera populates its own `Properties` with the intrinsic parameters from the frame system (camera? the next line seems to indicate that?).

You can then get those parameters from your camera through the [camera API](/components/camera/#getproperties).

The segmenter now returns point clouds within the `GeometryInFrame` object it captures.

- To identify a single object above a flat plane, set `with_geometries: false`.

The segmenter now returns a single point within the `GeometryInFrame` object it captures.

I think this way it provides one choice point and only one mention of the frame system. Though I am not sure if the second one also needed the frame system - I guess that is also currently confusing: do I need to configure the frame system in both scenarios or just one?

You don't need to/shouldn't have configured the frame system if false, but if you do and still set to false, that's fine, you just will get a point back anyways! Is that clear enough now? The parameter determines the behavior of the segmenter-- if you set to true and didn't configure anything I assume it will error out

viambot · 2023-09-14T19:10:07Z

Overall readability score: 54.48 (🔴 -0.01)

File	Readability
segmentation.md	61.57 (🔴 -4.45)

View detailed metrics

🟢 - Shows an increase in readability
🔴 - Shows a decrease in readability

File	Readability	FRE	GF	ARI	CLI	DCRS
segmentation.md	61.57	50.77	10.42	13.4	11.02	6.42
	🔴 -4.45	🔴 -8.97	🔴 -0.29	🔴 -1	🔴 -0.81	🟢 +0.04

Averages:

	Readability	FRE	GF	ARI	CLI	DCRS
Average	54.48	45.88	10.88	13.36	11.88	7.77
	🔴 -0.01	🔴 -0.03	🟢 +0	🟢 +0	🟢 +0	🟢 +0

View metric targets

Metric	Range	Ideal score
Flesch Reading Ease	100 (very easy read) to 0 (extremely difficult read)	60
Gunning Fog	6 (very easy read) to 17 (extremely difficult read)	8 or less
Auto. Read. Index	6 (very easy read) to 14 (extremely difficult read)	8 or less
Coleman Liau Index	6 (very easy read) to 17 (extremely difficult read)	8 or less
Dale-Chall Readability	4.9 (very easy read) to 9.9 (extremely difficult read)	6.9 or less

sguequierre · 2023-09-14T19:10:08Z

docs/services/vision/segmentation.md

+You can then get those parameters from your camera through the [camera API](/components/camera/#getproperties).
+The segmenter now returns point clouds within the `GeometryInFrame` object it captures.
+
+If you choose not to configure the frame system to provide instrinsic parameters, set `with_geometries: false`.


@npentrel Added your suggestions here and above, I think! Let me know if it still doesn't make sense and we can maybe sync up next week to get this right?

I think I'm still not clear what I am identifying when with_geometries: false. On lines 208/209, we say the segmenter identifies well separated objects above a flat plane, and if we want then we should set with_geometries: true. But am I still identifying well separated objects when I have that set to false? Is the difference just that I don't get the coordinates of where those are?

As a reader I still don't fully understand why I'd choose one over the other. An example may help here.

docs/services/vision/segmentation.md

npentrel

Let's check whether we can get example output for the with and without geometries options - that would probably help the reader.

This LGTM to me now, but please have another person read over this too to ensure it's understandable

viambot · 2023-09-22T16:52:00Z

You can view a rendered version of the docs from this PR at https://docs-test.viam.dev/389216a95af84d8a808c6a3997f6853bf35fdc31/public

sguequierre added 3 commits August 29, 2023 10:55

make initial page

6eb7f69

inital commit add obstacles_depth segmenter

b09d241

build out page

99d243f

viambot added the safe to build This pull request is marked safe to build from a trusted zone label Aug 30, 2023

sguequierre added 2 commits August 30, 2023 11:32

fixup add better info for attributes

69f6e2f

Update segmentation.md

566328d

sguequierre commented Aug 30, 2023

View reviewed changes

sguequierre requested a review from kharijarrett August 30, 2023 15:36

sguequierre commented Aug 30, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre added 2 commits August 30, 2023 16:26

Update docs/services/vision/segmentation.md

3eafa6e

Update segmentation.md make example bools true

e3f9bd9

sguequierre commented Aug 30, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre commented Aug 30, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre commented Aug 30, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

Apply suggestions from code review

5149335

npentrel reviewed Aug 31, 2023

View reviewed changes

kharijarrett requested changes Sep 1, 2023

View reviewed changes

Apply naomi's inline suggestions from code review part 1

65324fd

Co-authored-by: Naomi Pentrel <[email protected]>

sguequierre commented Sep 1, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre commented Sep 1, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre commented Sep 1, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre commented Sep 1, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

Apply suggestions from code review switch return_pcds and `with_geo…

133bbcb

…metries` Co-authored-by: Naomi Pentrel <[email protected]>

sguequierre commented Sep 1, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

Apply khari's suggestion for line 22

52e93ee

kharijarrett reviewed Sep 1, 2023

View reviewed changes

Apply khari suggestion for theta parameter from code review

edccf5c

sguequierre requested review from kharijarrett and npentrel September 5, 2023 14:33

sguequierre requested a review from npentrel September 8, 2023 16:55

sguequierre added 2 commits September 11, 2023 11:55

Update segmentation.md

77e5772

Apply suggestions from code review

7a085d0

sguequierre commented Sep 11, 2023

View reviewed changes

Merge branch 'main' into DOCS-982/document-obstacles_depth-model

0242478

sguequierre commented Sep 12, 2023

View reviewed changes

npentrel reviewed Sep 14, 2023

View reviewed changes

Apply suggestions from code review

7cdd0f0

sguequierre commented Sep 14, 2023

View reviewed changes

sguequierre added 2 commits September 14, 2023 15:43

Merge branch 'main' into DOCS-982/document-obstacles_depth-model

8ba172a

Merge branch 'main' into DOCS-982/document-obstacles_depth-model

3a88872

sguequierre requested a review from npentrel September 17, 2023 13:13

Update segmentation.md

5595d2b

sguequierre commented Sep 21, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

sguequierre commented Sep 21, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

Apply suggestions from code review

12f5f80

sguequierre commented Sep 21, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

Apply suggestions from code review

867b301

sguequierre commented Sep 21, 2023

View reviewed changes

docs/services/vision/segmentation.md Outdated Show resolved Hide resolved

npentrel reviewed Sep 21, 2023

View reviewed changes

sguequierre added 4 commits September 22, 2023 12:27

Update segmentation.md clean up wording

2bb85f4

Update segmentation.md

99f1dd7

Update segmentation.md

40807f7

Update segmentation.md

389216a

sguequierre requested a review from npentrel September 22, 2023 16:40

sguequierre merged commit c9dc76c into viamrobotics:main Sep 22, 2023

sguequierre deleted the DOCS-982/document-obstacles_depth-model branch September 22, 2023 19:38

sguequierre restored the DOCS-982/document-obstacles_depth-model branch September 22, 2023 19:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DOCS-982: Document `obstacles_depth` model of segmenter in vision service #1721

DOCS-982: Document `obstacles_depth` model of segmenter in vision service #1721

sguequierre commented Aug 30, 2023

sguequierre Aug 30, 2023

npentrel Aug 31, 2023

sguequierre Sep 1, 2023

sguequierre Sep 5, 2023

kharijarrett left a comment

kharijarrett Sep 1, 2023 •

edited

Loading

sguequierre Sep 1, 2023

kharijarrett Sep 1, 2023

kharijarrett Sep 5, 2023

kharijarrett Sep 1, 2023

sguequierre Sep 1, 2023

kharijarrett Sep 1, 2023

sguequierre Sep 1, 2023

kharijarrett Sep 1, 2023

sguequierre Sep 5, 2023

kharijarrett Sep 1, 2023

sguequierre Sep 5, 2023

sguequierre Sep 11, 2023

npentrel Sep 11, 2023

sguequierre Sep 12, 2023 •

edited

Loading

npentrel Sep 14, 2023

sguequierre Sep 14, 2023

viambot commented Sep 14, 2023 •

edited

Loading

sguequierre Sep 14, 2023

npentrel Sep 21, 2023

npentrel left a comment

viambot commented Sep 22, 2023

	\| `theta_max_deg` \| Optional \| The largest slope in degrees acceptable for non-obstacle surface ramps. <br> Default: `45` </br> \|
	\| `theta_max_deg` \| Optional \| The largest slope in degrees acceptable for non-obstacle surface ramps. <br> Default: `45` </br> \|


		To configure an `obstacles_depth` segmenter, first decide if you want to use the segmenter with or without intrinsic parameters.

		This is determined by the value of the configuration attribute `return_pcds`:

-Configure the `with_geometries` attribute according to whether you've [configured the frame system](/services/frame-system/#configuration) to provide intrinsic parameters:
+First, choose whether you want to configure the `with_geometries` attribute to provide intrinsic parameters from [configuring the frame system](/services/frame-system/#configuration).
+The options are as follows:

DOCS-982: Document obstacles_depth model of segmenter in vision service #1721

DOCS-982: Document obstacles_depth model of segmenter in vision service #1721

Conversation

sguequierre commented Aug 30, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kharijarrett left a comment

Choose a reason for hiding this comment

kharijarrett Sep 1, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sguequierre Sep 12, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

viambot commented Sep 14, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

npentrel left a comment

Choose a reason for hiding this comment

viambot commented Sep 22, 2023

DOCS-982: Document `obstacles_depth` model of segmenter in vision service #1721

DOCS-982: Document `obstacles_depth` model of segmenter in vision service #1721

kharijarrett Sep 1, 2023 •

edited

Loading

sguequierre Sep 12, 2023 •

edited

Loading

viambot commented Sep 14, 2023 •

edited

Loading