Draft Fleet content (#3727)

docs/dev/reference/machine-to-machine-comms.md

@@ -0,0 +1,118 @@
+---
+title: "Machine-to-Machine Communication: End-to-End Flow"
+linkTitle: "Machine-to-Machine Communication"
+weight: 60
+type: "docs"
+description: "Explanation of how a machine and its parts interact at the communication layer."
+aliases:
+  - "/internals/robot-to-robot-comms/"
+  - "/internals/machine-to-machine-comms/"
+  - "/architecture/machine-to-machine-comms/"
+toc_hide: true
+---
+
+When building a smart machine application in the [Viam app](https://app.viam.com), a user typically begins by configuring their machine which can consist of one or more {{< glossary_tooltip term_id="part" text="parts" >}}.
+Next they will test that it is wired up properly using the Viam app's Control page.
+Once they've ensured everything is wired up properly, they will build their main application and the business logic for their machine using one of Viam's language SDKs.
+This SDK-based application is typically run on either the main part of the machine or a separate computer dedicated to running the business logic for the machine.
+
+Below, we describe the flow of information through a Viam-based multipart machine and then get into the specifics of what backs these connections and communications APIs.
+
+## High-level inter-robot/SDK communication
+
+To begin, let's define our machine's topology:
+
+![robot communication diagram](/internals/robot-to-robot-comms/robot-communication-diagram.png)
+
+This machine is made of two parts and a separate SDK-based application, which we'll assume is on a third machine, though it could just as easily run on the main part without any changes.
+
+- The first and main part, RDK Part 1, consists of a Raspberry Pi and a single USB connected camera called Camera.
+
+- The second and final part, RDK Part 2, consists of a Raspberry Pi connected to a robotic arm over ethernet and a gantry over GPIO.
+
+RDK Part 1 will establish a bidirectional gRPC/{{< glossary_tooltip term_id="webrtc" >}} connection to RDK Part 2.
+RDK Part 1 is considered the controlling peer (client).
+RDK Part 2 is consider the controlled peer (server).
+
+Let's suppose our SDK application uses the camera to track the largest object in the scene and instructs the arm to move to that same object.
+
+Since RDK Part 1 is the main part and has access to all other parts, the application will connect to it using the SDK.
+Once connected, it will take the following series of actions:
+
+<OL>
+<li>Get segmented point clouds from the camera and the object segmentation service.</li>
+
+<li>Find the largest object by volume.</li>
+
+<li>Take the object's center pose and tell the motion service to move the arm to that point.</li>
+
+<li>Go back to 1.</li>
+</OL>
+Let's breakdown how these steps are executed.
+
+<ol>
+<li>Get segmented point clouds from the camera and the object segmentation service:</li>
+
+![robot communication diagram](/internals/robot-to-robot-comms/getobjectpointcloud-flow.png)
+
+<OL type="a">
+<li>The SDK will send a GetObjectPointClouds request with Camera being referenced in the message to RDK Part 1's Object Segmentation Service.</li>
+
+<li>RDK Part 1 will look up the camera referenced, call the GetPointCloud method on it.</li>
+
+<li>The Camera will return the PointCloud data to RDK Part</li>
+
+<li>RDK Part 1 will use a point cloud segmentation algorithm to segment geometries in the PointCloud.</li>
+{{% alert title="Important" color="note" %}}
+The points returned are respective to the reference frame of the camera.
+This will become important in a moment.
+{{% /alert %}}
+<li>The set of segmented point clouds and their bounding geometries are sent back to the SDK-based application.</li>
+</ol>
+
+<li>Find the largest object by volume:</li>
+<ol type="a">
+<li>The application will iterate over the geometries of the segmented point clouds returned to it and find the object with the greatest volume and record its center pose.</li>
+</ol>
+
+<li>Take the object's center pose and tell the motion service to move the arm to that point:</li>
+
+![motion service move flow](/internals/robot-to-robot-comms/motion-service-move-flow.png)
+
+<ol type="a">
+<li>The SDK application will send a Move request for the arm to the motion service on RDK Part 1 with the destination set to the center point determined by the application.</li>
+
+<li>RDK Part 1's motion service will break down the Move request and perform the necessary frame transforms before sending the requests along to the relevant components.
+This is where the frame system comes into play.
+Our center pose came from the camera but we want to move the arm to that position even though the arm lives in its own frame.
+The frame system logic in the RDK automatically handles the transformation logic between these two reference frames while also handling understanding how the frame systems are connected across the two parts.</li>
+
+<li>Having computed the pose in the reference frame of the arm, the motion service takes this pose, and sends a plan on how to move the arm in addition to the gantry to achieve this new goal pose to RDK Part 2.
+The plan consists of a series of movements that combine inverse kinematics, mathematics, and constraints based motion planning to get the arm and gantry to their goal positions.</li>
+
+<li>In executing the plan, which is being coordinated on RDK Part 1, Part 1 will send messages to the Arm and Gantry on RDK Part 2.
+RDK Part 2 will be unaware of the actual plan and instead will only receive distinct messages to move the components individually.</li>
+
+<li>The arm and gantry connected to RDK Part 2 return an acknowledgement of the part Move requests to RDK Part 2.</li>
+
+<li>RDK Part 2 returns an acknowledgement of the Motion Move request to RDK Part 1.</li>
+
+<li>RDK Part 1 returns an acknowledgement of the Motion Move request to the SDK application.</li>
+</ol>
+
+## Low-level inter-robot/SDK communication
+
+All component and service types in the RDK, and the Viam API for that matter, are represented as [Protocol Buffers (protobuf)](https://developers.google.com/protocol-buffers) services.
+protobuf is a battle tested Interface Description Language (IDL) that allows for specifying services, their methods, and the messages that comprise those methods.
+Code that uses protobuf is autogenerated and compiles messages into a binary form.
+
+[gRPC](https://grpc.io/) is responsible for the transport and communication of protobuf messages when calling protobuf methods.
+It generally works over a TCP, TLS backed HTTP2 connection operating over framing see [gRPC's HTTP2 documentation](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) for more.
+
+The RDK uses protobuf and gRPC to enable access and control to its components and services.
+That means if there are two arms in a machine configuration, there is only one Arm service that handles the Remote Procedure Calls (RPC) for all arms configured.
+
+In addition to gRPC, the RDK uses [WebRTC](https://webrtcforthecurious.com/) video and audio streams and data channels to enable peer to peer (P2P) communication between machine parts as well as SDKs and the Remote Control interface.
+
+An outline of how WebRTC is used lives on [Go.dev](https://pkg.go.dev/go.viam.com/[email protected]/rpc#hdr-Connection), but in short, an RDK is always waiting on the Viam app ([app.viam.com](https://app.viam.com)) to inform it of a connection requesting to be made to it whereby it sends details about itself and how to connect on a per connection basis.
+Once a connection is made, the Viam app is no longer involved in any packet transport and leaves it up to the two peers to communicate with each other.

docs/dev/tools/troubleshooting.md → docs/dev/tools/common-errors.md

@@ -1,6 +1,6 @@
 ---
-title: "Troubleshooting"
-linkTitle: "Troubleshooting"
+title: "Common Errors & Known Issues"
+linkTitle: "Common Errors"
 weight: 50
 type: "docs"
 description: "A guide to troubleshooting a Viam-based machine or system of machines with fixes to common problems."
@@ -11,45 +11,15 @@ date: "2022-01-01"
 This document lists common errors encountered when working with `viam-server` and the [Viam app](https://app.viam.com), and provides simple steps to resolve them.
 While many common issues and their possible resolutions are presented here, this list is not comprehensive.
 
+To view logs or get a remote shell on a machine see [Troubleshoot](/manage/troubleshoot/troubleshoot/).
+
 If you have encountered an error that is not listed here, we'd love to hear from you on our [Community Discord](https://discord.gg/viam)!
 Please post the error message you received along with how you were able to trigger it and we'll see if we can help.
 
 ## Status
 
 For information on the status of [app.viam.com](https://app.viam.com), visit [status.viam.com](https://status.viam.com/).
 
-## Enable debug level logs
-
-The default log level for `viam-server` and any running resources is `"Info"`.
-If you are not seeing helpful logs, you can try changing the log level to `"Debug"`.
-
-{{< tabs >}}
-{{% tab name="For individual resources" %}}
-
-Add the `log_configuration` option to the resource's JSON configuration:
-
-```json
-"log_configuration": {
-    "level": "Debug"
-},
-"attributes": { ... }
-```
-
-{{% /tab %}}
-{{% tab name="For viam-server" %}}
-
-Add `"debug": true` to the machine's configuration:
-
-```json
-{
-  "debug": true,
-  "components": [{ ... }]
-}
-```
-
-{{% /tab %}}
-{{< /tabs >}}
-
 ## Common installation errors
 
 ### The authenticity of host 'hostname.local' can't be established

docs/manage/_index.md

@@ -8,3 +8,26 @@ no_list: true
 open_on_desktop: true
 overview: true
 ---
+
+{{< how-to-expand "Deploy software to machines" "4" "BEGINNER-FRIENDLY" >}}
+{{< cards >}}
+{{% card link="/how-tos/configure/" noimage="true" %}}
+{{% card link="/how-tos/one-to-many/" noimage="true" %}}
+{{% card link="/how-tos/provision-setup/" noimage="true" %}}
+{{% card link="/how-tos/provision/" noimage="true" %}}
+{{< /cards >}}
+{{< /how-to-expand >}}
+
+{{< how-to-expand "Manage a large fleet of machines" "2" "INTERMEDIATE" >}}
+{{< cards >}}
+{{% card link="/how-tos/manage-fleet/" noimage="true" %}}
+{{% card link="/how-tos/deploy-packages/" noimage="true" %}}
+{{< /cards >}}
+{{< /how-to-expand >}}
+
+{{< how-to-expand "Troubleshooting" "2" "INTERMEDIATE" >}}
+{{< cards >}}
+{{% card link="/how-tos/manage-fleet/" noimage="true" %}}
+{{% card link="/how-tos/deploy-packages/" noimage="true" %}}
+{{< /cards >}}
+{{< /how-to-expand >}}

docs/manage/deploy/one-to-many.md

@@ -0,0 +1,154 @@
+---
+title: "Configure multiple similar machines"
+linkTitle: "Configure many machines"
+weight: 40
+type: "docs"
+tags: ["data management", "data", "services"]
+images: ["/how-tos/one-to-many/new-fragment.png"]
+description: "Viam has a built-in tool called fragments for using the same configuration on multiple machines."
+aliases:
+  - /use-cases/one-to-many/
+languages: []
+viamresources: []
+platformarea: ["fleet"]
+level: "Beginner"
+date: "2024-08-09"
+# updated: ""  # When the tutorial was last entirely checked
+cost: "0"
+---
+
+Viam has a built-in tool called _{{< glossary_tooltip term_id="fragment" text="fragments" >}}_ for using the same configuration on multiple machines.
+A fragment can configure just one resource a machine uses, or all the resources it uses.
+If most of your machines are similar but not identical, you can use fragments and then manually add modifications for individual machines.
+
+When you update a fragment, it updates the configurations of all machines that use that fragment.
+
+{{< alert title="In this page" color="tip" >}}
+
+1. [Create a configuration fragment](#create-a-fragment)
+1. [Apply a fragment to multiple machines](#add-a-fragment-to-multiple-machines)
+1. [Modify an otherwise identical configuration](#modify-a-fragment)
+
+{{< /alert >}}
+
+For information on provisioning many machines, see [Provisioning](/fleet/provision/).
+
+## Create a fragment
+
+{{< table >}}
+{{% tablestep link="/configure/" %}}
+**1. Configure one machine**
+
+Start by configuring one of your machines.
+
+In the [Viam app](https://app.viam.com), use the **CONFIGURE** tab to build a configuration for all components and services you want to use on all your machines.
+
+{{<imgproc src="/how-tos/one-to-many/config.png" resize="800x" class="fill aligncenter" style="width: 400px" declaredimensions=true alt="Configuration builder UI">}}
+
+{{% /tablestep %}}
+{{% tablestep %}}
+**2. Copy the raw JSON**
+
+In your machine's **CONFIGURE** tab, switch to **JSON** and copy the raw JSON.
+
+{{<imgproc src="/how-tos/one-to-many/raw-json.png" resize="700x" class="fill aligncenter" style="width: 400px" declaredimensions=true alt="JSON subtab of the CONFIGURE tab">}}
+
+{{% /tablestep %}}
+{{% tablestep link="/fleet/fragments/" %}}
+**3. Create a fragment**
+
+On the **FLEET** page, go to the [**FRAGMENTS** tab](https://app.viam.com/fragments).
+
+Click **Create fragment**, and paste the copied JSON configuration into it.
+
+Set your privacy settings.
+There are three options for this:
+
+- **Public:** Any user inside or outside of your organization will be able to view and use this fragment.
+- **Private:** No user outside of your organization will be able to view or use this fragment.
+- **Unlisted:** Any user inside or outside of your organization, with a direct link, will be able to view and use this fragment.
+
+Click **Save**.
+
+If you want to edit the fragment later, do it from this screen.
+
+{{<imgproc src="/how-tos/one-to-many/new-fragment.png" resize="700x" class="fill aligncenter" style="width: 350px" declaredimensions=true alt="app.viam.com/fragment interface">}}
+
+{{% /tablestep %}}
+{{% tablestep %}}
+{{<imgproc src="/how-tos/one-to-many/delete.png" class="fill alignleft" resize="500x" style="width: 200px" declaredimensions=true alt="Delete">}}
+**4. Delete the original configuration (optional)**
+
+Now that the configuration is saved as a fragment, you can delete each resource in the original config from your machine and _replace the config with the fragment_ in the next step.
+In this way, you can keep all your machines up to date whenever you change the fragment.
+
+{{% /tablestep %}}
+{{< /table >}}
+
+## Add a fragment to multiple machines
+
+With your fragment created, you can add it to as many machines as you'd like.
+You can do this manually, as described here, or using [provisioning](/fleet/provision/).
+
+{{< table >}}
+{{% tablestep %}}
+{{<imgproc src="appendix/try-viam/rover-resources/fragments/fragments_list.png" resize="800x" class="fill alignleft imgzoom" style="width: 250px" declaredimensions=true alt="Add fragment">}}
+**1. Add the fragment to one machine**
+
+On your machine's **CONFIGURE** tab, click the **+** button and select **Insert fragment**.
+Search for your fragment and add it.
+
+Click **Save** in the upper right corner of the screen.
+
+{{% /tablestep %}}
+{{% tablestep %}}
+{{<imgproc src="/how-tos/one-to-many/repeat.svg" class="fill alignleft" style="width: 120px"  declaredimensions=true alt="Repeat">}}
+**2. Repeat for each machine**
+
+Repeat step 1 for each of the machines that you want to configure in the same way.
+
+If some of your machines have slight differences, you can still add the fragment and then add fragment overwrites in the next section.
+
+{{% /tablestep %}}
+{{< /table >}}
+
+## Modify fragment settings
+
+If your machines are similar but not identical, you can use a fragment with all of them, and then [overwrite parts of it](/fleet/fragments/#modify-the-config-of-a-machine-that-uses-a-fragment) to customize select fields of the configuration without modifying the upstream fragment.
+
+{{% alert title="Note" color="note" %}}
+If you modify fields within a fragment, your modifications will act as overwrites.
+If you later update the upstream fragment, your modifications will still apply.
+{{% /alert %}}
+
+{{< table >}}
+{{% tablestep link="/fleet/fragments/#modify-the-config-of-a-machine-that-uses-a-fragment" %}}
+{{<gif webm_src="/how-tos/fragment-overwrite.webm" mp4_src="/how-tos/fragment-overwrite.mp4" alt="A motor config panel from a fragment being edited with different direction and pwm pin values." max-width="500px" class="aligncenter" >}}
+
+<!-- markdownlint-disable MD036 -->
+
+**1. Edit your machine's config**
+
+Go to the **CONFIGURE** tab of the machine whose config you want to modify, and make your edits just as you would edit a non-fragment resource.
+
+You can click the **{}** button to switch to advanced view and see the changes.
+
+Don't forget to **Save**.
+
+{{% /tablestep %}}
+{{% tablestep %}}
+**2. Check your machine's logs**
+
+After configuring fragment overwrites, check your machine's [**LOGS** tab](/cloud/machines/#logs).
+If there are problems with overwrites to the fragment, the overwrites will not be partially applied and the configuration changes will not take effect until the configuration is fixed.
+
+{{% /tablestep %}}
+{{% tablestep %}}
+{{<imgproc src="/how-tos/one-to-many/reset.png" class="fill alignleft" resize="500x" style="width: 250px"  declaredimensions=true alt="Reset to fragment">}}
+**3. (Optional) Revert fragment modifications**
+
+If you need to restore the original fragment, click the **...** in the upper right corner of the card you modified, and click **Revert changes**.
+Now, the fragment will be identical to the upstream fragment.
+
+{{% /tablestep %}}
+{{< /table >}}