diff --git a/docs/dev/reference/machine-to-machine-comms.md b/docs/dev/reference/machine-to-machine-comms.md new file mode 100644 index 0000000000..f461354460 --- /dev/null +++ b/docs/dev/reference/machine-to-machine-comms.md @@ -0,0 +1,118 @@ +--- +title: "Machine-to-Machine Communication: End-to-End Flow" +linkTitle: "Machine-to-Machine Communication" +weight: 60 +type: "docs" +description: "Explanation of how a machine and its parts interact at the communication layer." +aliases: + - "/internals/robot-to-robot-comms/" + - "/internals/machine-to-machine-comms/" + - "/architecture/machine-to-machine-comms/" +toc_hide: true +--- + +When building a smart machine application in the [Viam app](https://app.viam.com), a user typically begins by configuring their machine which can consist of one or more {{< glossary_tooltip term_id="part" text="parts" >}}. +Next they will test that it is wired up properly using the Viam app's Control page. +Once they've ensured everything is wired up properly, they will build their main application and the business logic for their machine using one of Viam's language SDKs. +This SDK-based application is typically run on either the main part of the machine or a separate computer dedicated to running the business logic for the machine. + +Below, we describe the flow of information through a Viam-based multipart machine and then get into the specifics of what backs these connections and communications APIs. + +## High-level inter-robot/SDK communication + +To begin, let's define our machine's topology: + +![robot communication diagram](/internals/robot-to-robot-comms/robot-communication-diagram.png) + +This machine is made of two parts and a separate SDK-based application, which we'll assume is on a third machine, though it could just as easily run on the main part without any changes. + +- The first and main part, RDK Part 1, consists of a Raspberry Pi and a single USB connected camera called Camera. + +- The second and final part, RDK Part 2, consists of a Raspberry Pi connected to a robotic arm over ethernet and a gantry over GPIO. + +RDK Part 1 will establish a bidirectional gRPC/{{< glossary_tooltip term_id="webrtc" >}} connection to RDK Part 2. +RDK Part 1 is considered the controlling peer (client). +RDK Part 2 is consider the controlled peer (server). + +Let's suppose our SDK application uses the camera to track the largest object in the scene and instructs the arm to move to that same object. + +Since RDK Part 1 is the main part and has access to all other parts, the application will connect to it using the SDK. +Once connected, it will take the following series of actions: + +
    +
  1. Get segmented point clouds from the camera and the object segmentation service.
    2. + +
  2. Find the largest object by volume.
    3. + +
  3. Take the object's center pose and tell the motion service to move the arm to that point.
    4. + +
  4. Go back to 1.
    5. +
+Let's breakdown how these steps are executed. + +
    +
  1. Get segmented point clouds from the camera and the object segmentation service:
    2. + +![robot communication diagram](/internals/robot-to-robot-comms/getobjectpointcloud-flow.png) + +
      +
    1. The SDK will send a GetObjectPointClouds request with Camera being referenced in the message to RDK Part 1's Object Segmentation Service.
      2. + +
    2. RDK Part 1 will look up the camera referenced, call the GetPointCloud method on it.
      3. + +
    3. The Camera will return the PointCloud data to RDK Part
      4. + +
    4. RDK Part 1 will use a point cloud segmentation algorithm to segment geometries in the PointCloud.
      5. +{{% alert title="Important" color="note" %}} +The points returned are respective to the reference frame of the camera. +This will become important in a moment. +{{% /alert %}} +
    5. The set of segmented point clouds and their bounding geometries are sent back to the SDK-based application.
      6. +
    + +
  2. Find the largest object by volume:
    3. +
      +
    1. The application will iterate over the geometries of the segmented point clouds returned to it and find the object with the greatest volume and record its center pose.
      2. +
    + +
  3. Take the object's center pose and tell the motion service to move the arm to that point:
    4. + +![motion service move flow](/internals/robot-to-robot-comms/motion-service-move-flow.png) + +
      +
    1. The SDK application will send a Move request for the arm to the motion service on RDK Part 1 with the destination set to the center point determined by the application.
      2. + +
    2. RDK Part 1's motion service will break down the Move request and perform the necessary frame transforms before sending the requests along to the relevant components. +This is where the frame system comes into play. +Our center pose came from the camera but we want to move the arm to that position even though the arm lives in its own frame. +The frame system logic in the RDK automatically handles the transformation logic between these two reference frames while also handling understanding how the frame systems are connected across the two parts.
      3. + +
    3. Having computed the pose in the reference frame of the arm, the motion service takes this pose, and sends a plan on how to move the arm in addition to the gantry to achieve this new goal pose to RDK Part 2. +The plan consists of a series of movements that combine inverse kinematics, mathematics, and constraints based motion planning to get the arm and gantry to their goal positions.
      4. + +
    4. In executing the plan, which is being coordinated on RDK Part 1, Part 1 will send messages to the Arm and Gantry on RDK Part 2. +RDK Part 2 will be unaware of the actual plan and instead will only receive distinct messages to move the components individually.
      5. + +
    5. The arm and gantry connected to RDK Part 2 return an acknowledgement of the part Move requests to RDK Part 2.
      6. + +
    6. RDK Part 2 returns an acknowledgement of the Motion Move request to RDK Part 1.
      7. + +
    7. RDK Part 1 returns an acknowledgement of the Motion Move request to the SDK application.
      8. +
    + +## Low-level inter-robot/SDK communication + +All component and service types in the RDK, and the Viam API for that matter, are represented as [Protocol Buffers (protobuf)](https://developers.google.com/protocol-buffers) services. +protobuf is a battle tested Interface Description Language (IDL) that allows for specifying services, their methods, and the messages that comprise those methods. +Code that uses protobuf is autogenerated and compiles messages into a binary form. + +[gRPC](https://grpc.io/) is responsible for the transport and communication of protobuf messages when calling protobuf methods. +It generally works over a TCP, TLS backed HTTP2 connection operating over framing see [gRPC's HTTP2 documentation](https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md) for more. + +The RDK uses protobuf and gRPC to enable access and control to its components and services. +That means if there are two arms in a machine configuration, there is only one Arm service that handles the Remote Procedure Calls (RPC) for all arms configured. + +In addition to gRPC, the RDK uses [WebRTC](https://webrtcforthecurious.com/) video and audio streams and data channels to enable peer to peer (P2P) communication between machine parts as well as SDKs and the Remote Control interface. + +An outline of how WebRTC is used lives on [Go.dev](https://pkg.go.dev/go.viam.com/utils@v0.0.3/rpc#hdr-Connection), but in short, an RDK is always waiting on the Viam app ([app.viam.com](https://app.viam.com)) to inform it of a connection requesting to be made to it whereby it sends details about itself and how to connect on a per connection basis. +Once a connection is made, the Viam app is no longer involved in any packet transport and leaves it up to the two peers to communicate with each other. diff --git a/docs/dev/tools/troubleshooting.md b/docs/dev/tools/common-errors.md similarity index 94% rename from docs/dev/tools/troubleshooting.md rename to docs/dev/tools/common-errors.md index 80019125cb..dbbaf0c632 100644 --- a/docs/dev/tools/troubleshooting.md +++ b/docs/dev/tools/common-errors.md @@ -1,6 +1,6 @@ --- -title: "Troubleshooting" -linkTitle: "Troubleshooting" +title: "Common Errors & Known Issues" +linkTitle: "Common Errors" weight: 50 type: "docs" description: "A guide to troubleshooting a Viam-based machine or system of machines with fixes to common problems." @@ -11,6 +11,8 @@ date: "2022-01-01" This document lists common errors encountered when working with `viam-server` and the [Viam app](https://app.viam.com), and provides simple steps to resolve them. While many common issues and their possible resolutions are presented here, this list is not comprehensive. +To view logs or get a remote shell on a machine see [Troubleshoot](/manage/troubleshoot/troubleshoot/). + If you have encountered an error that is not listed here, we'd love to hear from you on our [Community Discord](https://discord.gg/viam)! Please post the error message you received along with how you were able to trigger it and we'll see if we can help. @@ -18,38 +20,6 @@ Please post the error message you received along with how you were able to trigg For information on the status of [app.viam.com](https://app.viam.com), visit [status.viam.com](https://status.viam.com/). -## Enable debug level logs - -The default log level for `viam-server` and any running resources is `"Info"`. -If you are not seeing helpful logs, you can try changing the log level to `"Debug"`. - -{{< tabs >}} -{{% tab name="For individual resources" %}} - -Add the `log_configuration` option to the resource's JSON configuration: - -```json -"log_configuration": { - "level": "Debug" -}, -"attributes": { ... } -``` - -{{% /tab %}} -{{% tab name="For viam-server" %}} - -Add `"debug": true` to the machine's configuration: - -```json -{ - "debug": true, - "components": [{ ... }] -} -``` - -{{% /tab %}} -{{< /tabs >}} - ## Common installation errors ### The authenticity of host 'hostname.local' can't be established diff --git a/docs/manage/_index.md b/docs/manage/_index.md index a8e61399bc..ae09955b5a 100644 --- a/docs/manage/_index.md +++ b/docs/manage/_index.md @@ -8,3 +8,26 @@ no_list: true open_on_desktop: true overview: true --- + +{{< how-to-expand "Deploy software to machines" "4" "BEGINNER-FRIENDLY" >}} +{{< cards >}} +{{% card link="/how-tos/configure/" noimage="true" %}} +{{% card link="/how-tos/one-to-many/" noimage="true" %}} +{{% card link="/how-tos/provision-setup/" noimage="true" %}} +{{% card link="/how-tos/provision/" noimage="true" %}} +{{< /cards >}} +{{< /how-to-expand >}} + +{{< how-to-expand "Manage a large fleet of machines" "2" "INTERMEDIATE" >}} +{{< cards >}} +{{% card link="/how-tos/manage-fleet/" noimage="true" %}} +{{% card link="/how-tos/deploy-packages/" noimage="true" %}} +{{< /cards >}} +{{< /how-to-expand >}} + +{{< how-to-expand "Troubleshooting" "2" "INTERMEDIATE" >}} +{{< cards >}} +{{% card link="/how-tos/manage-fleet/" noimage="true" %}} +{{% card link="/how-tos/deploy-packages/" noimage="true" %}} +{{< /cards >}} +{{< /how-to-expand >}} diff --git a/docs/manage/deploy/one-to-many.md b/docs/manage/deploy/one-to-many.md new file mode 100644 index 0000000000..0135bd7aad --- /dev/null +++ b/docs/manage/deploy/one-to-many.md @@ -0,0 +1,154 @@ +--- +title: "Configure multiple similar machines" +linkTitle: "Configure many machines" +weight: 40 +type: "docs" +tags: ["data management", "data", "services"] +images: ["/how-tos/one-to-many/new-fragment.png"] +description: "Viam has a built-in tool called fragments for using the same configuration on multiple machines." +aliases: + - /use-cases/one-to-many/ +languages: [] +viamresources: [] +platformarea: ["fleet"] +level: "Beginner" +date: "2024-08-09" +# updated: "" # When the tutorial was last entirely checked +cost: "0" +--- + +Viam has a built-in tool called _{{< glossary_tooltip term_id="fragment" text="fragments" >}}_ for using the same configuration on multiple machines. +A fragment can configure just one resource a machine uses, or all the resources it uses. +If most of your machines are similar but not identical, you can use fragments and then manually add modifications for individual machines. + +When you update a fragment, it updates the configurations of all machines that use that fragment. + +{{< alert title="In this page" color="tip" >}} + +1. [Create a configuration fragment](#create-a-fragment) +1. [Apply a fragment to multiple machines](#add-a-fragment-to-multiple-machines) +1. [Modify an otherwise identical configuration](#modify-a-fragment) + +{{< /alert >}} + +For information on provisioning many machines, see [Provisioning](/fleet/provision/). + +## Create a fragment + +{{< table >}} +{{% tablestep link="/configure/" %}} +**1. Configure one machine** + +Start by configuring one of your machines. + +In the [Viam app](https://app.viam.com), use the **CONFIGURE** tab to build a configuration for all components and services you want to use on all your machines. + +{{}} + +{{% /tablestep %}} +{{% tablestep %}} +**2. Copy the raw JSON** + +In your machine's **CONFIGURE** tab, switch to **JSON** and copy the raw JSON. + +{{}} + +{{% /tablestep %}} +{{% tablestep link="/fleet/fragments/" %}} +**3. Create a fragment** + +On the **FLEET** page, go to the [**FRAGMENTS** tab](https://app.viam.com/fragments). + +Click **Create fragment**, and paste the copied JSON configuration into it. + +Set your privacy settings. +There are three options for this: + +- **Public:** Any user inside or outside of your organization will be able to view and use this fragment. +- **Private:** No user outside of your organization will be able to view or use this fragment. +- **Unlisted:** Any user inside or outside of your organization, with a direct link, will be able to view and use this fragment. + +Click **Save**. + +If you want to edit the fragment later, do it from this screen. + +{{}} + +{{% /tablestep %}} +{{% tablestep %}} +{{}} +**4. Delete the original configuration (optional)** + +Now that the configuration is saved as a fragment, you can delete each resource in the original config from your machine and _replace the config with the fragment_ in the next step. +In this way, you can keep all your machines up to date whenever you change the fragment. + +{{% /tablestep %}} +{{< /table >}} + +## Add a fragment to multiple machines + +With your fragment created, you can add it to as many machines as you'd like. +You can do this manually, as described here, or using [provisioning](/fleet/provision/). + +{{< table >}} +{{% tablestep %}} +{{}} +**1. Add the fragment to one machine** + +On your machine's **CONFIGURE** tab, click the **+** button and select **Insert fragment**. +Search for your fragment and add it. + +Click **Save** in the upper right corner of the screen. + +{{% /tablestep %}} +{{% tablestep %}} +{{}} +**2. Repeat for each machine** + +Repeat step 1 for each of the machines that you want to configure in the same way. + +If some of your machines have slight differences, you can still add the fragment and then add fragment overwrites in the next section. + +{{% /tablestep %}} +{{< /table >}} + +## Modify fragment settings + +If your machines are similar but not identical, you can use a fragment with all of them, and then [overwrite parts of it](/fleet/fragments/#modify-the-config-of-a-machine-that-uses-a-fragment) to customize select fields of the configuration without modifying the upstream fragment. + +{{% alert title="Note" color="note" %}} +If you modify fields within a fragment, your modifications will act as overwrites. +If you later update the upstream fragment, your modifications will still apply. +{{% /alert %}} + +{{< table >}} +{{% tablestep link="/fleet/fragments/#modify-the-config-of-a-machine-that-uses-a-fragment" %}} +{{}} + + + +**1. Edit your machine's config** + +Go to the **CONFIGURE** tab of the machine whose config you want to modify, and make your edits just as you would edit a non-fragment resource. + +You can click the **{}** button to switch to advanced view and see the changes. + +Don't forget to **Save**. + +{{% /tablestep %}} +{{% tablestep %}} +**2. Check your machine's logs** + +After configuring fragment overwrites, check your machine's [**LOGS** tab](/cloud/machines/#logs). +If there are problems with overwrites to the fragment, the overwrites will not be partially applied and the configuration changes will not take effect until the configuration is fixed. + +{{% /tablestep %}} +{{% tablestep %}} +{{}} +**3. (Optional) Revert fragment modifications** + +If you need to restore the original fragment, click the **...** in the upper right corner of the card you modified, and click **Revert changes**. +Now, the fragment will be identical to the upstream fragment. + +{{% /tablestep %}} +{{< /table >}} diff --git a/docs/manage/deploy/ota.md b/docs/manage/deploy/ota.md index 8d4c6600e2..da0098f4e1 100644 --- a/docs/manage/deploy/ota.md +++ b/docs/manage/deploy/ota.md @@ -6,4 +6,97 @@ layout: "docs" type: "docs" no_list: true description: "TODO" +images: ["/registry/module-puzzle-piece.svg"] +description: "You can use a fragment to deploy software packages to many machines, as well as to keep those software packages versioned." +languages: [] +viamresources: [] +platformarea: ["registry", "fleet"] +level: "Intermediate" +date: "2024-08-28" +# updated: "" # When the tutorial was last entirely checked +cost: "0" --- + +Viam has a built-in tool called _{{< glossary_tooltip term_id="fragment" text="fragments" >}}_ for using the same configuration on multiple machines. +You can use a fragment to deploy software packages to many machines, as well as to keep those software packages versioned. + +## Create a fragment + +{{< table >}} +{{% tablestep link="/configure/" %}} +**1. Configure your software** + +Start by adding a new machine in the [Viam app](https://app.viam.com). +You do not need to follow the setup instructions. + +Use the **CONFIGURE** tab to add the component or service you want to deploy across your machines. + +{{}} + +{{% /tablestep %}} +{{% tablestep %}} +**2. Copy the raw JSON** + +In your machine's **CONFIGURE** tab, switch to **JSON** and copy the raw JSON. + +{{}} + +{{% /tablestep %}} +{{% tablestep link="/fleet/fragments/" %}} +**3. Create a fragment** + +Go to [app.viam.com/fragments](https://app.viam.com/fragments). + +Add a fragment, and paste the copied JSON configuration into it. + +{{}} + +Set your privacy settings. +There are three options for this: + +- **Public:** Any user inside or outside of your organization will be able to view and use this fragment. +- **Private:** No user outside of your organization will be able to view or use this fragment. +- **Unlisted:** Any user inside or outside of your organization, with a direct link, will be able to view and use this fragment. + +Click **Save**. + +If you want to edit the fragment later, do it from this screen. + +{{% /tablestep %}} +{{% tablestep %}} +{{}} +**4. Delete the original machine configuration (optional)** + +Now that the configuration is saved as a fragment, you can delete the machine you created in step 1. +We only created this machine to easily generate the JSON config for the fragment. + +{{% /tablestep %}} +{{< /table >}} + +## Add the fragment to multiple machines + +With your fragment created, you can add it to as many machines as you'd like: + +{{< alert title="Tip" color="tip" >}} +You can add multiple fragments to one machine. +{{< /alert >}} + +{{< table >}} +{{% tablestep %}} +{{}} +**1. Add the fragment to one machine** + +On your machine's **CONFIGURE** tab, click the **+** button and select **Insert fragment**. +Search for your fragment and add it. + +Click **Save** in the upper right corner of the screen. + +{{% /tablestep %}} +{{% tablestep %}} +{{}} +**2. Repeat for each machine** + +Repeat step 1 for each of the machines that you want to add and manage the package for. + +{{% /tablestep %}} +{{< /table >}} diff --git a/docs/manage/deploy/provision.md b/docs/manage/deploy/provision.md deleted file mode 100644 index e5a2cb83c2..0000000000 --- a/docs/manage/deploy/provision.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -linkTitle: "Provision devices" -title: "Provision devices" -weight: 40 -layout: "docs" -type: "docs" -no_list: true -description: "TODO" ---- diff --git a/docs/manage/deploy/provision/_index.md b/docs/manage/deploy/provision/_index.md new file mode 100644 index 0000000000..050a571cc5 --- /dev/null +++ b/docs/manage/deploy/provision/_index.md @@ -0,0 +1,261 @@ +--- +linkTitle: "Provision devices" +title: "Provision devices using viam-agent" +weight: 40 +layout: "docs" +type: "docs" +no_list: true +description: "Provision a machine as it first comes online with a pre-defined configuration - in the factory or when the machine is taken into service." +images: ["/platform/provisioning-demo.gif"] +videos: ["/platform/provisioning-demo.webm", "/platform/provisioning-demo.mp4"] +tags: ["fleet management", "viam-server", "viam-agent"] +# SMEs: James, Ale +aliases: + - "/build/provision/" + - "/fleet/provision/" +date: "2024-08-16" +# updated: "" # When the content was last entirely checked +--- + +You can use Viam's software provisioning manager (`agent-provisioning`), to provision a machine as it first comes online with a pre-defined configuration. +This is useful when deploying a fleet of machines directly from the factory to a customer, or when bundling proprietary software on your Viam machine. + +Provisioning is a feature of [`viam-agent`](/configure/agent/), which you can install as part of your manufacturing process. +`agent-provisioning` will then perform the rest of the first-time setup for your machine once an [end user sets up the machine](#end-user-experience). + +Consider a company that sells machines that monitor weather conditions on a maritime craft and provide navigation advice based on those readings. +Such a machine might use Viam to regularly capture and upload a stream of sensor readings, for example. +To parse the readings and provide tailored guidance to a ship's captain, the company writes their own proprietary application which includes live analytics and speech generation for conveying advice to the captain. + +Using `agent-provisioning`, this company can ship their machines directly to customers with `viam-agent` installed. +When a customer sets up their machine, `viam-agent` installs `viam-server`. +By having the end customer set up the machine, the company: + +- eliminates per-device setup and individualization at the factory +- allows for tailored configurations per customer as needed +- allows customer to provide their own WiFi credentials + +{{< alert title="Support Notice" color="note" >}} + +Provisioning is supported and tested only on Debian 11 (Bullseye), and 12 (Bookworm) but should work on most distros using NetworkManager v1.42 (or newer) as well. +For Bullseye, the installation of `viam-agent` changes the network configuration to use NetworkManager. + +{{< /alert >}} + +For a guide on how to configure provisioning for your machine, see: + +{{< cards >}} +{{% card link="/how-tos/provision-setup/" %}} +{{< /cards >}} + +## End user experience + +End users receive a machine, and use either a captive web portal or mobile app to complete the machine setup. + +One option is to use the [Viam mobile app](/fleet/control/#control-interface-in-the-viam-mobile-app). +The Viam mobile app allows end users to create a new machine in the app, and `agent-provisioning` will then install `viam-server` and run it with a provided configuration. + +To add your branding, you can build your own mobile app and use the [Flutter SDK](https://flutter.viam.dev/viam_protos.provisioning.provisioning/ProvisioningServiceClient-class.html) or the [TypeScript SDK](https://github.com/viamrobotics/viam-typescript-sdk/blob/main/src/app/provisioning-client.ts) to connect to `viam-agent` and provision your machines. + +If you are not using Flutter or TypeScript and would like to use provisioning, please [contact us](mailto:support@viam.com). + +For an end-user guide to setting up their machine, see: + +{{< cards >}} +{{% card link="/how-tos/provision/" %}} +{{< /cards >}} + +This is the general process for provisioning depending on whether you are using a captive web portal or a mobile app: + +{{< tabs >}} +{{% tab name="Mobile app" min-height="703px" %}} + +{{