DOCS-1842: Add handling for disk full (#2905)

docs/data/capture/_index.md

@@ -68,7 +68,8 @@ To capture data from one or more machines, you must first add the [data manageme
         "capture_dir": "",
         "tags": [],
         "capture_disabled": false,
-        "sync_disabled": true
+        "sync_disabled": true,
+        "delete_every_nth_when_disk_full": 5
       }
     }
   ]
@@ -367,6 +368,31 @@ The following example captures data from the `ReadImage` method of a camera:
 
 {{% /expand%}}
 
+## Automatic data deletion
+
+If [cloud sync](/data/cloud-sync/) is enabled, the data management service deletes captured data once it has successfully synced to the cloud.
+
+The data management service will also automatically delete local data in the event your machine's local storage fills up.
+Local data is automatically deleted when _all_ of the following conditions are met:
+
+- Data capture is enabled on the data manager service
+- Local disk usage percentage is greater than or equal to 90%
+- The Viam capture directory is at least 50% of the current local disk usage
+
+If local disk usage is greater than or equal to 90%, but the Viam capture directory is not at least 50% of that usage, a warning log message will be emitted instead and no action will be taken.
+
+Automatic file deletion only applies to files in the specified Viam capture directory, which is set to `~/.viam/capture` by default.
+Data outside of this directory is not touched by automatic data deletion.
+
+If your machine captures a large amount of data, or frequently goes offline for long periods of time while capturing data, consider moving the Viam capture directory to a larger, dedicated storage device on your machine if available. You can change the capture directory using the `capture_dir` attribute.
+
+You can also control how local data is deleted if your machine's local storage becomes full, using the `delete_every_nth_when_disk_full` attribute:
+
+- The `delete_every_nth_when_disk_full` attribute controls how many files to delete when local storage meets the above fullness criteria.
+  The data management service will delete every Nth file that has been captured upon reaching this threshold.
+  The default value is `5`, meaning that every fifth captured file will be deleted.
+  This value should be suitable for most cases.
+
 ## View captured data
 
 To view captured data for a machine, click on the data icon next to the **Save** button on the top right of the app.

docs/data/cloud-sync.md

@@ -70,6 +70,34 @@ You can use a {{< glossary_tooltip term_id="module" text="module" >}} to sync da
 For example, if you rely on mobile data but have intermittent WiFi connection in certain locations or at certain times of the day, you may want to trigger sync to only occur when these conditions are met.
 To set up triggers for syncing see [Trigger Sync](/data/trigger-sync/).
 
+### Configure sync threads
+
+When cloud sync is enabled, the data management service will use up to `1000` concurrent CPU threads by default to sync data to the Viam cloud, depending on how much data is being synced.
+You can adjust the permitted thread count with the `maximum_num_sync_threads` attribute.
+
+The default value of `1000` concurrent threads is sufficient for most use cases, but if you are using limited hardware, are operating under heavy CPU load, or are syncing a large amount of data at once, consider lowering this value as needed.
+
+{{%expand "Click to view the JSON configuration with 250 threads configured" %}}
+
+```json {class="line-numbers linkable-line-numbers"}
+{
+  "components": [],
+  "services": [
+    {
+      "name": "data_manager",
+      "type": "data_manager",
+      "attributes": {
+        "sync_interval_mins": 0.1,
+        "capture_dir": "",
+        "maximum_num_sync_threads": 250
+      }
+    }
+  ]
+}
+```
+
+{{% /expand%}}
+
 ### Pause sync
 
 You can pause Cloud Sync at any time by navigating to your machine's **CONFIGURE** tab and disabling **Syncing** for your [data management service](../).
@@ -134,15 +162,16 @@ In the example pictured here, the data management service syncs the configured c
   For example, if the service has uploaded 33% of the data and then the internet connection is severed, sync is interrupted.
   Once the service retries and successfully connects, data synchronization resumes at 33%.
 
-- **Storage** When a machine loses its internet connection, it cannot resume cloud sync until it can reach the Viam cloud again.
+- **Storage** Data that is successfully synced to the cloud is automatically deleted from local storage.
+
+  When a machine loses its internet connection, it cannot resume cloud sync until it can reach the Viam cloud again.
 
   To ensure that the machine can store all data captured while it has no connection, you need to provide enough local data storage.
 
   {{< alert title="Warning" color="warning" >}}
 
-  Currently, the data management service can use the entire available disk space to store data.
-  If the machine loses connectivity and remains disconnected, data capture can eventually use all disk space.
-  Currently, Viam does not safeguard against this.
+  If your machine's disk fills up beyond a certain threshold, the data management service will delete captured data to free up additional space and maintain a working machine.
+  See [automatic data deletion](/data/capture/#automatic-data-deletion) for configuration options.
 
   {{< /alert >}}
 

docs/tutorials/services/data-mlmodel-tutorial.md

@@ -75,6 +75,7 @@ The [data management](/data/) service has two parts: [data capture](/data/captur
 
 Data capture and data sync are frequently used together, and are both enabled by default when you add the data management service to your machine.
 However, if you want to manage your machine's captured data yourself, you can enable data capture but disable data sync.
+If you are capturing data to a device with limited storage, or intend to capture a large amount of data, see [automatic data deletion](/data/capture/#automatic-data-deletion).
 
 To capture data from your machine and sync to the Viam app, add the data management service and configure data capture for at least one component.