feat: add egress information for telemetry/licensing (#612)

docs/self_hosting/egress.mdx

@@ -0,0 +1,92 @@
+---
+sidebar_label: Egress for Subscription Metrics and Operational Metadata
+sidebar_position: 7
+---
+
+# Egress for Subscription Metrics and Operational Metadata
+
+Self-Hosted LangSmith instances store all information locally and will never send sensitive information outside of your network. We currently only track platform usage for billing purposes according to the entitlements in your order. In order to better remotely support our customers, we do require egress to `https://beacon.langchain.com`.
+
+In the future, we will be introducing support diagnostics to help us ensure that the LangSmith platform is running at an optimal level within your environment.
+
+:::warning Important
+**This will require egress to `https://beacon.langchain.com` from your network.**
+:::
+
+Generally, data that we send to Beacon can be categorized as follows:
+
+- Subscription Metrics
+  - Subscription metrics are used to determine level of access and utilization of LangSmith. This includes, but are not limited to:
+    - Number of traces
+    - Seats allocated per contract
+    - Seats in currently use
+- Operational Metadata
+  - This metadata will contain and collect the above subscription metrics to assist with remote support, allowing the LangChain team to diagnose and troubleshoot performance issues more effectively and proactively.
+
+## Example Payloads
+
+In an effort to maximize transparency, we provide sample payloads here:
+
+### License Verification
+
+**Endpoint:**
+
+`POST beacon.langchain.com/v1/beacon/verify`
+
+**Request:**
+
+```json
+{
+  "license: "<YOUR_LICENSE_KEY>"
+}
+```
+
+**Response:**
+
+```json
+{
+  "token": "Valid JWT" //Short-lived JWT token to avoid repeated license checks
+}
+```
+
+### Usage Reporting
+
+**Endpoint:**
+
+`POST beacon.langchain.com/v1/beacon/ingest-traces`
+
+**Request:**
+
+```json
+{
+  "license": "<YOUR_LICENSE_KEY>",
+  "trace_transactions": [
+    {
+      "id": "af28dfea-5358-463d-a2dc-37df1da72498",
+      "tenant_id": "3a1c2b6f-4430-4b92-8a5b-79b8b567bbc1",
+      "session_id": "b26ae531-cdb3-42a5-8bcf-05355199fe27",
+      "trace_count": 5,
+      "start_insertion_time": "2025-01-06T10:00:00Z",
+      "end_insertion_time": "2025-01-06T11:00:00Z",
+      "start_interval_time": "2025-01-06T09:00:00Z",
+      "end_interval_time": "2025-01-06T10:00:00Z",
+      "status": "completed",
+      "num_failed_send_attempts": 0,
+      "transaction_type": "type1",
+      "organization_id": "c5b5f53a-4716-4326-8967-d4f7f7799735"
+    }
+  ]
+}
+```
+
+**Response:**
+
+```json
+{
+  "inserted_count": 1 //Number of transactions successfully ingested
+}
+```
+
+## Our Commitment
+
+LangChain will not store any sensitive information in the Subscription Metrics or Operational Metadata. Any data collected will not be shared with a third party. If you have any concerns about the data being sent, please reach out to your account team.

docs/self_hosting/faq.mdx

@@ -9,7 +9,6 @@ description: "Frequently Asked Questions"
 ### _I can't create API keys or manage users in the UI, what's wrong?_
 
 - You have likely deployed LangSmith without setting up SSO. LangSmith requires SSO to manage users and API keys. You can find more information on setting up SSO in the [configuration section.](../self_hosting/configuration/sso)
--
 
 #### _How does load balancing/ingress work_?
 
@@ -28,12 +27,15 @@ You can find more information on setting up SSO in the [configuration section.](
 
 #### _Does my application need egress to function properly?_
 
-Our deployment only needs egress for a few things:
+Our deployment only needs egress for a few things (most of which can reside within your VPC):
 
 - Fetching images (If mirroring your images, this may not be needed)
-- Talking to any LLMs
+- Talking to any LLM endpoints
 - Talking to any external storage services you may have configured
 - Fetching OAuth information
+- Subscription Metrics and Operational Metadata (if not running in offline mode)
+  - Requires egress to `https://beacon.langchain.com`
+  - See [Egress](../self_hosting/egress) for more information
 
 Your VPC can set up rules to limit any other access.  
 Note: We require the `X-Organization-Id` and `X-Tenant-Id` headers to be allowed to be passed through to the backend service. These are used to determine which organization and workspace (previously called "tenant") the request is for.

docs/self_hosting/index.md

@@ -24,6 +24,7 @@ Step-by-step guides that cover the installation, configuration, and scaling of y
 - [Usage](./self_hosting/usage): How to use your self-hosted instance of LangSmith.
 - [Organization Charts](./self_hosting/organization_charts): View trace counts for your organization and workspaces.
 - [Upgrades](./self_hosting/upgrades): How to upgrade your self-hosted instance of LangSmith.
+- [Egress for Subscription Metrics and Operational Metadata](./self_hosting/egress): Egress requirements for Subscription Metrics and Operational Metadata.
 - [Release notes](./self_hosting/release_notes): The latest release notes for LangSmith.
 - - [Week of August 26, 2024 - LangSmith v0.7](./self_hosting/release_notes#week-of-august-26-2024---langsmith-v07): Release notes for version 0.7 of LangSmith.
   - [Week of June 17, 2024 - LangSmith v0.6](./self_hosting/release_notes#week-of-june-17-2024---langsmith-v05): Release notes for version 0.6 of LangSmith.

docs/self_hosting/installation/docker.mdx

@@ -30,8 +30,10 @@ This guide provides instructions for installing and setting up your environment
    ```bash
    openssl rand -base64 32
    ```
+1. Egress to `https://beacon.langchain.com` (if not running in offline mode)
+   1. LangSmith requires egress to `https://beacon.langchain.com` for license verification and usage reporting. This is required for LangSmith to function properly. You can find more information on egress requirements in the [Egress](/self_hosting/egress) section.
 1. Configuration
-1. There are several configuration options that you can set in the `.env` file. You can find more information on the available configuration options in the [Configuration](/self_hosting/configuration) section.
+   1. There are several configuration options that you can set in the `.env` file. You can find more information on the available configuration options in the [Configuration](/self_hosting/configuration) section.
 
 ## Running via Docker Compose
 

docs/self_hosting/installation/kubernetes.mdx

@@ -63,6 +63,8 @@ Ensure you have the following tools/items ready. Some items are marked optional:
    ```bash
    openssl rand -base64 32
    ```
+1. Egress to `https://beacon.langchain.com` (if not running in offline mode)
+   1. LangSmith requires egress to `https://beacon.langchain.com` for license verification and usage reporting. This is required for LangSmith to function properly. You can find more information on egress requirements in the [Egress](/self_hosting/egress) section.
 1. Configuration
    1. There are several configuration options that you can set in the `langsmith_config.yaml` file. You can find more information on specific configuration options in the [Configuration](/self_hosting/configuration) section.
    1. If you are new to Kubernetes or Helm, we’d recommend starting with one of the example configurations in the examples directory of the Helm Chart repository here: [LangSmith helm chart examples](https://github.com/langchain-ai/helm/tree/main/charts/langsmith/examples).