Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix blob storage and otel text #629

Merged
merged 1 commit into from
Jan 16, 2025
Merged

Fix blob storage and otel text #629

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/observability/how_to_guides/tracing/trace_with_opentelemetry.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ OTEL_EXPORTER_OTLP_HEADERS="x-api-key=<your langsmith api key>"

```bash
OTEL_EXPORTER_OTLP_ENDPOINT=https://api.smith.langchain.com/otel
OTEL_EXPORTER_OTLP_HEADERS="x-api-key=<your langsmith api key>,LANGSMITH_PROJECT=<project name>"
OTEL_EXPORTER_OTLP_HEADERS="x-api-key=<your langsmith api key>,Langsmith-Project=<project name>"
```

### 2. Log a trace
Expand Down Expand Up @@ -136,7 +136,7 @@ TRACELOOP_HEADERS=x-api-key=<your_langsmith_api_key>
#### **Optional: Specify a custom project name other than "default"**

```bash
TRACELOOP_HEADERS=x-api-key=<your_langsmith_api_key>,LANGSMITH_PROJECT=<langsmith_project_name>
TRACELOOP_HEADERS=x-api-key=<your_langsmith_api_key>,Langsmith-Project=<langsmith_project_name>
```

### 2. Initialize the SDK
Expand Down
6 changes: 3 additions & 3 deletions docs/self_hosting/configuration/blob_storage.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ import {

# Enable blob storage

By default, LangSmith stores run inputs, outputs, and errors in ClickHouse. In addition, LangSmith will store run manifests in Postgres. If you so choose, you can instead store this information in blob storage, which has a couple of notable benefits:
By default, LangSmith stores run inputs, outputs, errors, manifests, extras, and events in ClickHouse. If you so choose, you can instead store this information in blob storage, which has a couple of notable benefits:

1. In high trace environments, inputs, outputs, errors, and manifests may balloon the size of your databases.
1. In high trace environments, inputs, outputs, errors, manifests, extras, and events may balloon the size of your databases.
1. If using LangSmith Managed ClickHouse, you may want sensitive information in blob storage that resides in your environment.
To alleviate this, LangSmith supports storing run manifests, inputs, outputs, and errors in an external blob storage system.
To alleviate this, LangSmith supports storing run inputs, outputs, errors, manifests, extras, events, and attachments in an external blob storage system.

## Requirements

Expand Down
4 changes: 2 additions & 2 deletions docs/self_hosting/langsmith_managed_clickhouse.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ Using LangSmith Managed Clickhouse with your Self-Hosted LangSmith instance is f

This [reference doc](../reference/data_formats/run_data_format) explains the format we use to store runs (spans), which are the building blocks of traces.

Our definition of sensitive information as it relates to application data are `inputs`, `outputs`, and `errors` of a run, since these fields can contain prompts and completions from LLMs.
Our definition of sensitive information as it relates to application data are `inputs`, `outputs`, `errors`, `manifests`, `extras`, and `events` of a run, since these fields can contain prompts and completions from LLMs.

With LangSmith-managed ClickHouse, we store `inputs`, `outputs`, and `errors` in cloud object storage (S3 or GCS) within your cloud and store the rest of the run data in ClickHouse. This ensures that sensitive information doesn't leave your VPC.
With LangSmith-managed ClickHouse, we store `inputs`, `outputs`, `errors`, `manifests`, `extras`, and `events` in cloud object storage (S3 or GCS) within your cloud and store the rest of the run data in ClickHouse. This ensures that sensitive information doesn't leave your VPC.

:::

Expand Down
2 changes: 1 addition & 1 deletion docs/self_hosting/release_notes.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ This release adds a number of new features, improves the performance of the Thre
### Infrastructure Changes

- Added a Helm configuration option for Time To Live for traces. When set, this setting will apply only to net-new ingested traces and by changing this setting, _traces will be automatically and irrevocably deleted from Clickhouse after expiration of the TTL._ For more details see [TTL and Data Retention](./configuration/ttl). You may also need to change/audit your project default TTL settings.
- Added configuration option to enable `blobStorage`. This will move run inputs, outputs, and errors to blob storage to lower load on ClickHouse/reduce disk usage. Currently only S3 and GCP are supported. For more details see [Enable Blob Storage](./configuration/blob_storage).
- Added configuration option to enable `blobStorage`. This will move run inputs, outputs, errors, manifests, extras, and events to blob storage to lower load on ClickHouse/reduce disk usage. Currently only S3 and GCP are supported. For more details see [Enable Blob Storage](./configuration/blob_storage).
- Default Resource/Limits for all resources. Note that you may need to tweak your cluster settings.
- By default we will use:
- 16 CPU
Expand Down
Loading