Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the LangChain integration #15229

Merged
merged 10 commits into from
Jul 21, 2023
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,10 @@ datadog_checks_base/tests/**/test_db_statements.py @DataDog/database-monitoring
/oracle/*.md @DataDog/agent-integrations @DataDog/database-monitoring @DataDog/documentation
/oracle/manifest.json @DataDog/agent-integrations @DataDog/database-monitoring @DataDog/documentation

# APM Integrations
/langchain/ @DataDog/apm-core-python @DataDog/llm-obs @DataDog/agent-integrations @DataDog/documentation
/openai/ @DataDog/apm-core-python @DataDog/llm-obs @DataDog/agent-integrations @DataDog/documentation


# Windows agent
datadog_checks_base/datadog_checks/base/checks/win/ @DataDog/windows-agent @DataDog/agent-integrations
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/config/labeler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,8 @@ integration/kubernetes_state_core:
- kubernetes_state_core/**/*
integration/kyototycoon:
- kyototycoon/**/*
integration/langchain:
- langchain/**/*
integration/lighttpd:
- lighttpd/**/*
integration/linkerd:
Expand Down
1 change: 1 addition & 0 deletions langchain/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# CHANGELOG - LangChain
121 changes: 121 additions & 0 deletions langchain/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# LangChain

## Overview

Get cost estimation, prompt and completion sampling, error tracking, performance metrics, and more out of [LangChain][1] Python library requests using Datadog metrics, APM, and logs.

## Setup

### Installation

1. Enable APM and StatsD in your Datadog Agent. For example, in Docker:

```
docker run -d --cgroupns host \
--pid host \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /proc/:/host/proc/:ro \
-v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
-e DD_API_KEY=<DATADOG_API_KEY> \
-p 127.0.0.1:8126:8126/tcp \
-p 127.0.0.1:8125:8125/udp \
-e DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true \
-e DD_APM_ENABLED=true \
gcr.io/datadoghq/agent:latest
```

2. Install the Datadog APM Python library.

```
pip install ddtrace>=1.17
```


3. Prefix your LangChain Python application command with `ddtrace-run`.

```
DD_SERVICE="my-service" DD_ENV="staging" DD_API_KEY=<DATADOG_API_KEY> ddtrace-run python <your-app>.py
```

**Note**: If the Agent is using a non-default hostname or port, be sure to also set `DD_AGENT_HOST`, `DD_TRACE_AGENT_PORT`, or `DD_DOGSTATSD_PORT`.

See the [APM Python library documentation][2] for more advanced usage.


### Configuration

See the [APM Python library documentation][3] for all the available configuration options.


#### Log Prompt & Completion Sampling

To enable log prompt and completion sampling, set the `DD_LANGCHAIN_LOGS_ENABLED=1` environment variable. By default, 10% of traced requests will emit logs containing the prompts and completions.

To adjust the log sample rate, see the [APM library documentation][3].

**Note**: Logs submission requires `DD_API_KEY` to be specified when running `ddtrace-run`.


### Validation

Validate that the APM Python library can communicate with your Agent using:

```
ddtrace-run --info
```

You should see the following output:

```
Agent error: None
```

#### Debug Logging

Pass the `--debug` flag to `ddtrace-run` to enable debug logging.

```
ddtrace-run --debug
```

This displays any errors sending data:

```
ERROR:ddtrace.internal.writer.writer:failed to send, dropping 1 traces to intake at http://localhost:8126/v0.5/traces after 3 retries ([Errno 61] Connection refused)
WARNING:ddtrace.vendor.dogstatsd:Error submitting packet: [Errno 61] Connection refused, dropping the packet and closing the socket
DEBUG:ddtrace.contrib._trace_utils_llm.py:sent 2 logs to 'http-intake.logs.datadoghq.com'
```

## Data Collected

### Metrics

See [metadata.csv][4] for a list of metrics provided by this integration.

### Events

The LangChain integration does not include any events.

### Service Checks

The LangChain integration does not include any service checks.


## Troubleshooting

Need help? Contact [Datadog support][5].

## Further Reading

Additional helpful documentation, links, and articles:

- [Monitor your OpenAI usage with Datadog][6]
- [Monitor Azure OpenAI with Datadog][7]
Yun-Kim marked this conversation as resolved.
Show resolved Hide resolved

[1]: https://docs.langchain.com/docs/
[2]: https://ddtrace.readthedocs.io/en/stable/installation_quickstart.html
[3]: https://ddtrace.readthedocs.io/en/stable/integrations.html#langchain
[4]: https://github.com/DataDog/integrations-core/blob/master/langchain/metadata.csv
[5]: https://docs.datadoghq.com/help/
[6]: https://www.datadoghq.com/blog/monitor-openai-with-datadog/
[7]: https://www.datadoghq.com/blog/monitor-azure-openai-with-datadog/
Loading
Loading