Skip to content

Update api specs llm config #73

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 117 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
117 commits
Select commit Hold shift + click to select a range
0b3b54f
Update introduction.mdx
richhuth May 21, 2024
b8fdcae
Update introduction.mdx
richhuth May 21, 2024
b513dab
Update introduction.mdx
richhuth May 21, 2024
dbb4dc3
Update introduction.mdx
richhuth May 21, 2024
a3b70ce
LangWatch Docs
richhuth May 21, 2024
982b253
Merged with previous langevals/docs to finish the documentation page
Eugenumber1 May 21, 2024
beedff6
Changed the default mode to light to follow the branding
Eugenumber1 May 22, 2024
840c765
Fixes all around the docs
rogeriochaves May 22, 2024
ba718aa
Smol change
rogeriochaves May 22, 2024
40355ce
remove highlights
richhuth May 23, 2024
05a22c8
Added the unit testing tutorial
Eugenumber1 May 24, 2024
c7ddeef
Add DSPy visualization docs
rogeriochaves May 29, 2024
0b59e8a
Add triggers
richhuth May 31, 2024
d8a5876
Add dataset to features
richhuth May 31, 2024
f020a12
Extended the unit testing tutorial
Eugenumber1 Jun 3, 2024
562921c
Added rag evaluation tutorial
Eugenumber1 Jun 3, 2024
e2d65fe
Reorganize docs and write the typescript one
rogeriochaves Jun 3, 2024
8a5fc96
Finish typescript guide
Eugenumber1 Jun 3, 2024
b43ed1c
noZoom on the badges
Eugenumber1 Jun 3, 2024
dda600e
Add docs on capturing exceptions
Eugenumber1 Jun 4, 2024
0d1b2be
Add azure example on the typescript guide as well
Eugenumber1 Jun 4, 2024
0531688
Some updates on the docs, put github repo on top
Eugenumber1 Jun 4, 2024
5d76d7b
Add capturing the usage on the docs examples
Eugenumber1 Jun 6, 2024
d7bf829
Add better explanation on why capturing the usage
Eugenumber1 Jun 6, 2024
e46fc4b
Fixed many broken links
Eugenumber1 Jun 7, 2024
8ceac8a
Fix integration links
Eugenumber1 Jun 7, 2024
1b0bcdc
annotations
richhuth Jun 7, 2024
8b27ead
Merge branch 'main' of https://github.com/langwatch/docs
richhuth Jun 7, 2024
fc47318
Update links
richhuth Jun 7, 2024
1c9afd5
Added the links to the API of each evaluator from the Evaluators page
Eugenumber1 Jun 7, 2024
f737e1c
Added the DSPy RAG tutorial
Eugenumber1 Jun 10, 2024
e249c48
Added some fixes to the page
Eugenumber1 Jun 10, 2024
76ec347
added notebook reference at the Extensive Unit Testing
Eugenumber1 Jun 12, 2024
59d3355
Update docs for outputs to no longer be a list for typescript and REST
Eugenumber1 Jun 12, 2024
58e5b1b
Add annotations API calls
richhuth Jun 12, 2024
d90a74d
Added notebooks to DSPy vis and to RAG eval
Eugenumber1 Jun 13, 2024
1de2466
Update docs for new python sdk
Eugenumber1 Jun 14, 2024
6907c3d
Improve introduction and overview docs
Eugenumber1 Jun 15, 2024
d4bb682
Small reorgs and fixes, add collab to dspy rag example and move conce…
Eugenumber1 Jun 15, 2024
c2591f8
Add reference links
Eugenumber1 Jun 15, 2024
de1dbca
Add LiteLLM integration docs
Eugenumber1 Jun 17, 2024
4caf4d1
Updated the naming of the pages to stay consistent with Xxxxx Yyyyyy …
Eugenumber1 Jun 14, 2024
e860564
Added Pipeline Eval tutorial
Eugenumber1 Jun 19, 2024
1ba684b
Add langchain.js integration guide
Eugenumber1 Jun 19, 2024
b4eb0b2
Add self-hosting guide
Eugenumber1 Jun 22, 2024
8ce507d
Oops, our us
Eugenumber1 Jun 22, 2024
f508c94
More infra details and how to point to your instance on self-hosting …
Eugenumber1 Jun 23, 2024
cd2c887
We have migrated from mysql to postgres, update the self-hosting guid…
Eugenumber1 Jun 24, 2024
4836cab
Postgres not mysql
Eugenumber1 Jun 24, 2024
5e952a7
Add flowise guide
Eugenumber1 Jun 24, 2024
9d51922
Fix docs on how to capture metadata
Eugenumber1 Jul 1, 2024
ce6727e
Update flowise screenshots with the right link
Eugenumber1 Jul 1, 2024
80380df
updated openapi, added haystack and prompt injection
Eugenumber1 Jul 5, 2024
4dbf668
Change NEXT_PUBLIC_AUTH_PROVIDER to NEXTAUTH_PROVIDER as new version …
Eugenumber1 Jul 6, 2024
9375b47
Add Traces API
richhuth Jul 16, 2024
95a21b2
Add annotation Scores
richhuth Jul 16, 2024
6ed9a9a
Add Scores and Evals
richhuth Jul 19, 2024
3abc962
Add trace details to api
richhuth Jul 19, 2024
f3ab641
add api
richhuth Jul 19, 2024
75d2786
Remove empty space
Eugenumber1 Jul 19, 2024
32b670c
Add the /api prefix on all endpoints
Eugenumber1 Jul 19, 2024
8fb7c7d
Fix openai spec for the new version of mintlify
Eugenumber1 Jul 19, 2024
17044fd
Bump
Eugenumber1 Jul 19, 2024
41ae00a
Add docs on DSPy integration
Eugenumber1 Jul 19, 2024
56631a3
Update openapi.json
richhuth Jul 22, 2024
c4faf2d
Change name
richhuth Jul 23, 2024
6c479ec
Langflow integration docs
Eugenumber1 Jul 25, 2024
482074b
Add custom events screenshot and update langflow screenshot
Eugenumber1 Jul 31, 2024
6e6c527
Added how to choose your evaluator guyide together with some photos
Eugenumber1 Jul 31, 2024
a67ecfc
check if the images are properly deployed now
Eugenumber1 Jul 31, 2024
19df825
final commit after properly adding the pictures
Eugenumber1 Jul 31, 2024
af1ea9b
add unshare
richhuth Aug 6, 2024
165b376
Add OpenTelemetry docs
Eugenumber1 Aug 7, 2024
b34ecc9
Add a bunch more open telemetry integration code snippets!
Eugenumber1 Aug 7, 2024
3e07121
Update labels docs
Eugenumber1 Aug 7, 2024
d89eb04
Add overview docs for evaluation and rest api docs for custom evaluat…
Eugenumber1 Aug 11, 2024
0582445
Add python custom evaluation tracing docs
Eugenumber1 Aug 11, 2024
47ab972
Revert "Add python custom evaluation tracing docs"
Eugenumber1 Aug 11, 2024
8d1de8d
Revert "Revert "Add python custom evaluation tracing docs""
Eugenumber1 Aug 11, 2024
1751a50
add cookbooks
richhuth Aug 16, 2024
3ef8ceb
Go Live
Eugenumber1 Aug 19, 2024
a383333
add cookbook
Eugenumber1 Aug 20, 2024
7f1f585
Replace self-hosting page
Eugenumber1 Aug 21, 2024
cad6965
Update evaluations on code and guardrails guides
Eugenumber1 Aug 24, 2024
b96fedb
Update docs for typescript guardrails as well
Eugenumber1 Aug 29, 2024
4fc0cde
Fixed flowise doc links
Eugenumber1 Sep 16, 2024
ea96f8a
Add guide on changing the main input and output
Eugenumber1 Sep 17, 2024
236e42a
onprem messag
Eugenumber1 Sep 24, 2024
293eec8
Oops, forgot an r
Eugenumber1 Oct 7, 2024
2c62823
Add search traces docs
Eugenumber1 Oct 11, 2024
c460d58
Move /api/collector down and rename to same env vars as python/typesc…
Eugenumber1 Nov 16, 2024
8d6ddde
Remove Batch Evaluations
Eugenumber1 Nov 21, 2024
dcb6135
Add optimization studio tutorials
Eugenumber1 Nov 27, 2024
1bf93b6
Add vercel ai sdk integration docs
Eugenumber1 Dec 15, 2024
9987825
Update docs to include nodejs ai sdk setup
Eugenumber1 Dec 27, 2024
45e6059
Make API key more explicit on the ai sdk instructions
Eugenumber1 Dec 29, 2024
303e15d
Search api now uses scrollId
Eugenumber1 Jan 17, 2025
08e9fcf
Search api now uses scrollId
Eugenumber1 Jan 17, 2025
4b97f48
Add scores and queues
Eugenumber1 Feb 13, 2025
723d293
Add link to status page
Eugenumber1 Feb 22, 2025
5257f44
Add guide on capturing additional langflow metadata
Eugenumber1 Feb 26, 2025
da4d62a
Add guide on capturing additional langflow metadata
Eugenumber1 Feb 26, 2025
270bc86
Add llms.txt protip
Eugenumber1 Mar 2, 2025
5b13815
Testing
Eugenumber1 Mar 2, 2025
3d9c447
test2
Eugenumber1 Mar 2, 2025
98d32a3
test3
Eugenumber1 Mar 2, 2025
a7c715d
Add # LangWatch title to llms.txt for better machine readability
Eugenumber1 Mar 2, 2025
5338259
Added open inference bedrock example
0xdeafcafe Mar 24, 2025
6ed81e6
Merge pull request #1 from langwatch/docs/add-openinf-bedrock
rogeriochaves Mar 24, 2025
93e69c7
Regenerate llms.txt to include bedrock
Eugenumber1 Mar 24, 2025
0da3c25
added section on instrumenting Microsoft Autogen
0xdeafcafe Mar 31, 2025
2418932
change ordering
0xdeafcafe Mar 31, 2025
2d8d0df
Add dataset entries docs
Eugenumber1 Apr 8, 2025
6988a94
Merge pull request #2 from langwatch/feat/add-openinf-autogen
rogeriochaves Apr 8, 2025
8853c66
Regenerate llms.txt to include autogen
Eugenumber1 Apr 8, 2025
72f253a
Add mcp docs
Eugenumber1 Apr 9, 2025
6047761
Add mcp docs
Eugenumber1 Apr 9, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"[mdx]": {
"editor.formatOnSave": false
}
}
20 changes: 5 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,22 @@
# Mintlify Starter Kit
# LangWatch and LangEvals Documentation

Click on `Use this template` to copy the Mintlify starter kit. The starter kit contains examples including

- Guide pages
- Navigation
- Customizations
- API Reference pages
- Use of popular components
This is the documentation repository for the [LangWatch](https://github.com/langwatch/langwatch) and [LangEvals](https://github.com/langwatch/langevals) projects.

### Development

Install the [Mintlify CLI](https://www.npmjs.com/package/mintlify) to preview the documentation changes locally. To install, use the following command
Install the [Mintlify CLI](https://www.npmjs.com/package/mintlify) to preview the documentation changes locally. To install, use the following command:

```
npm i -g mintlify
```

Run the following command at the root of your documentation (where mint.json is)
Run the following command:

```
mintlify dev
```

### Publishing Changes

Install our Github App to auto propagate changes from your repo to your deployment. Changes will be deployed to production automatically after pushing to the default branch. Find the link to install on your dashboard.

#### Troubleshooting
Install our Github App to auto propagate changes from your repo to your deployment. Changes will be deployed to production automatically after pushing to the default branch. Find the link to install on your dashboard.

- Mintlify dev isn't running - Run `mintlify install` it'll re-install dependencies.
- Page loads as a 404 - Make sure you are running in a folder with `mint.json`
4 changes: 4 additions & 0 deletions api-reference/annotations/create-annotation-trace.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: 'Create annotation for single trace'
openapi: 'POST /api/annotations/trace/{id}'
---
5 changes: 5 additions & 0 deletions api-reference/annotations/delete-annotation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: 'Delete single annotation'
openapi: 'DELETE /api/annotations/{id}'
---

4 changes: 4 additions & 0 deletions api-reference/annotations/get-all-annotations-trace.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: 'Get annotationa for single trace'
openapi: 'GET /api/annotations/trace/{id}'
---
4 changes: 4 additions & 0 deletions api-reference/annotations/get-annotation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: 'Get annotations'
openapi: 'GET /api/annotations'
---
4 changes: 4 additions & 0 deletions api-reference/annotations/get-single-annotation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: 'Get single annotation'
openapi: 'GET /api/annotations/{id}'
---
26 changes: 26 additions & 0 deletions api-reference/annotations/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: 'Overview'
description: 'Annotations are used to annotate traces with additional information'
---

## Intro

With the Annotations API, you can annotate traces with additional information. This is useful if you want to add additional information to a trace, such as a comment or a thumbs up/down reaction.

## Authentication

To make a call to the Annotations API, you will need to pass through your LangWatch API key in the header as `X-Auth-Token`. Your API key can be found on the setup page under settings.


#### Allowed Methods

- `GET /api/annotations` - Get a list of annotations
- `GET /api/annotations/:id` - Get a single annotation
- `DELETE /api/annotations/:id` - Delete a single annotation
- `PATCH /api/annotations/:id` - Update a single annotation
- `GET /api/annotations/trace/:id` - Get the annotations for a single trace
- `POST /api/annotations/trace/:id` - Create an annotation for a single trace




5 changes: 5 additions & 0 deletions api-reference/annotations/patch-annotation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: 'Patch single annotation'
openapi: 'PATCH /api/annotations/{id}'
---

4 changes: 4 additions & 0 deletions api-reference/datasets/post-dataset-entries.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
title: 'Add entries to a dataset'
openapi: 'POST /api/dataset/{slug}/entries'
---
4 changes: 0 additions & 4 deletions api-reference/endpoint/get.mdx

This file was deleted.

195 changes: 0 additions & 195 deletions api-reference/openapi.json

This file was deleted.

Loading