-
Notifications
You must be signed in to change notification settings - Fork 311
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Opik-703] Add guardrails integration docs (#1086)
* Add documentation page to python-sdk-docs * Update index.rst * Add guardrails example, make the integration work properly in jupyter notebooks without asyncio * Add markdown file for guardrails * Update docs * Delete the markdown file * Update overview.md * Update README.md * Add guardrails-ai.md * Update guardrails-ai.md * Update sidebars * Fix lint errors * Updae guardrails hub install command * Update sidebars * Update guardrails-ai.md * Fix typo in guardrails-ai.md * Add guardrails api key to codeblock tests * skip guardrails code block * add nltk installation to BLEU metric documentation code block * Fix lint errors * Fix typo
- Loading branch information
1 parent
0619865
commit 2319663
Showing
15 changed files
with
334 additions
and
66 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
155 changes: 155 additions & 0 deletions
155
apps/opik-documentation/documentation/docs/cookbook/guardrails-ai.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,155 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "---\n", | ||
| "description: Cookbook that showcases Opik's integration with the Guardrails AI Python SDK\n", | ||
| "---\n", | ||
| "\n", | ||
| "# Using Opik with Guardrails AI\n", | ||
| "\n", | ||
| "[Guardrails AI](https://github.com/guardrails-ai/guardrails) is a framework for validating the inputs and outputs \n", | ||
| "\n", | ||
| "For this guide we will use a simple example that logs guardrails validation steps as traces to Opik, providing them with the validation result tags.\n", | ||
| "\n", | ||
| "## Creating an account on Comet.com\n", | ||
| "\n", | ||
| "[Comet](https://www.comet.com/site?from=llm&utm_source=opik&utm_medium=colab&utm_content=openai&utm_campaign=opik) provides a hosted version of the Opik platform, [simply create an account](https://www.comet.com/signup?from=llm&utm_source=opik&utm_medium=colab&utm_content=openai&utm_campaign=opik) and grab you API Key.\n", | ||
| "\n", | ||
| "> You can also run the Opik platform locally, see the [installation guide](https://www.comet.com/docs/opik/self-host/overview/?from=llm&utm_source=opik&utm_medium=colab&utm_content=openai&utm_campaign=opik) for more information." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "%pip install --upgrade opik guardrails-ai" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import opik\n", | ||
| "\n", | ||
| "opik.configure(use_local=False)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Preparing our environment\n", | ||
| "\n", | ||
| "In order to use Guardrails AI, we will configure the OpenAI API Key, if you are using any other providers you can replace this with the required API key:" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 2, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "import os\n", | ||
| "import getpass\n", | ||
| "\n", | ||
| "if \"OPENAI_API_KEY\" not in os.environ:\n", | ||
| " os.environ[\"OPENAI_API_KEY\"] = getpass.getpass(\"Enter your OpenAI API key: \")" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "We will also need to install the guardrails check for politeness from the Guardrails Hub" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": null, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "!guardrails hub install hub://guardrails/politeness_check" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "## Logging validation traces\n", | ||
| "\n", | ||
| "In order to log traces to Opik, you will need to call the track the Guard object with `track_guardrails` function." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 6, | ||
| "metadata": {}, | ||
| "outputs": [], | ||
| "source": [ | ||
| "from guardrails import Guard, OnFailAction\n", | ||
| "from guardrails.hub import PolitenessCheck\n", | ||
| "\n", | ||
| "from opik.integrations.guardrails import track_guardrails\n", | ||
| "\n", | ||
| "politeness_check = PolitenessCheck(\n", | ||
| " llm_callable=\"gpt-3.5-turbo\", on_fail=OnFailAction.NOOP\n", | ||
| ")\n", | ||
| "\n", | ||
| "guard: Guard = Guard().use_many(politeness_check)\n", | ||
| "guard = track_guardrails(guard, project_name=\"guardrails-integration-example\")\n", | ||
| "\n", | ||
| "guard.validate(\n", | ||
| " \"Would you be so kind to pass me a cup of tea?\",\n", | ||
| ")\n", | ||
| "guard.validate(\n", | ||
| " \"Shut your mouth up and give me the tea.\",\n", | ||
| ");" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Every validation will now be logged to Opik as a trace" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "The trace will now be viewable in the Opik platform:\n", | ||
| "\n", | ||
| "" | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3 (ipykernel)", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.10.12" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 4 | ||
| } |
Oops, something went wrong.