Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cerebras): Adds Cerebras chat model integration #7457

Merged
merged 6 commits into from
Jan 2, 2025
Merged

feat(cerebras): Adds Cerebras chat model integration #7457

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
54c30d5
Adds Cerebras chat model integration
jacoblee93 Jan 2, 2025
3601a00
Fix lint, add to universal chat model
jacoblee93 Jan 2, 2025
143047b
Add non-streaming path
jacoblee93 Jan 2, 2025
353653e
Fix
jacoblee93 Jan 2, 2025
c030ba1
Remove log
jacoblee93 Jan 2, 2025
49bca95
Docstring and polish
jacoblee93 Jan 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
341 changes: 341 additions & 0 deletions docs/core_docs/docs/integrations/chat/cerebras.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,341 @@
{
"cells": [
{
"cell_type": "raw",
"id": "afaf8039",
"metadata": {
"vscode": {
"languageId": "raw"
}
},
"source": [
"---\n",
"sidebar_label: Cerebras\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "e49f1e0d",
"metadata": {},
"source": [
"# ChatCerebras\n",
"\n",
"[Cerebras](https://cerebras.ai/) is a model provider that serves open source models with an emphasis on speed. The Cerebras CS-3 system, powered by the the Wafer-Scale Engine-3 (WSE-3), represents a new class of AI supercomputer that sets the standard for generative AI training and inference with unparalleled performance and scalability.\n",
"\n",
"With Cerebras as your inference provider, you can:\n",
"\n",
"- Achieve unprecedented speed for AI inference workloads\n",
"- Build commercially with high throughput\n",
"- Effortlessly scale your AI workloads with our seamless clustering technology\n",
"\n",
"Our CS-3 systems can be quickly and easily clustered to create the largest AI supercomputers in the world, making it simple to place and run the largest models. Leading corporations, research institutions, and governments are already using Cerebras solutions to develop proprietary models and train popular open-source models.\n",
"\n",
"Want to experience the power of Cerebras? Check out our [website](https://cerebras.ai/) for more resources and explore options for accessing our technology through the Cerebras Cloud or on-premise deployments!\n",
"\n",
"For more information about Cerebras Cloud, visit [cloud.cerebras.ai](https://cloud.cerebras.ai/). Our API reference is available at [inference-docs.cerebras.ai](https://inference-docs.cerebras.ai).\n",
"\n",
"## Overview\n",
"\n",
"### Integration details\n",
"\n",
"| Class | Package | Local | Serializable | [PY support](https://python.langchain.com/docs/integrations/chat/cerebras) | Package downloads | Package latest |\n",
"| :--- | :--- | :---: | :---: | :---: | :---: | :---: |\n",
"| [ChatCerebras](https://api.js.langchain.com/classes/langchain_cerebras.ChatCerebras.html) | [`@langchain/cerebras`](https://www.npmjs.com/package/@langchain/cerebras) | ❌ | ❌ | ✅ | ![NPM - Downloads](https://img.shields.io/npm/dm/@langchain/cerebras?style=flat-square&label=%20&) | ![NPM - Version](https://img.shields.io/npm/v/@langchain/cerebras?style=flat-square&label=%20&) |\n",
"\n",
"### Model features\n",
"\n",
"See the links in the table headers below for guides on how to use specific features.\n",
"\n",
"| [Tool calling](/docs/how_to/tool_calling) | [Structured output](/docs/how_to/structured_output/) | JSON mode | [Image input](/docs/how_to/multimodal_inputs/) | Audio input | Video input | [Token-level streaming](/docs/how_to/chat_streaming/) | [Token usage](/docs/how_to/chat_token_usage_tracking/) | [Logprobs](/docs/how_to/logprobs/) |\n",
"| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n",
"| ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | \n",
"\n",
"## Setup\n",
"\n",
"To access ChatCerebras models you'll need to create a Cerebras account, get an API key, and install the `@langchain/cerebras` integration package.\n",
"\n",
"### Credentials\n",
"\n",
"Get an API Key from [cloud.cerebras.ai](https://cloud.cerebras.ai) and add it to your environment variables:\n",
"\n",
"```bash\n",
"export CEREBRAS_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"If you want to get automated tracing of your model calls you can also set your [LangSmith](https://docs.smith.langchain.com/) API key by uncommenting below:\n",
"\n",
"```bash\n",
"# export LANGCHAIN_TRACING_V2=\"true\"\n",
"# export LANGCHAIN_API_KEY=\"your-api-key\"\n",
"```\n",
"\n",
"### Installation\n",
"\n",
"The LangChain ChatCerebras integration lives in the `@langchain/cerebras` package:\n",
"\n",
"```{=mdx}\n",
"\n",
"import IntegrationInstallTooltip from \"@mdx_components/integration_install_tooltip.mdx\";\n",
"import Npm2Yarn from \"@theme/Npm2Yarn\";\n",
"\n",
"<IntegrationInstallTooltip></IntegrationInstallTooltip>\n",
"\n",
"<Npm2Yarn>\n",
" @langchain/cerebras @langchain/core\n",
"</Npm2Yarn>\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "a38cde65-254d-4219-a441-068766c0d4b5",
"metadata": {},
"source": [
"## Instantiation\n",
"\n",
"Now we can instantiate our model object and generate chat completions:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "cb09c344-1836-4e0c-acf8-11d13ac1dbae",
"metadata": {},
"outputs": [],
"source": [
"import { ChatCerebras } from \"@langchain/cerebras\" \n",
"\n",
"const llm = new ChatCerebras({\n",
" model: \"llama-3.3-70b\",\n",
" temperature: 0,\n",
" maxTokens: undefined,\n",
" maxRetries: 2,\n",
" // other params...\n",
"})"
]
},
{
"cell_type": "markdown",
"id": "2b4f3e15",
"metadata": {},
"source": [
"## Invocation"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "62e0dbc3",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"AIMessage {\n",
" \"id\": \"run-17c7d62d-67ac-4677-b33a-18298fc85e35\",\n",
" \"content\": \"J'adore la programmation.\",\n",
" \"additional_kwargs\": {},\n",
" \"response_metadata\": {\n",
" \"id\": \"chatcmpl-2d1e2de5-4239-46fb-af2a-6200d89d7dde\",\n",
" \"created\": 1735785598,\n",
" \"model\": \"llama-3.3-70b\",\n",
" \"system_fingerprint\": \"fp_2e2a2a083c\",\n",
" \"object\": \"chat.completion\",\n",
" \"time_info\": {\n",
" \"queue_time\": 0.00009063,\n",
" \"prompt_time\": 0.002163031,\n",
" \"completion_time\": 0.012339628,\n",
" \"total_time\": 0.01640915870666504,\n",
" \"created\": 1735785598\n",
" }\n",
" },\n",
" \"tool_calls\": [],\n",
" \"invalid_tool_calls\": [],\n",
" \"usage_metadata\": {\n",
" \"input_tokens\": 55,\n",
" \"output_tokens\": 9,\n",
" \"total_tokens\": 64\n",
" }\n",
"}\n"
]
}
],
"source": [
"const aiMsg = await llm.invoke([\n",
" {\n",
" role: \"system\",\n",
" content: \"You are a helpful assistant that translates English to French. Translate the user sentence.\",\n",
" },\n",
" { role: \"user\", content: \"I love programming.\" },\n",
"])\n",
"aiMsg"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "d86145b3-bfef-46e8-b227-4dda5c9c2705",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"J'adore la programmation.\n"
]
}
],
"source": [
"console.log(aiMsg.content)"
]
},
{
"cell_type": "markdown",
"id": "ce0414fe",
"metadata": {},
"source": [
"## Json invocation"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "3f0a7a2a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{ aiInvokeMsgContent: '{\"result\":4}', aiBindMsg: '{\"result\":4}' }\n"
]
}
],
"source": [
"const messages = [\n",
" {\n",
" role: \"system\",\n",
" content: \"You are a math tutor that handles math exercises and makes output in json in format { result: number }.\",\n",
" },\n",
" { role: \"user\", content: \"2 + 2\" },\n",
"];\n",
"\n",
"const aiInvokeMsg = await llm.invoke(messages, { response_format: { type: \"json_object\" } });\n",
"\n",
"// if you want not to pass response_format in every invoke, you can bind it to the instance\n",
"const llmWithResponseFormat = llm.bind({ response_format: { type: \"json_object\" } });\n",
"const aiBindMsg = await llmWithResponseFormat.invoke(messages);\n",
"\n",
"// they are the same\n",
"console.log({ aiInvokeMsgContent: aiInvokeMsg.content, aiBindMsg: aiBindMsg.content });"
]
},
{
"cell_type": "markdown",
"id": "18e2bfc0-7e78-4528-a73f-499ac150dca8",
"metadata": {},
"source": [
"## Chaining\n",
"\n",
"We can [chain](/docs/how_to/sequence/) our model with a prompt template like so:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "e197d1d7-a070-4c96-9f8a-a0e86d046e0b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"AIMessage {\n",
" \"id\": \"run-5c8a9f25-0f57-499b-9c2b-87bd07135feb\",\n",
" \"content\": \"Ich liebe das Programmieren.\",\n",
" \"additional_kwargs\": {},\n",
" \"response_metadata\": {\n",
" \"id\": \"chatcmpl-abd1e9eb-b873-492e-9e30-0d13dfc3a145\",\n",
" \"created\": 1735785607,\n",
" \"model\": \"llama-3.3-70b\",\n",
" \"system_fingerprint\": \"fp_2e2a2a083c\",\n",
" \"object\": \"chat.completion\",\n",
" \"time_info\": {\n",
" \"queue_time\": 0.00009499,\n",
" \"prompt_time\": 0.002095266,\n",
" \"completion_time\": 0.008807576,\n",
" \"total_time\": 0.012718439102172852,\n",
" \"created\": 1735785607\n",
" }\n",
" },\n",
" \"tool_calls\": [],\n",
" \"invalid_tool_calls\": [],\n",
" \"usage_metadata\": {\n",
" \"input_tokens\": 50,\n",
" \"output_tokens\": 7,\n",
" \"total_tokens\": 57\n",
" }\n",
"}\n"
]
}
],
"source": [
"import { ChatPromptTemplate } from \"@langchain/core/prompts\"\n",
"\n",
"const prompt = ChatPromptTemplate.fromMessages(\n",
" [\n",
" [\n",
" \"system\",\n",
" \"You are a helpful assistant that translates {input_language} to {output_language}.\",\n",
" ],\n",
" [\"human\", \"{input}\"],\n",
" ]\n",
")\n",
"\n",
"const chain = prompt.pipe(llm);\n",
"await chain.invoke(\n",
" {\n",
" input_language: \"English\",\n",
" output_language: \"German\",\n",
" input: \"I love programming.\",\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "3a5bb5ca-c3ae-4a58-be67-2cd18574b9a3",
"metadata": {},
"source": [
"## API reference\n",
"\n",
"For detailed documentation of all ChatCerebras features and configurations head to the API reference: https://api.js.langchain.com/classes/langchain_cerebras.ChatCerebras.html"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "TypeScript",
"language": "typescript",
"name": "tslab"
},
"language_info": {
"codemirror_mode": {
"mode": "typescript",
"name": "javascript",
"typescript": true
},
"file_extension": ".ts",
"mimetype": "text/typescript",
"name": "typescript",
"version": "3.7.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
5 changes: 5 additions & 0 deletions langchain/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,7 @@
"@jest/globals": "^29.5.0",
"@langchain/anthropic": "*",
"@langchain/aws": "*",
"@langchain/cerebras": "*",
"@langchain/cohere": "*",
"@langchain/core": "workspace:*",
"@langchain/google-genai": "*",
Expand Down Expand Up @@ -460,6 +461,7 @@
"peerDependencies": {
"@langchain/anthropic": "*",
"@langchain/aws": "*",
"@langchain/cerebras": "*",
"@langchain/cohere": "*",
"@langchain/core": ">=0.2.21 <0.4.0",
"@langchain/google-genai": "*",
Expand All @@ -480,6 +482,9 @@
"@langchain/aws": {
"optional": true
},
"@langchain/cerebras": {
"optional": true
},
"@langchain/cohere": {
"optional": true
},
Expand Down
5 changes: 5 additions & 0 deletions langchain/src/chat_models/universal.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ async function _initChatModelHelper(
const { ChatGroq } = await import("@langchain/groq");
return new ChatGroq({ model, ...passedParams });
}
case "cerebras": {
const { ChatCerebras } = await import("@langchain/cerebras");
return new ChatCerebras({ model, ...passedParams });
}
case "bedrock": {
const { ChatBedrockConverse } = await import("@langchain/aws");
return new ChatBedrockConverse({ model, ...passedParams });
Expand Down Expand Up @@ -598,6 +602,7 @@ export async function initChatModel<
* - mistralai (@langchain/mistralai)
* - groq (@langchain/groq)
* - ollama (@langchain/ollama)
* - cerebras (@langchain/cerebras)
* @param {string[] | "any"} [fields.configurableFields] - Which model parameters are configurable:
* - undefined: No configurable fields.
* - "any": All fields are configurable. (See Security Note in description)
Expand Down
Loading
Loading