Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System messages don't work with Anthropic's Claude. #7807

Open
raffareis opened this issue Mar 6, 2025 · 1 comment · May be fixed by #7808
Open

System messages don't work with Anthropic's Claude. #7807

raffareis opened this issue Mar 6, 2025 · 1 comment · May be fixed by #7808
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature

Comments

@raffareis
Copy link

When using Zep Memory this creates a message with role="system".

langchainjs/libs/langchain-community/src/memory/zep.ts

Line 148 in e376af8

? [new SystemMessage(memory.summary.content)]

However Claude doesn't support this:

Image

System messages are only permitted as the first passed message.

Error: System messages are only permitted as the first passed message.
    at /usr/src/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]_e6adc3a3fb2c7552f96b3ce4c6b5c0f9/node_modules/@langchain/anthropic/dist/utils/message_inputs.cjs:200:19
    at Array.map (<anonymous>)
    at _convertMessagesToAnthropicPayload (/usr/src/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]_e6adc3a3fb2c7552f96b3ce4c6b5c0f9/node_modules/@langchain/anthropic/dist/utils/message_inputs.cjs:188:52)
    at ChatAnthropic._streamResponseChunks (/usr/src/node_modules/.pnpm/@[email protected]_@[email protected][email protected][email protected][email protected]_e6adc3a3fb2c7552f96b3ce4c6b5c0f9/node_modules/@langchain/anthropic/dist/chat_models.cjs:715:94)
    at _streamResponseChunks.next (<anonymous>)
    at ChatAnthropic._streamIterator (/usr/src/node_modules/.pnpm/@[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_/node_modules/@langchain/core/dist/language_models/chat_models.cjs:100:34)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ChatAnthropic.transform (/usr/src/node_modules/.pnpm/@[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_/node_modules/@langchain/core/dist/runnables/base.cjs:402:9)
    at async RunnableBinding.transform (/usr/src/node_modules/.pnpm/@[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_/node_modules/@langchain/core/dist/runnables/base.cjs:912:9)
    at async ToolCallingAgentOutputParser.transform (/usr/src/node_modules/.pnpm/@[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_/node_modules/@langchain/core/dist/runnables/base.cjs:391:26)
    at async RunnableSequence._streamIterator (/usr/src/node_modules/.pnpm/@[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_/node_modules/@langchain/core/dist/runnables/base.cjs:1349:30)

Claude doesn't have a role="system" message, instead it has a "system" base parameter on its API:

Image

https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/system-prompts

https://docs.anthropic.com/en/api/messages#body-messages
@dosubot DosuBot
Copy link

dosubot bot commented Mar 6, 2025

Hey there, @raffareis! I'm here to help you with any bugs, questions, or contributions you might have. Let's tackle this issue together!

To address the issue with integrating Zep Memory and Anthropic's Claude API, you can use the zepMemoryContextToSystemPrompt function to extract the system prompt from Zep memory and pass it as the top-level "system" parameter required by Claude. This avoids using a SystemMessage with a role="system", which Claude does not support.

Here's a brief guide on how to implement this:

  1. Extract System Prompt: Use the zepMemoryContextToSystemPrompt function to get the system prompt from Zep memory.

  2. Condense Memory: Use the condenseZepMemoryIntoHumanMessage function to condense the Zep memory into a single human message.

  3. Integrate with Claude: Pass the extracted system prompt as the top-level "system" parameter and the condensed human message as the input message to Claude's API.

Here's a code snippet to illustrate this:

import { Memory } from "@getzep/zep-cloud";
import { condenseZepMemoryIntoHumanMessage, zepMemoryContextToSystemPrompt } from "./path_to_your_functions";

// Assume `memory` is an instance of Zep Memory
const memory: Memory = /* your Zep memory instance */;

// Convert Zep memory into a human message
const humanMessage = condenseZepMemoryIntoHumanMessage(memory);

// Use the system prompt and human message with Claude API
const systemPrompt = zepMemoryContextToSystemPrompt(memory);

// Pass `systemPrompt` as the top-level "system" parameter
// and `humanMessage` as the input message to Claude API

This approach ensures compatibility with Claude's API by using the top-level "system" parameter instead of a SystemMessage [1].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label Mar 6, 2025
raffareis added a commit to xmacna/langchainjs that referenced this issue Mar 6, 2025
@raffareis
fix(memory): add Claude-compatible mode to ZepMemory
aebe041 
Add separateMessages option to ZepMemory class to address Claude API
limitation with system messages. When set to false, all memory content
is condensed into a single HumanMessage instead of using SystemMessage.
Default is true for backward compatibility.
Fixes langchain-ai#7807
@raffareis raffareis linked a pull request Mar 6, 2025 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(memory): add Claude-compatible mode to ZepMemory xmacna/langchainjs
1 participant
@raffareis