Skip to content

fix(trimmer): trim last message when allowPartial: true and strategy: first #8287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

schonmann
Copy link

@schonmann schonmann commented May 30, 2025

Hello team! 👋

While consuming the trimMessages abstraction, I've noticed that for the following example:

const messages = [
  new SystemMessage(ENTITY_SUMMARIZER_PROMPT),
  new HumanMessage(
    `Please analyze this entity data: ${JSON.stringify(entity)}`
  ),
];

const trimmedMessages = await trimMessages({
  maxTokens: 128_000 * 0.9,
  strategy: 'first',
  allowPartial: true,
  textSplitter,
  tokenCounter,
}).invoke(messages, config);

The trimmedMessages would always strip away the HumanMessage completely – even though allowPartial is true, and we provided a textSplitter which splits text into small chunks.

Looking at the _firstMaxTokens code plus the equivalent Python algorithm, I saw a discrepancy in the way we're iterating. This PR fixes the issue by just navigating the entire array (including the last message).

@dosubot dosubot bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label May 30, 2025
Copy link

vercel bot commented May 30, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ✅ Ready (Inspect) Visit Preview Jun 22, 2025 1:22am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Jun 22, 2025 1:22am

@dosubot dosubot bot added the auto:bug Related to a bug, vulnerability, unexpected error with an existing feature label May 30, 2025
@schonmann schonmann changed the title fix(trimmer): consider last message on partial trim fix(trimmer): trim last message when allowPartial: true and strategy: first May 30, 2025
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels May 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto:bug Related to a bug, vulnerability, unexpected error with an existing feature size:M This PR changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant