Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce the InferenceChatModel for langchain #206429

Draft
wants to merge 43 commits into
base: main
Choose a base branch
from
Draft

Introduce the InferenceChatModel for langchain #206429

wants to merge 43 commits into from
+2,300 −82

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Jan 13, 2025 •
edited
Loading

Summary

Part of #206710

This PR introduces the InferenceChatModel class, which is a langchain chatModel utilizing the inference APIs (chatComplete) under the hood.

Creating instances of InferenceChatModel can either be done by manually importing the class from the new @kbn/inference-langchain package, or by using the new createChatModel API exposes from the inference plugin's start contract.

The main upside of using this chatModel is that the unification and normalization layers are already being taken care of by the inference plugin, making sure that the underlying models are being used with the exact same capabilities. More details on the upsides and reasoning in the associated issue.

Usage

Usage is very straightforward

const chatModel = await inferenceStart.getChatModel({
  request,
  connectorId: myInferenceConnectorId,
  chatModelOptions: {
    temperature: 0.2,
  },
});

// just use it as another langchain chatModel, e.g.
const response = await chatModel.stream('What is Kibana?');
for await (const chunk of response) {
     // do something with the chunk
}

Important

This PR is only adding the implementation, and not wiring it anywhere or using it in any existing code. This is meant to be done in a later stage. Merging that implementation first would allow to have distinct PR for the integration with search (playground) and security (assistant + other workflows)

@pgayvallet pgayvallet added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 backport:version Backport to applied version labels v8.18.0 labels Jan 13, 2025
@pgayvallet
Copy link
Contributor Author

/ci

@elasticmachine
Copy link
Contributor

🤖 Jobs for this PR can be triggered through checkboxes. 🚧

ℹ️ To trigger the CI, please tick the checkbox below 👇

  • Click to trigger kibana-pull-request for this PR!
  • Click to trigger kibana-deploy-project-from-pr for this PR!

@pgayvallet
Copy link
Contributor Author

/ci

2 similar comments
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet force-pushed the kbn-xxx-inference-langchain branch from 04575f1 to f2a2df5 Compare January 15, 2025 07:15
@pgayvallet
Copy link
Contributor Author

/ci

1 similar comment
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet mentioned this pull request Jan 15, 2025
Open
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet pgayvallet force-pushed the kbn-xxx-inference-langchain branch from 99c5439 to dad02ea Compare January 27, 2025 08:10
@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@pgayvallet
Copy link
Contributor Author

/ci

@elasticmachine
Copy link
Contributor

elasticmachine commented Jan 29, 2025 •
edited
Loading

💔 Build Failed

Failed CI Steps

History

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes v8.18.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pgayvallet @elasticmachine @kibanamachine