Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Langchain - Vector search on the PDF using langchain abstraction #1893

Closed
srini047 opened this issue Jun 11, 2023 · 4 comments · Fixed by #2012
Closed

Langchain - Vector search on the PDF using langchain abstraction #1893

srini047 opened this issue Jun 11, 2023 · 4 comments · Fixed by #2012
Assignees
@srini047
Labels
enhancement New feature or request templates maintainer To be prioritize for notebook templates maintainer

Comments

@srini047
Copy link
Member

Describe the solution you'd like

  1. Load the PDF file.
  2. Split the document into smaller chunks.
  3. Embed each chunk using OpenAI's ada-002 model.
  4. Store the embeddings in an in-memory Faiss Vector DB.
  5. Ask questions about the PDF.

Additional context

cc @jravenel @FlorentLvr
@srini047 srini047 added enhancement New feature or request templates maintainer To be prioritize for notebook templates maintainer labels Jun 11, 2023
@srini047 srini047 self-assigned this Jun 11, 2023
@FlorentLvr
Copy link
Contributor

@srini047! That's good idea! Did you work on it already?

@jravenel
Copy link
Contributor

It's what Maxime did I think, with embedding

@srini047
Copy link
Member Author

@srini047! That's good idea! Did you work on it already?

@FlorentLvr I'm on it.

It's what Maxime did I think, with embedding

@Dr0p42 Can you share the link or references on how did you build it?

@jravenel
Copy link
Contributor

Maybe we should use weviate, open source for the win :)

@srini047 srini047 linked a pull request Jul 12, 2023 that will close this issue
Langchain vector search PDF #2012
Merged
@srini047 srini047 mentioned this issue Jul 12, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@srini047 srini047

Labels
enhancement New feature or request templates maintainer To be prioritize for notebook templates maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Langchain vector search PDF jupyter-naas/awesome-notebooks
3 participants
@jravenel @FlorentLvr @srini047