Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

User-Provided Text Splitters for Document Chunking #1615

Open
Pipboyguy opened this issue Jul 19, 2024 · 0 comments
Open

User-Provided Text Splitters for Document Chunking #1615

Pipboyguy opened this issue Jul 19, 2024 · 0 comments
Assignees
@Pipboyguy

Comments

@Pipboyguy
Copy link
Collaborator

Pipboyguy commented Jul 19, 2024 •
edited
Loading

Feature description

Implement support for user-provided text splitters in dlt for flexible document chunking.

Are you a dlt user?

Yes

Use case

Users need custom control over document chunking for various document formats.

Proposed solution

Add an option to specify a custom text splitter callback when configuring the resource.

Related issues

#1587
@Pipboyguy Pipboyguy self-assigned this Jul 19, 2024
@Pipboyguy Pipboyguy moved this from Todo to In Progress in dlt core library Jul 19, 2024
@Pipboyguy Pipboyguy changed the title Document Splitters User-Provided Text Splitters for Document Chunking Jul 19, 2024
This was referenced Jul 21, 2024
Closed
Merged
@HectorBst HectorBst mentioned this issue Dec 22, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Pipboyguy Pipboyguy

Labels
None yet
Projects
dlt core library
Status: Todo
Milestone
No milestone
Development

When branches are created from issues, their pull requests are automatically linked.

1615-document-splitters dlt-hub/dlt
1 participant
@Pipboyguy