Add structural cue chunking based inspired by JinaAI's implementation#92

Open

AswanthManoj wants to merge 1 commit intoadithya-s-k:mainfrom AswanthManoj:main

+157

Commits on Aug 23, 2024

Added structural cue chunking strategy based on JinaAI's tokenizer chunking
AswanthManoj
committed