Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement full text index #22

Open
proycon opened this issue Oct 14, 2023 · 1 comment
Open

Implement full text index #22

proycon opened this issue Oct 14, 2023 · 1 comment
Assignees
@proycon
Labels
enhancement New feature or request

Comments

@proycon
Copy link
Contributor

proycon commented Oct 14, 2023 •
edited
Loading

Searching for text currently iterates through the whole text. To make quicker lookups possible, a full text index could be implemented (e.g. using suffixarrays), at the cost of (significant) space. This would be an opt-in feature, via a Config parameter.
@proycon proycon added the enhancement New feature or request label Oct 14, 2023
@proycon proycon self-assigned this Oct 14, 2023
@proycon proycon mentioned this issue Oct 14, 2023
Open
@proycon
Copy link
Contributor Author

proycon commented May 17, 2024

tantivity would seem a good choice for such a feature. It's inspired by Lucene (but twice as fast), and has powerful capabilities. It can also handle offloading indices to disk and restrict the memory impact such a feature has. It could even be considered for #23 as well, especially where text is concerned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@proycon proycon

Labels
enhancement New feature or request
Projects
STAM: Stand-off Text Annotation Model
Status: Todo
Milestone
No milestone
Development

No branches or pull requests
1 participant
@proycon