[not an issue] using it for my logbot!

[bakkot]

I just updated my matrix log bot, which lives on Github Pages, with full-text search! For example.

It works better than it has any right to. I can do a full text search across 240 MB of text just a hundred KB of network traffic. On a static host.

I still want to get tantivy working working, but SQL does quite well for the moment.

This project is a genuine marvel. Thanks for building it!

[phiresky]

awesome!

I see you have a pretty small page size set, I'd try increasing it - it'll increase the amount of fetched data, but it'll reduce the overall fetch time since the sequential request pattern means the request latency dominates the total request time.

tantivy may not improve it much the data fetched much for small databases (e.g. <1 GB) since the initial overhead of requests it has to do in any case is higher than in SQLite (that overhead could be removed with more work but I haven't done it), even if the FTS itself is much more efficient.

By the way (semi-related), Element uses tantivy for it's search engine, and the missing wasm+IndexedDB support of tantivy is the reason that Element search in encrypted messages doesn't work in the browser: matrix-org/seshat#84