Is it possible to search for partial terms?

[Mistralys]

As documented, Loupe uses Prefix searching. I wonder if there is a way to force Loupe to do a partial term search?

Example: Finding an entry titled AeonOfTime by searching for Time.

With the default Loupe settings, I can find that entry searching for Aeon, but not Time.

[Toflar]

No it's not possible. As you stated, Loupe uses the prefixed search approach just like MeiliSearch and potentially others.

[Toflar]

Sure 😊 Just for clarification: I'm not saying this is not a valid use case. Quite the contrary, it very much is! Your example sounds like a movie title or the title of a game so it could be fictional (forgive me if that was a wrong guess) but it does happen in natural language with "compound words". For example "railroad", "sunflower" or "blueberry".
In English it's not much of a problem because there aren't many long compound words like that (or I can't think of any). There are of course compound words that are "open" like "living room furniture" or whatever but those are not a problem for us in this case 😊 However, in other languages, especially German which is known for its very long compound words, this can be problematic (like "Rindfleischetikettierungsüberwachungsaufgabenübertragungsgesetz" - yes, that's one word).

But the thing is, this becomes complex very, very quickly and you need NLP (Natural Language Processing) tools that are able to split those compound words into their individual parts. And then there's other areas in language processing like words that when used together, cause us humans think of something very specific, like "pink panther". For the search engine it's just "pink" and "panther" but for us, those two words belong together, right 😊 That's where AI, embeddings and vector search can help etc.

In other words: If you have that requirement, then there are solutions. But they are highly likely not built with just PHP and SQLite 😉

[Mistralys]

Thanks for the detailed explanation 😊

Yes, it's exactly that kind of compound word I was wondering about. Also, I like your german example. Not something you'll say out loud very often unless you have big lungs and a nimble tongue at least!

My initial assumption was that Loupe's search query would allow something like a MySQL LIKE '%Time' query, but I think I understand a bit better what Loupe does, and why. It's fine, really - for my purposes it works very well 👍

[Toflar]

Ah, yes I missed the most important part in my explanation why this does not work. LIKE '%Time' already shows the problem: You cannot use an index and things become slow. The advantage of a prefix search is that you can search with e.g. only 3 letters and find the relevant results very quickly. You can type sun and find sunflower (or sum depending on typo tolerance settings). The disadvantage is that you cannot find sunflower when you type flower. So it's just a matter of what your requirements are.
You can always try to improve the results by splitting those words yourself and index them separately.

[Mistralys]

Ah, now I fully understand what you meant by having to use NLP tools. I think some basic word splitting when I build the index could be a good proof of concept.

Awesome, I will do some more research into what I could do with it. Thanks again! 😊

[Mistralys]

A small addendum: In my project, I solved the issue by adding a dedicated searchTerms field, in which it is possible to add any terms that may not be found by the Loupe search. Since the data is curated by hand to begin with, this is only a small overhead and solves the problem without resorting to automated word splitting.

For example, in the record I mentioned that contains AeonOfTime, I added time to the search terms. For my purposes, this works really well :D