Searching and Filtering in the API

[TheMcs000]

I would suggest that we will have a parameter for each attribute. Furthermore I would have multiple parameters for each attribute (I saw a page with an idea i really liked): You could search in different ways: "contains", "does not contain", "begins with", "ends with", "equals", "does not equal". For numbers: "greater than", "lower than". And i really liked that idea. In the UI you could nicely implement this by having a dropdown next to the search input. The GET Parameter could look something like this: title_contains=someTitle or title_beginswith=something

---

@codello says: I do not currently have a good concept of how to implement searching on an API level. I almost feel like we would need some kind of (custom) querying language to accommodate all cases for searching. Maybe we should move this to it's own issue to develop a good idea of what criteria should be searchable by what operations.

---

• API Specification: Song Queries #40
• Backend Implementation
• Frontend Implementation

[codello]

Edit: After some thought and some discussion with @kenowi-dev I have updated this comment to an actual proposal.

---

What feature do you want to propose?

This is a proposal for an API for searching songs. While this proposal is only concerned with searching songs, the API is easily extensible to other searches as well. The API takes heavy inspiration from the syntax used by the GitHub Issue Search.

Why does Karman need this feature?

Searching is one of the core features for any library management application. Obviously this is not the most Simple API for this purpose. There are however some major advantages to a design like this (see Alternatives Considered), especially when it comes to usability and extensibility.

How do users access the feature?

In the first instance this is an API feature. It would be accessed via endpoints that list songs such as ˋ /v1/songsˋ or ˋ/v1/uploads/{uuid}/songsˋ. From the external perspective the feature is available via a GET parameter (ˋqueryˋ) in which a search query in a custom format is passed. The format is described below.

This search query format might be passed on to the UI to allow users to craft their own searches. This is not part of this proposal.

Describe alternatives you've considered

• Searching via specialized parameters (such as ˋtitle_containsˋ, ˋyear_ltˋ, etc.) would be a little simpler to implement. However I believe the implementation overhead of this proposal is not very big and the advantages are significant. Most notable it provides an intuitive way to support custom fields in the search whereas specialized parameters can be a little more difficult to provide for the end user.
• Limited search functionality: We could forego advanced searches completely and limit searches to specific fields and specific operations. While this would probably make for a simplified documentation it significantly reduces the usability of the app as a library management software.

Additional context

The following paragraphs describe the format of a query.

A query is divided into space-separated fields. Each of these fields can have one of two formats:

• A simple string in the format value or "value".
• A filter specifier in the format key:value or key:"value", optionally prefixed by a qualifier such as (-)

If any value contains spaces the quoted format must be used. All values are case insensitive.

Each field corresponds to a search/filter clause, all of which are ANDed:

• A filter clause in the format key:value or key:"value" restricts results to songs that match the filter. For example a filter title:Hello would restrict results to songs that contain the string hello in their title. Similarly for other fields like artist, comment, and potentially custom fields.
• Numeric types can support additional filters, such as "greater than". To find songs published before the year 2000 one might use year:<2000. To find songs from the 90s year:1990..1999 could be used.
• A negated filter clause in the format -key:value or -key:"value" restricts results to songs that do not match that filter. For example -artist:Foo would only return songs that do not contain the substring foo in their artist.
• In the future other filter specifiers might be used, such as a ~ prefix for fuzzy matching.
• A simple string clause adds a word to the search words. For example foo "hello world" adds the two search words foo and hello world. Results will be restricted to songs that contain each of these search words in one of their fields (like title, artist, lyrics, …)
• There can be special filter clauses that do not correspond to fields. For example something like goldennotes:true could be used to find songs that have golden notes.
• String searching could work similar to the GitHub search in that we can restrict the fields that are used for the search. For example hello in:title,artitst would find songs that have hello in the title or the artist field. We could also support the NOT operator to restrict searches to results that do not contain a specific string field.

As an example consider the search query title:Hello -artist:Foo Somebody NOT once in:lyrics. This search would find songs that

• Contain Hello in their title, and
• Do not contain Foo in their artists, and
• Do contain the word somebody in their lyrics, and
• Do not contain the word once in their lyrics.

Implementation Considerations

On the backend I imagine the implementation pretty straightforward: Every clause basically results in an additional WHERE clause. Complex searches would of course result in complex queries but that's more because of the nature of the query than because of the query syntax. If necessary optimizations for certain queries can easily performed on the DB level or by detecting special filter clauses. Also not all combinations of filters have to be supported by the backend.

A potential parser for this kind of search format should probably be pretty lenient. Instead of raising errors when the query is not correctly formatted we could probably achieve a better experience by just returning an empty result or by ignoring parts of the query.