Skip to content

Commit

Permalink
Integrate new filtering (#59)
Browse files Browse the repository at this point in the history
* Integrate filtering operators on nuclia.py

* better doc format

* better doc format

* Add unit test for filter parsing

* Add unit test for filter parsing

* Update and delete by resource slug

* Adapt changes
  • Loading branch information
lferran authored Feb 21, 2024
1 parent 3bf2788 commit 04ed358
Show file tree
Hide file tree
Showing 3 changed files with 45 additions and 10 deletions.
32 changes: 32 additions & 0 deletions docs/05-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,3 +54,35 @@ Based on a `find` request, Nuclia uses a generative AI to answer the question ba
search = sdk.NucliaSearch()
search.chat(query="My question")
```

## Filtering

Any endpoint that involves search (`search`, `find` and `chat`) also support more advanced filtering expressions. Expressions can have one of the following operators:

- `all`: this is the default. Will make search return results containing all specified filter labels.
- `any`: returns results containing at least one of the labels.
- `none`: returns results that do not contain any of the labels.
- `not_all`: returns results that do not contain all specified labels.

Note that multiple expressions can be chained in the `filters` parameter and the conjunction of all of them will be computed.

Here are some examples:

- CLI:

```bash
nuclia kb search find --query="My search" --filters="[{'any':['/icon/application/pdf','/icon/image/mp4']}]"
```

- SDK:

```python
from nuclia import sdk
from nucliadb_models.search import Filter

search = sdk.NucliaSearch()
search.chat(
query="My question",
filters=[Filter(any=['/classification.labels/region/Europe','/classification.labels/region/Asia'])],
)
```
1 change: 0 additions & 1 deletion nuclia/sdk/kbs.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ def list(self, account: Optional[str] = None):
if self._auth._config.kbs_token is not None
else []
)

return result
else:
matching_account = retrieve_account(
Expand Down
22 changes: 13 additions & 9 deletions nuclia/sdk/search.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

from nucliadb_models.search import (
ChatRequest,
Filter,
FindRequest,
KnowledgeboxFindResults,
Relations,
Expand Down Expand Up @@ -48,8 +49,8 @@ def search(
self,
*,
query: Union[str, SearchRequest] = "",
filters: Optional[List[str]] = None,
**kwargs
filters: Optional[Union[List[str], List[Filter]]] = None,
**kwargs,
):
"""
Perform a search query.
Expand All @@ -72,18 +73,21 @@ def find(
query: Union[str, FindRequest] = "",
highlight: Optional[bool] = False,
relations: Optional[bool] = False,
filters: Optional[List[str]] = None,
**kwargs
filters: Optional[Union[List[str], List[Filter]]] = None,
**kwargs,
):
"""
Perform a find query.
See https://docs.nuclia.dev/docs/api#tag/Search/operation/Find_Knowledge_Box_kb__kbid__find_post
"""

ndb: NucliaDBClient = kwargs["ndb"]
if isinstance(query, str) and highlight is not None:
req = FindRequest(query=query, highlight=highlight, filters=(filters or []))
req = FindRequest(
query=query,
highlight=highlight,
filters=filters or [],
)
elif isinstance(query, FindRequest):
req = query
else:
Expand All @@ -99,8 +103,8 @@ def chat(
self,
*,
query: Union[str, ChatRequest],
filters: Optional[List[str]] = None,
**kwargs
filters: Optional[Union[List[str], List[Filter]]] = None,
**kwargs,
):
"""
Answer a question.
Expand All @@ -109,7 +113,7 @@ def chat(
"""
ndb: NucliaDBClient = kwargs["ndb"]
if isinstance(query, str):
req = ChatRequest(query=query, filters=(filters or []))
req = ChatRequest(query=query, filters=filters or [])
else:
req = query
response = ndb.chat(req)
Expand Down

0 comments on commit 04ed358

Please sign in to comment.