Skip to content

Commit

Permalink
Start of search research docs
Browse files Browse the repository at this point in the history
The plan is to better understand how OS works so I can craft better
queries so I can have a better search functionality.
  • Loading branch information
ivankocienski committed Oct 23, 2024
1 parent 93356e5 commit dffd967
Show file tree
Hide file tree
Showing 2 changed files with 59 additions and 1 deletion.
57 changes: 57 additions & 0 deletions docs/partner-search-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Partner Search Notes

Very much a **WORK IN PROGRESS**.

This is getting fairly complex with a lot of "in air" ideas that I'd like to get down before cleaning up all the hacky code I have written.

Elastic Search and OpenSearch are the same thing. They can be run locally in a docker container (see `/docker/search`) but in production they are normally a service you connect to (by HTTPS). Written in Java.

The way these things work is they ingest all your data into indices and then you query them and get back all that data - so synchronising your DB with your index is a chore that must be done.

Searchkick is a gem that wraps OS and ES and makes them slightly less cumbersome but more Rails specific.

## Resources
- [OS docs](https://opensearch.org/docs/latest/getting-started/)
- [Cheat sheet](https://gist.github.com/ruanbekker/e8a09604b14f37e8d2f743a87b930f93)
- [SearchKick](https://github.com/ankane/searchkick)

## Operation

There are two parts to search: ingesting and search.

### Ingesting data

### Search

Very complicated (powerful) API that boils down to:

```ruby
partners = Partner.search(name_value, <options>)
# the SearchKick way
```

Where options is a DSL.

#### Finding documents by string

```
(but in `bool` field?)
multi-match
"analyzer": "searchkick_search"
"analyzer": "searchkick_search2"
(with)
boost set to 10 and 1
"fuzziness": not set and 1
"fuzzy_transpositions": true,
"prefix_length": 0,
"max_expansions": 3,
```

#### Filtering documents

#### Ordering output


3 changes: 2 additions & 1 deletion hacks/ingest-partners-into-index.rb
Original file line number Diff line number Diff line change
Expand Up @@ -181,4 +181,5 @@ def result
#
# where('name LIKE ?', "%#{name_pattern}%")
# }

partners_test/_search
{"query":{"bool":{"should":[{"dis_max":{"queries":[{"multi_match":{"query":"zulu alpha","boost":10,"operator":"and","analyzer":"searchkick_search","fields":["*.analyzed"],"type":"best_fields"}},{"multi_match":{"query":"zulu alpha","boost":10,"operator":"and","analyzer":"searchkick_search2","fields":["*.analyzed"],"type":"best_fields"}},{"multi_match":{"query":"zulu alpha","boost":1,"operator":"and","analyzer":"searchkick_search","fuzziness":1,"prefix_length":0,"max_expansions":3,"fuzzy_transpositions":true,"fields":["*.analyzed"],"type":"best_fields"}},{"multi_match":{"query":"zulu alpha","boost":1,"operator":"and","analyzer":"searchkick_search2","fuzziness":1,"prefix_length":0,"max_expansions":3,"fuzzy_transpositions":true,"fields":["*.analyzed"],"type":"best_fields"}}]}}]}},"timeout":"11000ms","_source":["id"],"size":10000}

0 comments on commit dffd967

Please sign in to comment.