Skip to content
This repository has been archived by the owner on Apr 20, 2020. It is now read-only.

Use standard benchmarking for results #4

Open
henrywallace opened this issue Jun 20, 2018 · 1 comment
Open

Use standard benchmarking for results #4

henrywallace opened this issue Jun 20, 2018 · 1 comment

Comments

@henrywallace
Copy link

See https://github.com/erikbern/ann-benchmarks

@alexklibisz
Copy link
Owner

This is an interesting thing I've thought about a little and I'm not sure it's actually a good idea to use the exact same metrics, but I might be wrong. Here's my reasoning: the plugin is designed to handle many parallel requests across (theoretically) arbitrarily many elasticsearch nodes. In contrast, as far as I can tell, Erik's benchmarks measure raw serial speed (queries per second IIRC). Comparing recall is in principle fine, but my LSH implementation is very simplistic in the scheme of things and an identical implementation could be evaluated without all of the elasticsearch orchestration.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants