Skip to content

Commit

Permalink
Address review comments
Browse files Browse the repository at this point in the history
Signed-off-by: Naveen Tatikonda <[email protected]>
  • Loading branch information
naveentatikonda committed Jun 26, 2024
1 parent b4935a5 commit 2885942
Showing 1 changed file with 6 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ leading to higher memory requirements and increased operational costs. Faiss sca

## Why use Faiss scalar quantization?

When you index vectors in [OpenSearch 2.13](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.13.0.md) or later versions, you can configure your k-NN index to apply a technique called _scalar quantization_. Scalar quantization converts each dimension of a vector from a 32-bit floating-point (`fp32`) to a 16-bit floating-point (`fp16`) representation. Using the Faiss scalar quantizer (SQfp16), integrated in the k-NN plugin, you can get up to a 50% memory savings with a very minimal loss of recall (see [Benchmarking results](#benchmarking-results)). When used with [SIMD optimization](https://opensearch.org/docs/latest/search-plugins/knn/knn-index#simd-optimization-for-the-faiss-engine),
When you index vectors in [OpenSearch 2.13](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.13.0.md) or later versions, you can configure your k-NN index to apply _scalar quantization_. Scalar quantization converts each dimension of a vector from a 32-bit floating-point (`fp32`) to a 16-bit floating-point (`fp16`) representation. Using the Faiss scalar quantizer (SQfp16), integrated in the k-NN plugin, you can get up to a 50% memory savings with a very minimal loss of recall (see [Benchmarking results](#benchmarking-results)). When used with [SIMD optimization](https://opensearch.org/docs/latest/search-plugins/knn/knn-index#simd-optimization-for-the-faiss-engine),

Check failure on line 23 in _posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.TableHeadings] 'm' is a table heading and should be in sentence case. Raw Output: {"message": "[OpenSearch.TableHeadings] 'm' is a table heading and should be in sentence case.", "location": {"path": "_posts/2024-06-19-optimizing-opensearch-with-fp16-quantization.md", "range": {"start": {"line": 23, "column": 62}}}, "severity": "ERROR"}
SQfp16 quantization can also significantly reduce search latencies and improve indexing throughput.

## How to use Faiss scalar quantization
Expand Down Expand Up @@ -117,11 +117,11 @@ to compare the indexing, search performance, and quality of search results of Fa
enabled on x86 architecture with AVX2 optimization.

**Note**: Without SIMD optimization (AVX2 or NEON) or with AVX2 disabled (on x86 architecture), the quantization process introduces additional overhead, which leads to an increase in latency.
For information about processors that support AVX2, see [CPUs with AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2). In AWS, all community Amazon Machine Images (AMIs) with [HVM](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html) support AVX2 optimization for the x86 architecture.
For information about processors that support AVX2, see [CPUs with AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX2). In an AWS environment, all community Amazon Machine Images (AMIs) with [HVM](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html) support AVX2 optimization for the x86 architecture.

### Benchmarking results using small workloads

We ran the following tests on a single-node cluster without any replicas using the following datasets.
We ran the following tests on a single-node cluster without any replicas.


#### Configuration
Expand Down Expand Up @@ -226,24 +226,22 @@ For more information about estimating the required memory and number of data nod

## Conclusion

Faiss FP16 scalar quantization is a powerful technique that provides significant memory savings while maintaining high recall performance similar to full-precision vectors. By converting vectors to a 16-bit floating-point representation, it can reduce memory requirements by up to 50%. When combined with SIMD optimization, FP16 scalar quantization also enhances indexing throughput and reduces search latencies, leading to better overall performance. This method strikes an excellent balance between memory efficiency and accuracy, making it a valuable tool for large-scale similarity search applications.
Faiss SQfp16 scalar quantization is a powerful technique that provides significant memory savings while maintaining high recall performance similar to full-precision vectors. Converting vectors to a 16-bit floating-point representation can reduce memory requirements by up to 50%. When combined with SIMD optimization, SQfp16 scalar quantization also enhances indexing throughput and reduces search latency, leading to better overall performance. This method strikes an excellent balance between memory efficiency and accuracy, making it a valuable tool for large-scale similarity search applications.

## Future scope

To achieve even greater memory efficiency, we plan to introduce `int8` quantization support using a [Faiss scalar quantizer](https://github.com/opensearch-project/k-NN/issues/1723) and [Lucene scalar quantizer](https://github.com/opensearch-project/k-NN/issues/1277).
This advanced technique will enable a remarkable 75% reduction in memory requirements, or 4x compression, compared to full-precision vectors while maintaining high recall performance.
The quantizers will accept `fp32` vectors as input, perform online training, and quantize the data into byte-sized vectors, eliminating the need for external quantization or extra training steps.

Furthermore, we aim to release binary vector support, enabling an unprecedented 32x compression rate. This groundbreaking approach will further reduce memory consumption.
By combining these cutting-edge quantization techniques, we will provide a comprehensive solution for efficient similarity search, balancing memory optimization and
accurate retrieval.
Furthermore, we aim to release binary vector support, enabling an unprecedented 32x compression rate. This groundbreaking approach will further reduce memory consumption.

Our commitment to continuous innovation delivers state-of-the-art technologies to tackle large-scale similarity search challenges while minimizing resource
requirements and maximizing cost-effectiveness.

## Appendix: Memory and data node requirement estimation

The following calculations provide an estimation of the memory and number of data nodes required for the 100M, 768-dimension large workload benchmarking test:
Here are some estimates of the amount of memory and number of data nodes needed for the 100M, 768 dimension large workload benchmarking test:

```
// Faiss HNSW SQfp16 Memory Estimation
Expand Down

0 comments on commit 2885942

Please sign in to comment.