diff --git a/examples/huggingface/pytorch/text-generation/h2o/README.md b/examples/huggingface/pytorch/text-generation/h2o/README.md index 2a251344746..22064c00b29 100644 --- a/examples/huggingface/pytorch/text-generation/h2o/README.md +++ b/examples/huggingface/pytorch/text-generation/h2o/README.md @@ -1,6 +1,6 @@ # H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models -**Heavy-Hitter Oracal (H2O)** is a novel approach for implementing the KV cache wihich significantly reduces memory footprint. +**Heavy-Hitter Oracal (H2O)** is a novel approach for implementing the KV cache which significantly reduces memory footprint. This methods base on the fact that the accumulated attention scores of all tokens in attention blocks adhere to a power-law distribution. It suggests that there exists a small set of influential tokens that are critical during generation, named heavy-hitters (H2). H2 provides an opportunity to step away from the combinatorial search problem and identify an eviction policy that maintains accuracy.