Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

adds LRU cache with lazy eviction #159

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

adds LRU cache with lazy eviction #159

wants to merge 1 commit into from

Commits on Nov 26, 2022

  1. adds LRU cache with lazy eviction 

    The commit implements a variant of LRU cache with lazy eviction:
* Each entry maintains an associated ordinal value representing when the
  entry was last accessed.
* The cache is allowed to grow up to 2 times specified capacity with no
  evictions, at which point, the excess entries are evicted based on LRU
  policy resulting in an _amortized_ `O(1)` performance.

In many use cases which can allow the cache to store 2 times capacity
and can tolerate amortized nature of performance, this results in better
average performance as shown by the added benchmarks.

Additionally, with the existing implementation, `.get` requires a
mutable reference `&mut self`. In a multi-threaded setting, this
requires an exclusive write-lock on the cache even on the read path,
which can exacerbate lock contentions.
With lazy eviction, the ordinal values can be updated using atomic
operations, allowing shared lock for lookups.
    @behzadnouri
    behzadnouri committed Nov 26, 2022
    Configuration menu
    Copy the full SHA
    aaeef6e View commit details
    Browse the repository at this point in the history