Batch support for GetOrSet

[rzepinskip]

Hi,

I am curious whether you have explicitly decided not to support batch methods (if so what are the reasons - implementation burden / other patterns to use / no need etc.) or is it planned for the future?

BTW. Neat project ;)

[jodydonetti]

Hi,

I am curious whether you have explicitly decided not to support batch methods (if so what are the reasons - implementation burden / other patterns to use / no need etc.) or is it planned for the future?

Good question!

The project was born as an abstraction layer on top of the standard IMemoryCache and IDistributedCache interfaces, so the operations available are the ones available on those interfaces + the ones I felt added value (like the atomic GetOrSet, the GetOrDefault, etc) that - while not available there - maintained the same access pattern (eg: one entry at a time).

Do you already have an idea about how those methods would be modeled?
Like I can imagine instead of a single string key it would accept an IEnumerable<string> keys, but what would be the shape of the factory? And what would the method return, a Dictionary<string,TValue> or something else? Also we should think about a factory failing for a single entry while for the others succeed (in case the impl is a series of single GetOrDefault because the underlying cache can't handle multi-get), how should we handle that?
If you have any suggestion in this regard I'd be happy to know.

As of today the need for multi-key methods did not came up, but I'm open to explore that and put it in my backlog, which by now contains adding support for events #14 , a plugin system #15 (which combined will hopefully lead, with the kind help of @JoeShook , to the support of metrics #9 with multiple providers) and a distributed backplane #11 .

BTW. Neat project ;)

Thanks 😁 !

Any suggestion about any aspect (docs, etc) is more than welcome.

[rzepinskip]

It is mainly a performance improvement when underlying service (endpoint) supports batch requests. GetOrSet and factory method could follow the same interface: accept IReadOnlyCollection<string> and return IReadOnlyDictionary<string , TValue>. Failure for a single entry when cache does not support batch request may be signaled by AggregateException but using such cache for batch requests probably should be discouraged.

[Pikabanga]

I am also interested in this feature. At my previous job I built a cache with first class support for "IEnumerable" caches.

This enabled us to load all entities (either lazily or eagerly), which we then would use for full text search, to feed dropdowns with incremental loading, etc.

It would also be useful if this concept supported adding, updating and deleting single entities from the list.

[percramer]

I was searching for exactly the same thing. Feed dropdowns with data that is used a lot. I have got multiple dropdowns which show a list of relations and it would be nice if those would come from the cache instead of the database each time

[Place1]

I'll also add that this feature would be very useful to reduce round-trip times to redis when fetching batches of keys using MGET.

[jodydonetti]

Hi @Place1 , as stated:

The project was born as an abstraction layer on top of the standard IMemoryCacheand IDistributedCache interfaces, so the operations available are the ones available on those interfaces + the ones I felt added value (like the atomic GetOrSet, the GetOrDefault, etc) that - while not available there - maintained the same access pattern (eg: one entry at a time).

So basically I'm "limited by technology of our time" 😅

We also recently talked about multi get/set in the issue about the new caching abstractions in .net , I suggest to take a look at that since there are a lot of useful informations.

Hope this helps.

[onionhammer]

I'm also interested in the same thing; heres an example scenario:

    /// <summary>
    /// Get a product by its ID
    /// </summary>
    ValueTask<Product?> GetByIdAsync(Guid id, CancellationToken cancellationToken);

    /// <summary>
    /// Get products by their IDs
    /// </summary>
    Task<IReadOnlyList<Product>> GetByIdsAsync(IList<Guid> ids, CancellationToken cancellationToken);

Ideally, each item returned from GetByIds is also cached/retrievable for GetById, but the result of GetByIds shouldn't really be cached.

Here's an implementation

        // Attempt to get all products from cache
        List<Product> products = [];
        List<Guid> productsToLookup = [];

        foreach (var id in ids)
        {
            var product = await cache.TryGetAsync<Product>($"products:{id}", token: cancellationToken);

            if (product.HasValue)
                products.Add(product.Value);
            else
                productsToLookup.Add(id);
        }

        // Get the remaining products from the database
        if (productsToLookup.Count > 0)
        {
            var productsFromDb = await dbContext.Products
                .Where(p => productsToLookup.Contains(p.Id))
                .Include(p => p.Partner)
                .ToListAsync(cancellationToken);

            products.AddRange(productsFromDb);

            // Cache the products
            foreach (var product in productsFromDb)
                await cache.SetAsync($"products:{product.Id}", product, TimeSpan.FromHours(1), cancellationToken);
        }

        return products;

Imagine if it was

        var products = await cache.GetOrSetManyAsync(
            keyFactory: id => $"products:{id}",
            valueFactory: async (ids, ct) =>
            {
                var products = await dbContext.Products
                    .Where(p => ids.Contains(p.Id))
                    .Include(p => p.Partner)
                    .ToDictionaryAsync(p => p.Id, ct);

                return products;
            });

where the valueFactory was only called for ids which weren't in the cache, and potentially there's room for optimizing caching operations by storing multiple new items in the cache at once

[ZUOXIANGE]

Here's my simple implementation

  public static async ValueTask<Dictionary<string, T>> GetOrSetManyAsync<T>(this IFusionCache cache, IEnumerable<string> keys,
      TimeSpan duration, Func<IEnumerable<string>, Task<Dictionary<string, T>>> getDataAsync, CancellationToken cancellationToken = new())
  {
      keys = keys.Distinct().ToList();
      var dic = new Dictionary<string, T>();
      List<string> keysToLookup = [];

      foreach (var key in keys)
      {
          //MGET?
          var product = await cache.TryGetAsync<T>(key, token: cancellationToken);

          if (product.HasValue)
          {
              dic.Add(key, product.Value);
          }
          else
          {
              keysToLookup.Add(key);
          }
      }

      if (keysToLookup.Count > 0)
      {
          //Get data from the database
          var data = await getDataAsync(keysToLookup);
          foreach (var item in data)
          {
              await cache.SetAsync(item.Key, item.Value, duration, cancellationToken);
              dic.Add(item.Key, item.Value);
          }
      }

      return dic;
  }

[juantxorena]

I'm also very interested in this feature. I ended up implementing something similar to @onionhammer solution, but then the question is, how does this plays with cool things like AllowTimedOutFactoryBackgroundCompletion?