Question: Efficient Cache Invalidation Strategy

[mhayward-dev]

Hello, I am new to Redis and trying to improve my understanding so forgive my ignorance.

Problem:
I am seeing many chain SCAN calls in Azure App Insights that hold up the request and looking to find a much more efficient method to find and bulk delete keys with the same prefix.

Example problem:
In this case each SCAN is very fast but we see each scan taking >1ms across regions.

Implementation:

1. Setting a key for example {account123}:attribute1:attribute2

connection.GetDatabase().StringSetAsync(key, data, cacheTime, When.Always, flags: CommandFlags.FireAndForget);

2. Bulk removal for an account prefix where prefix is {account123}. We do not expect a high amount of records being returned around 5 - 20.

var pageSize = 50;
var database = connection.GetDatabase();

foreach (var endpoint in database.Multiplexer.GetEndPoints())
{
     var server = database.Multiplexer.GetServer(endpoint);
     var results = server.KeysAsync(pattern: $"{prefix}:*", pageSize: pageSize);

     await foreach (var pageResults in results.Segment(pageSize))
     {
         await database.KeyDeleteAsync(pageResults);
      }
}

How could I improve this solution to avoid waiting on the SCAN to complete? I am looking at HSET but in this case I need to set an expiry per name/value.

Thanks in advance.

[mgravell]

Right, so: this is a complex topic.

Recent Redis builds have HEXPIRE for per-field expiration on hashes, but: SE Redis does not yet support that, and IIRC neither does azure Redis yet (@philon-msft would know more about that).

There is no direct way to do what you want here. The SCAN approach is very inefficient, and the direct alternative requires maintaining a secondary index of impacted data OutputCache uses this approach), and is complex and also not cheap. However, in HybridCache I'm working on a mechanism to do this indirectly. The approach in progress is to store the expiration for the prefix explicitly in a separate key, and compare during lookup. Maybe simply switching to HybridCache and using the "tags" feature is the way to go here (when I've finished it).

[slorello89]

@mhayward-dev - one approach could be to use RediSearch to search for keys you want to invalidate rather than Scanning over all of your Redis Instances.

You'd just store your data in a hash, and have whatever the tag is you want to use for invalidating be a separate field in that tag (the prefix in the case of a scan). Let's just call that invalidationAttribute, first you'd create a secondary index on top of Redis:

## Creates a secondary index in Redis, all hash keys with prefix item: will be indexed
FT.CREATE targeted-invalidation-idx ON HASH PREFIX 1 item: SCHEMA invalidationAttribute TAG

then you'd create your items in Redis, they would have the invalidationAttribute as a field, and you could store your data in either other field-value pairs within the hash, or if you choose you could just have a big blob of binary / json /protbuf / whatever in a data field.

HSET item:1 invalidationAttribute bannana data "some arbitrary data"
HSET item:2 invalidationAttribute apple 

Then you can search for your keys using the search syntax:

FT.SEARCH targeted-invalidation-idx "@invalidationAttribute:{apple}" NOCONTENT

Then you can just call KeyDelete on all the keys you get back from that.

RediSearch is available currenting in the Enterprise Tier of Azure Cache for Redis, or in all non-flash tiers of Azure Managed Redis which went into public preview about a month ago.

All the Search commands are available in NRedisStack - which builds directly on top of SE.Redis.