Skip to content
This repository has been archived by the owner on Sep 16, 2024. It is now read-only.

feat(redis): implement redis cache for arns name resolutions #17

Closed
wants to merge 4 commits into from

Conversation

dtfiedler
Copy link
Collaborator

@dtfiedler dtfiedler commented Sep 4, 2024

Details:

The resolver was originally implemented when we were dealing with Smartweave contracts and lazy evaluation. We needed a way to evaluate the registry contract and ANT state quickly to avoid on-the-fly evaluations that 1. could take a long time (no checkpoints in smartweave) and 2. required a large amount of disk space (e.g. warp-sqlite). As we move to AO, the need for this type of evaluation and caching of state is no longer necessary. We can now go directly to a CU to get the necessary information and cache the results.

So, with that, this PR removes the evaluateArNSNames on service startup, replacing with a read-through cache that supports redis or lmdb caches for resolved arns data. When a name does not exist in the cache, it will go to AO to get it's resolution data (the ANT and txId to resolve to). It then caches that result for the TTL specified by the record (hence, read-through-cache).

There is a world where the arns-resolver is redundant as a separate service from ar-io-node. This new implementation (particularly using redis) could easily be dropped in directly into an ar-io-node, as it now has the ability to resolve names via AO with the introduction of this PR: ar-io/ar-io-node#189

Logs

resolver-1  | info: ArNSStore initialized {"class":"ArNSStore","kvStore":"RedisKvStore","prefix":"ArNS","timestamp":"2024-09-04T19:47:26.500Z"}
resolver-1  | info: Listening on port 6000 {"timestamp":"2024-09-04T19:47:26.537Z"}
resolver-1  | debug: Checking cache for record... {"arnsName":"dtf","timestamp":"2024-09-04T19:47:57.318Z"}
resolver-1  | debug: Cache miss for arns name {"arnsName":"dtf","timestamp":"2024-09-04T19:47:57.321Z"}
resolver-1  | debug: Successfully fetched arns name resolution {"arnsName":"dtf","cacheHit":false,"name":"dtf","timestamp":"2024-09-04T19:47:59.538Z","ttlSeconds":3600,"txId":"TzeIML94iFdoQKc6rz0JP74FLSXNscWc52q_uvXtC34"}
resolver-1  | debug: Checking cache for record... {"arnsName":"dtf","timestamp":"2024-09-04T19:48:07.724Z"}
resolver-1  | debug: Found cached arns name resolution {"arnsName":"dtf","timestamp":"2024-09-04T19:48:07.724Z"}
resolver-1  | debug: Successfully fetched arns name resolution {"arnsName":"dtf","cacheHit":true,"name":"dtf","timestamp":"2024-09-04T19:48:07.725Z","ttlSeconds":3600,"txId":"TzeIML94iFdoQKc6rz0JP74FLSXNscWc52q_uvXtC34"}
resolver-1  | debug: Checking cache for record... {"arnsName":"dylan","timestamp":"2024-09-04T19:48:14.394Z"}
resolver-1  | debug: Found cached arns name resolution {"arnsName":"dylan","timestamp":"2024-09-04T19:48:14.395Z"}
resolver-1  | debug: Successfully fetched arns name resolution {"arnsName":"dylan","cacheHit":true,"name":"dylan","timestamp":"2024-09-04T19:48:14.395Z","ttlSeconds":3600,"txId":"UyC5P5qKPZaltMmmZAWdakhlDXsBF6qmyrbWYFchRTk"}
resolver-1  | debug: Checking cache for record... {"arnsName":"permagate","timestamp":"2024-09-04T19:48:24.358Z"}
resolver-1  | debug: Cache miss for arns name {"arnsName":"permagate","timestamp":"2024-09-04T19:48:24.359Z"}
resolver-1  | debug: Successfully fetched arns name resolution {"arnsName":"permagate","cacheHit":false,"name":"permagate","timestamp":"2024-09-04T19:48:26.154Z","ttlSeconds":3600,"txId":"eUHEOYu1TjVk8CuQYzhQ9_jevCbJjihDrf3IJ-XvadQ"}

@dtfiedler dtfiedler force-pushed the PE-5957-redis-cache branch 2 times, most recently from 47ec09f to 28e2798 Compare September 4, 2024 19:33
@dtfiedler dtfiedler marked this pull request as ready for review September 4, 2024 23:38
Copy link

@djwhitt djwhitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems good overall, but suspect we're going to get bitten by the on-demand resolution at some point when AO infra is down. If we use a longer TTL in Redis but record and check the TTL for the record itself before serving the value from the cache, while still falling back to the cache if AO is down, we could avoid some cases where that would happen.

dtfiedler added 2 commits September 9, 2024 10:28
…o AO

If we have a cached value for the arns name in redis, and fail to get the updated one from AO, serve that. This should help protect downtime with AO infra between name resolutions.

Also introduces prometheus and metrics for cache hits.
@dtfiedler
Copy link
Collaborator Author

Replacing this PR with ar-io/ar-io-node#200 on the ar-io-node. We will mark this service as maintence mode and no longer a critical component for release 18+

@dtfiedler dtfiedler closed this Sep 10, 2024
@dtfiedler dtfiedler deleted the PE-5957-redis-cache branch September 10, 2024 15:18
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants