Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement caching system #73

Merged
merged 18 commits into from
Dec 19, 2024
Merged

Implement caching system #73

merged 18 commits into from
Dec 19, 2024

Conversation

albireox
Copy link
Member

@albireox albireox commented Dec 9, 2024

This implements caching for Valis queries.

Uses fastapi-cache to optionally cache Valis routes. Two backends are supported, Redis and memcached, which can be toggled using a new setting cache_backend (defaults to redis but memcached does not require ).

I added cache decorators to all the suggested routes in #69 plus some of the /query/list ones except the main query. The library only supports caching GET requests.If this is a dealbreaker we can consider something different for those routes.

On Redis, the key with the cached values is of the form fastapi-cache:valis-cache:get:/query/list/cartons:[] which includes the route and the query params used.

Closes #69

@albireox albireox requested a review from havok2063 as a code owner December 9, 2024 19:41
@albireox
Copy link
Member Author

albireox commented Dec 9, 2024

One thing that it doesn't seem to be implemented in fastapi-cache is the option to temporarily (or for easier development) disable caching. The implementation of a backend in fastapi-cache is relatively simple

class Backend(abc.ABC):
    @abc.abstractmethod
    async def get_with_ttl(self, key: str) -> Tuple[int, Optional[bytes]]:
        raise NotImplementedError

    @abc.abstractmethod
    async def get(self, key: str) -> Optional[bytes]:
        raise NotImplementedError

    @abc.abstractmethod
    async def set(self, key: str, value: bytes, expire: Optional[int] = None) -> None:
        raise NotImplementedError

    @abc.abstractmethod
    async def clear(self, namespace: Optional[str] = None, key: Optional[str] = None) -> int:
        raise NotImplementedError

It's probably not super hard to create a null backend that doesn't do anything and always executes the route.

@albireox
Copy link
Member Author

I think this is now ready for review. I've made a few changes:

  • We are now using an slightly custom version of the fastapi-cache cache decorator. This one supports POST routes and hashes the contents of the request body. With this we can decorate the main query route. A caveat is that cached routes cannot return an iterator or they will return an empty list in the first request (when they don't hit the cache). I think this is difficult to avoid the way that the decorator works. But I'm also unsure that the generator does much in these cases since the route does need to return the full data payload in one go (unless something like streaming is used).
  • I increased the default cache time to 6 months, not sure if we want more or less.
  • I replaced the memcached backend (I realised this does require installing software) with an in-memory backend that does not require anything external.
  • I added a null backend that does no caching.

Copy link
Contributor

@havok2063 havok2063 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I'll pull and test it out first before approving. Is there a way we can explicitly trigger the cache to clear, and wipe the redis or in-memory cache?

python/valis/cache.py Outdated Show resolved Hide resolved
python/valis/cache.py Outdated Show resolved Hide resolved
python/valis/routes/query.py Outdated Show resolved Hide resolved
python/valis/routes/query.py Outdated Show resolved Hide resolved
python/valis/routes/target.py Outdated Show resolved Hide resolved
python/valis/routes/target.py Show resolved Hide resolved
python/valis/cache.py Show resolved Hide resolved
@albireox
Copy link
Member Author

To allow clearing the cache I've added a function valis.cache.clear_redis_cache(). If called without arguments it will clear all the keys under fastapi-cache:valis-*.

@havok2063 havok2063 merged commit 6aa0530 into main Dec 19, 2024
2 checks passed
@albireox albireox deleted the albireox-issue-69 branch December 19, 2024 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

implement route caching
2 participants