Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Reduce memory use when obfuscating staging environment data
Django caches query results by default. See this comment from the Django docs: In a newly created QuerySet, the cache is empty. The first time a QuerySet is evaluated – and, hence, a database query happens – Django saves the query results in the QuerySet’s cache and returns the results that have been explicitly requested (e.g., the next element, if the QuerySet is being iterated over). Subsequent evaluations of the QuerySet reuse the cached results. -- https://docs.djangoproject.com/en/5.1/topics/db/queries/#caching-and-querysets This can result in significant memory use. The live box was running out of memory when we were attempting to obfuscate email details, presumably because that model contains some large text fields. We disable caching as we only need to iterate over each queryset once and, obviously, we need to address the issue of running out of memory.
- Loading branch information