IBX-5502: Added additional tag cleaning to limit down number of orphaned tag entries #372
Conversation
|
Kudos, SonarCloud Quality Gate passed!
|
| $this->cacheIdentifierGenerator->generateTag( | ||
| $locations = $this->persistenceHandler->locationHandler()->loadLocationsByContent($contentId); | ||
|
|
||
| $locationTags = array_map(function (Content\Location $location) { |
There was a problem hiding this comment.
| $locationTags = array_map(function (Content\Location $location) { | |
| $locationTags = array_map(function (Content\Location $location): string { |
alongosz
changed the title
|
I apologize - I've missed the CR.
Fixed.
It is not a tag, but a key. For example, this key is a member of a "Location" tag - those additional tag purgings resolves this key being orphaned when Content is sent to the Trash. |
|
Kudos, SonarCloud Quality Gate passed!
|
There was a problem hiding this comment.
Could you implement the code suggestions provided by @konradoboza https://github.com/ezsystems/ezplatform-kernel/pull/372/files#r1189636279
Co-authored-by: Konrad Oboza <[email protected]>
Co-authored-by: Konrad Oboza <[email protected]>
|
@kisztof done (test failing not related to pr) and thank you for reminding me. Happy new year! |
|
There was a problem hiding this comment.
My tests show, that one of the biggest culprits here is lp- tag, which can have multiple orphaned entries - therefore it might be a good idea to purge it as well, to avoid blocked memory in cost of potentially small performance loss - up for discussion.
Are you able to tell what performance losses are we observing here?
To be frank no, but looking at it this is a simple tag. Due to the way how caching works - worst-case scenario it will get recreated the next time it is required, and as it is In addition, freeing this memory to be reused should - especially in the long run - benefit to overall application performance (more memory, less evictions). |
|
v3.3Due to the how cache is implemented in Symfony-based projects using the Redis adapter (and probably - not only Redis), entries (Members) inside a Tag (Set) are removed only when the tag is purged. Due to that, we can encounter on multiple occasions a situation, when a Tag is purged, resulting in cache items (Keys) being purged as well, but only the Tag that triggered a purge is cleaned up. All other tags that contain entries pointing to cache items that no longer exist will remain there until the affected tag is purged - which often is "never".
This results in orphaned entries in tags (or even in whole orphaned tags) filling up the memory. Memory allocated to tags in Redis is nonevictable, therefore it is freed only when purged on purpose (
maxmemory-policysetting won't affect orphaned members/sets). On very big and/or constantly changing content structures, this unevictable memory allocated to orphaned tags and entries can significantly lower the amount of memory available for standard cache items to be stored.Additional Tag purging added in this PR reduces this effect, but - unfortunately - it is not possible to resolve it fully with current Symfony's cache adapters implementation, due to the fact that there is no available interface to remove an entry from a Tag, without purging it whole - and that would result in purging cache items that were nor related to the original action performed on content.
Additional tag purging introduced by PR:
ContentHandler::updateContent:Multiple content-related entries (
ibx-c-<contentId>-<versionNo>-0,ibx-c-<contentId>-<versionNo>-<languageCode>,ibx-cvi-<contentId>,ibx-cl-<contentId>-pfdare orphaned in tags:l-<locationId>,lp-<locationId>,c-<contentId>-v-<versionNo>,c-<contentId>-v-<prevVersionNo>. Added purging of those tags to avoid it.TrashHandler::trashSubtree:Multiple content-related entries are orphaned (
ibx-c-<contentId>-v<versionNumber>,ibx-cl-<contentId>,ibx-cvi-<contentId>,ibx-l-<locationId>-<languageCode>,ibx-c-<contentId>-<versionNo>-0,ibx-c-<contentId>-<versionNo>-<languageCode>,ibx-cvi-<contentId>... (many more) are orphaned in tags:l-<locationId>,c-<contentId>-v-<versionNo>.Handling those two occurrences helps to limit greatly the number of orphaned members.
My tests show, that one of the biggest culprits here is
lp-<locationIdOfParent>tag, which can have multiple orphaned entries - therefore it might be a good idea to purge it as well, to avoid blocked memory in cost of potentially small performance loss - up for discussion.This PR won't resolve the issue fully but limits the scope of it using available tools. To resolve this fully, we will need (in my opinion):
But those two are out of this PR scope.
Checklist:
$ composer fix-cs).@ezsystems/engineering-team).