allow enabling async metadata propagation #789
Conversation
Signed-off-by: Jörn Friedrich Dreyer <[email protected]>
butonic
force-pushed
the
async-postprocessing
branch
from
e34c9ed
to
e3e5fff
Compare
|
cc @kobergj |
| - name: OCIS_DECOMPOSEDFS_PROPAGATOR | ||
| value: "async" |
There was a problem hiding this comment.
owncloud.dev says:
The propagator used for decomposedfs. At the moment, only ‘sync’ is fully supported, ‘async’ is available as an experimental option.
There was a problem hiding this comment.
During load tests we saw that uploading 1000 files into a folder caused a third of them to fail because the propagatien ran into stale NFS file handles. By delaying the propagation we can aggregate the updates to the root metadata to one write every 5sec (configurable). This takes a lot of pressure of the parent node.
It was never productized this because we never had the time to test this in kubernetes. We did that yesterday and it allowed all 1000 files to be uploaded without error.
That being said I saw several Missing parent ID on node as mentioned in https://github.com/owncloud/enterprise/issues/6690#issuecomment-2459106891
Looking into that right now.
There was a problem hiding this comment.
It was never productized this because we never had the time to test this in kubernetes. We did that yesterday and it allowed all 1000 files to be uploaded without error.
Understood. Can we change the value for the oCIS product / oCIS Chart to hardcoded async in this case now? I'd like to keep the number of configurations permutations low. Same story as the async postprocessing back then...
There was a problem hiding this comment.
@micbar do you have an opinion on this?
This PR allows enabling the async pmetadata propagation. It defaults to off and a 5s delay.