Compactor is unable to proceed with compaction

[TomasKohout]

Hi guys,

we have a problem with Compactor. Initially there was a problem with eviction of compactor because we haven't set any requests for ephemeral-storage and unfortunately we had no idea that compactor is not compacting. Now we have a month of uncompacted data in Swift (we are using s3 bucket though).

The issue is that Compactor is unable to finish upload of compacted blocks. It always fails on this error and starts over.

{"caller":"compact.go:488","err":"compaction: group 0@5618499832519963523: upload of 01FK5KE690561FSR880JYGQ5TF failed: upload chunks: upload file data/compact/0@5618499832519963523/01FK5KE690561FSR880JYGQ5TF/chunks/000273 as 01FK5KE690561FSR880JYGQ5TF/chunks/000273: upload s3 object: One or more of the specified parts could not be found. The part might not have been uploaded, or the specified entity tag might not have matched the part's entity tag.","level":"error","msg":"retriable error","ts":"2021-10-29T11:27:39.314711026Z"}

Does someone have a remedy for this? My next thing what I'd try is to add repeat to upload in code, but that is a last resort for me.

Thanks for any help

[TomasKohout]

I've changed the code for uploading and now it works. I've added a few retries in a case of an error.