Having 1 billion of debug logs per month related to "STREAMED_XOR_CHUNKS"?

[Aracki]

I have really huge number of these logs in my Thanos Sidecar container:

ERROR 2024-08-26T17:29:59.327714138Z ts=2024-08-26T17:29:59.327631678Z caller=prometheus.go:317 level=debug msg="started handling ReadRequest_STREAMED_XOR_CHUNKS streamed read response."
ERROR 2024-08-26T17:29:59.346030558Z ts=2024-08-26T17:29:59.342300866Z caller=prometheus.go:317 level=debug msg="started handling ReadRequest_STREAMED_XOR_CHUNKS streamed read response."
ERROR 2024-08-26T17:29:59.359530163Z ts=2024-08-26T17:29:59.359391247Z caller=prometheus.go:400 level=debug msg="handled ReadRequest_STREAMED_XOR_CHUNKS request." frames=378
ERROR 2024-08-26T17:29:59.368072842Z ts=2024-08-26T17:29:59.367948319Z caller=prometheus.go:317 level=debug msg="started handling ReadRequest_STREAMED_XOR_CHUNKS streamed read response."
ERROR 2024-08-26T17:29:59.382843608Z ts=2024-08-26T17:29:59.382693628Z caller=prometheus.go:317 level=debug msg="started handling ReadRequest_STREAMED_XOR_CHUNKS streamed read response."
ERROR 2024-08-26T17:29:59.418191176Z ts=2024-08-26T17:29:59.418017731Z caller=prometheus.go:317 level=debug msg="started handling ReadRequest_STREAMED_XOR_CHUNKS streamed read response."

I couldn't find any information which would help me debug how and why this is being logged so many times. Any idea would be helpful? 🙏

[GiedriusS]

Sorry for the spam, I think it is a left-over from when this was being implemented. I think we can just remove it at this point. Care to open up a small PR for that?

[Aracki]

Not sure, but it seems it somehow has to do something with my thanos-sidecar not being able to push metrics to the bucket.

After I fixed that underlying issue, no more "STREAMED_XOR_CHUNKS" logs (debug level is enabled). 🤷

[Aracki]

Oh sorry, my bad - when I turn debug logs it actually appears. I'll check the code and maybe come up with a PR, thanks!