[21232] Fix Latency tests - 2.x backports (backport #5017)

[mergify]

Description

This PR takes the latency tests fix from #5009 and brings it to 2.x.

@Mergifyio backport 2.13.x 2.10.x

Contributor Checklist

• Commit messages follow the project guidelines.

• The code follows the style guidelines of this project.

• NO Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally

• Any new/modified methods have been properly documented using Doxygen.

• Any new configuration API has an equivalent XML API (with the corresponding XSD extension)

• Changes are backport compatible: they do NOT break ABI nor change library core behavior.

• Changes are API compatible.

• New feature has been added to the versions.md file (if applicable).

• NO New feature has been documented/Current behavior is correctly described in the documentation.

• Applicable backports have been included in the description.

Reviewer Checklist

• The PR has a milestone assigned.
• The title and description correctly express the PR's purpose.
• Check contributor checklist is correct.
• If this is a critical bug fix, backports to the critical-only supported branches have been requested.
• Check CI results: changes do not issue any warning.
• Check CI results: failing tests are unrelated with the changes.

---

This is an automatic backport of pull request #5017 done by [Mergify](https://mergify.com).