4.14 small node-density latencies are around 11s now

[afcollins]

Type of change

• Bug fix

Description

4.14 small node-density p99 Ready latencies are consistently around 11s. We shouldn't be wasting an entire cluster and the remainder of tests because the p99 isn't what we expect.

Latest result:

[2023-08-09, 12:50:46 UTC] {subprocess.py:93} INFO - time="2023-08-09 12:50:46" level=info msg="node-density: Ready 50th: 1294 99th: 10542 max: 14945 avg: 2171" file="pod_latency.go:174"
[2023-08-09, 12:50:46 UTC] {subprocess.py:93} INFO - time="2023-08-09 12:50:46" level=info msg="node-density: ReadyV2 50th: 2000 99th: 11000 max: 15000 avg: 2697" file="pod_latency.go:174"

http://airflow.apps.sailplane.perf.lab.eng.rdu2.redhat.com/dags/4.14-aws-ovn-small-cp/grid?dag_run_id=scheduled__2023-08-02T12%3A00%3A00%2B00%3A00&task_id=node-density&tab=logs

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.

Testing

Happens in CPT Airflow.