pageserver: coalesce index uploads when possible

[erikgrinaker]

Problem

With upload queue reordering in #10218, we can easily get into a situation with multiple index uploads queued back to back, which can't be parallelized. This will happen e.g. when multiple layer flushes enqueue layer/index/layer/index/... and the layers skip the queue and are uploaded in parallel.

These index uploads will incur serial S3 roundtrip latencies, and may block later operations.

Resolves #10096.
Requires #10218.

Summary of changes

When multiple back-to-back index uploads are ready to upload, only upload the most recent index and drop the rest.

[github-actions]

6450 tests run: 6190 passed, 1 failed, 259 skipped (full report)

---

Failures on Postgres 17

• test_timeline_archival_chaos: release-arm64

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_timeline_archival_chaos[release-pg17]"

Flaky tests (3)

Postgres 16

• test_isolation[4]: release-x86-64
• test_remote_storage_backup_and_restore[True-real_s3]: release-x86-64

Postgres 14

• test_physical_replication_config_mismatch_max_locks_per_transaction: release-arm64

Test coverage report is not available

_{The comment gets automatically updated with the latest test results
76698d5 at 2025-01-02T16:37:40.599Z :recycle:}