pageserver: reorder upload queue when possible

[erikgrinaker]

Problem

The upload queue currently sees significant head-of-line blocking. For example, index uploads act as upload barriers, and for every layer flush we schedule a layer and index upload, which effectively serializes layer uploads.

Resolves #10096.
Requires #10228.

Summary of changes

Allow upload queue operations to bypass the queue if they don't conflict with preceding operations.

[jcsp]

Let's unit test this against the most important invariants:

• A layer upload must happen before an index that references it
• A layer deletion must happen after an index that de-references it
• If a layer name that is re-used, the second upload must come after an index that de-references the earlier layer of the same name
• ...whichever others we can think of

[github-actions]

7095 tests run: 6796 passed, 2 failed, 297 skipped (full report)

---

Failures on Postgres 17

• test_timeline_archival_chaos: release-x86-64, release-arm64

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_timeline_archival_chaos[release-pg17] or test_timeline_archival_chaos[release-pg17]"

Flaky tests (4)

Postgres 17

• test_cancellations: debug-x86-64
• test_timeline_archival_chaos: release-x86-64, release-arm64

Postgres 16

• test_s3_wal_replay: release-arm64

Test coverage report is not available

_{The comment gets automatically updated with the latest test results
1df99f9 at 2024-12-30T17:08:24.977Z :recycle:}