Make WAL keys TLI aware #491
Conversation
|
This will break existing clusters as anything added to This needs to be dealt with somehow now that we're past 1.0. My suggestion is to completely separate key storage for wal keys and relation keys so that relation key file format can stay as it is. That would also let us get rid of that awful fake rlocator we use to find the wal key file. |
|
Yeah, that totally makes sense. Will do that with additional commit |
Before this commit, WAL keys didn't mind TLI at all. But after pg_rewind, for example, pg_wal/ may contain segments from two timelines. And the wal reader choosing the key may pick the wrong one because LSNs of different TLIs may overlap. There was also another bug: There is a key with the start LSN 0/30000 in TLI 1. And after the start in TLI 2, the wal writer creates a new key with the SN 0/30000, but in TLI 2. But the reader wouldn't fetch the latest key because w/o TLI, these are the same. This commit adds TLI to the Internal keys and makes use of it along with LSN for key compares.
Codecov Report❌ Patch coverage is ❌ Your project status has failed because the head coverage (81.94%) is below the target coverage (90.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## TDE_REL_17_STABLE #491 +/- ##
=====================================================
- Coverage 83.68% 81.94% -1.75%
=====================================================
Files 21 24 +3
Lines 2771 3018 +247
Branches 435 490 +55
=====================================================
+ Hits 2319 2473 +154
- Misses 368 444 +76
- Partials 84 101 +17
🚀 New features to boost your workflow:
|
dAdAbird
force-pushed
the
keys_tli
branch
2 times, most recently
from
17f892c to
d9c85aa
Compare
| @@ -369,22 +370,23 @@ pg_tde_delete_principal_key(Oid dbOid) | |||
| * needs keyfile_path | |||
| */ | |||
| void | |||
| pg_tde_wal_last_key_set_lsn(XLogRecPtr lsn, const char *keyfile_path) | |||
| pg_tde_wal_last_key_set_lsn(XLogRecPtr lsn, TimeLineID tli, const char *keyfile_path) | |||
There was a problem hiding this comment.
Why doesn't this take a WalLocation?
WIP, needs tests
Before this commit, WAL keys didn't mind TLI at all. But after pg_rewind, for example, pg_wal/ may contain segments from two timelines. And the wal reader choosing the key may pick the wrong one because LSNs of different TLIs may overlap. There was also another bug: There is a key with the start LSN 0/30000 in TLI 1. And after the start in TLI 2, the wal writer creates a new key with the SN 0/30000, but in TLI 2. But the reader wouldn't fetch the latest key because w/o TLI, these are the same.
This commit adds TLI to the Internal keys and makes use of it along with LSN for key compares.