Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MigratableKVStore trait #3481

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

tnull
Copy link
Contributor

@tnull tnull commented Dec 11, 2024

Closes #3387

We add a MigratableKVStore trait (suggestion for better naming are welcome) that describes methods required to write generalized KVStore-to-KVStore migration logic. In particular, it adds afn list_all_keys method that exhaustively lists all keys stored, which then can be used by a utility to transfer all of them to the new store before deleting the old state.

We implement this new trait for FilesystemStore.

TODOs (draft until then):

  • Add simple migration utility function.
  • Add test code for FilesystemStore to FilesystemStore migration.

Should be ready for review.

@tnull tnull requested a review from G8XSU December 11, 2024 15:59
@tnull tnull marked this pull request as draft December 11, 2024 15:59
@tnull tnull added this to the 0.1 milestone Dec 11, 2024
.. which will be used for generic `KVStore`-to-`KVStore` migration
logic.
@tnull tnull force-pushed the 2024-12-add-kvstore-migration-ext branch from 98239df to b75d953 Compare December 12, 2024 12:35
@tnull tnull changed the title Add KVStoreMigrator trait Add MigratableKVStore trait Dec 12, 2024
@tnull tnull marked this pull request as ready for review December 12, 2024 14:45
@tnull
Copy link
Contributor Author

tnull commented Dec 12, 2024

Added a migration util method and a test for FilesystemStore-to-FilesystemStore migration. Should be ready for review now.

lightning-persister/src/fs_store.rs Show resolved Hide resolved
lightning/src/util/persist.rs Outdated Show resolved Hide resolved
lightning-persister/src/fs_store.rs Outdated Show resolved Hide resolved
lightning-persister/src/fs_store.rs Outdated Show resolved Hide resolved
lightning-persister/src/fs_store.rs Show resolved Hide resolved
lightning/src/util/persist.rs Outdated Show resolved Hide resolved
@tnull tnull force-pushed the 2024-12-add-kvstore-migration-ext branch from 877a636 to bbcd31c Compare December 12, 2024 18:05
@tnull
Copy link
Contributor Author

tnull commented Dec 12, 2024

Addressed pending comments, let me know if I can squash fixups.

@tnull tnull force-pushed the 2024-12-add-kvstore-migration-ext branch from bbcd31c to adcbd91 Compare December 12, 2024 18:33
lightning/src/util/persist.rs Show resolved Hide resolved
lightning/src/util/persist.rs Outdated Show resolved Hide resolved
lightning/src/util/persist.rs Outdated Show resolved Hide resolved
@tnull tnull force-pushed the 2024-12-add-kvstore-migration-ext branch from adcbd91 to f391296 Compare December 12, 2024 21:13
lightning-persister/src/fs_store.rs Show resolved Hide resolved
lightning-persister/src/fs_store.rs Outdated Show resolved Hide resolved
lightning-persister/src/fs_store.rs Show resolved Hide resolved
lightning-persister/src/test_utils.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@G8XSU G8XSU left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, mod nits.

lightning-persister/src/fs_store.rs Outdated Show resolved Hide resolved
@tnull tnull force-pushed the 2024-12-add-kvstore-migration-ext branch from f391296 to ecdacd3 Compare December 13, 2024 08:46
@tnull
Copy link
Contributor Author

tnull commented Dec 13, 2024

Addressed pending feedback.

@TheBlueMatt
Copy link
Collaborator

Feel free to squash, IMO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

KVStore: Add the capability to list namespaces
3 participants