LSPS5 implementation #3662

martinsaposnic · 2025-03-11T20:12:16Z

A complete implementation for LSPS5 (spec defined here lightning/blips#55)

Reviewing commit by commit is recommended (~40% of the added lines are tests)

Notes:

Persistence of the state will come on a future PR
~~- Will rebase from lightning-liquidity: Introduce EventQueue notifier and wake BP for message processing #3509 once that PR is merged~~ DONE

ldk-reviews-bot · 2025-03-11T20:12:19Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

martinsaposnic · 2025-03-11T20:16:28Z

This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently.

codecov · 2025-03-11T20:21:35Z

Codecov Report

Attention: Patch coverage is 89.30995% with 189 lines in your changes missing coverage. Please review.

Project coverage is 89.11%. Comparing base (f507778) to head (1b43559).
Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning-liquidity/src/lsps5/client.rs	88.39%	49 Missing and 14 partials ⚠️
lightning-liquidity/src/lsps5/msgs.rs	84.81%	43 Missing and 17 partials ⚠️
lightning-liquidity/src/lsps0/ser.rs	73.78%	9 Missing and 18 partials ⚠️
lightning-liquidity/src/lsps5/service.rs	91.94%	18 Missing and 9 partials ⚠️
lightning-liquidity/src/lsps5/url_utils.rs	97.81%	5 Missing and 2 partials ⚠️
lightning-liquidity/src/manager.rs	93.65%	2 Missing and 2 partials ⚠️
lightning-liquidity/src/lsps0/msgs.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3662      +/-   ##
==========================================
- Coverage   89.13%   89.11%   -0.03%     
==========================================
  Files         157      161       +4     
  Lines      123851   125617    +1766     
  Branches   123851   125617    +1766     
==========================================
+ Hits       110395   111943    +1548     
- Misses      10779    10929     +150     
- Partials     2677     2745      +68

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tnull

Wow, thank you for looking into this! I did a first pass, and it looks pretty amazing already!

Before going too much into further details, here are a few general comments upfront:

I'm generally no fan of introducing additional dependencies here, an in particular not reqwest and tokio. I think following the pattern so far BroadcastNotifications could be a request that the user handles with any HTTP client they want and then could call back into LSPS5ServiceHandler. Alternatively, we could also use a trait similar to the current HTTPClient, but I don't think we want to keep the default implementation. Note that the blocking reqwest variant wraps a tokio runtime internally, and therefore should never (1, 2, ...) be used together. I guess technically we could consider a default async version of the trait that uses async reqwest, but I would prefer to simply have well-documented trait on our end that the user can implement however they choose to. Also note that stacking tokio runtimes is heavily discouraged in general, so assuming our users would themselves use a tokio runtime, we shouldn't wrap one in LSPS5ServiceHandler.
Note that lightning-liquidity is optionally no-std compliant, so please don't rely on std wherever possible, often it's just a matter of using core instead and importing the respective types from crate::prelude. If you really find yourselves needing to use std, make sure it's feature gated behind feature = "std" and we provide an alternative for users that don't support it.
Minor: Regarding formatting we're using tabs, not spaces. Feel free to run ./contrib/run-rustfmt.sh after each commit to run our formatting scripts.
This PR in its current scope is great, just want to note that eventually we need to add persistence for the state. As we haven't fully fleshed out the persistence strategy for lightning-liquidity in general yet, it's actually preferred to defer this to a follow-up, but just wanted to mention it. Also note that some changes to MessageQueue/EventQueue will happen in lightning-liquidity: Introduce EventQueue notifier and wake BP for message processing #3509, but will ping you to rebase once that has been merged. (just sidenotes here).

I hope these initial points make sense, let me know if you have any questions, or once you made corresponding changes and think this is ready for the next round of review!

ldk-reviews-bot · 2025-03-13T12:45:04Z

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

tnull · 2025-03-13T12:48:30Z

This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently.

Thanks for asking! I'm totally fine to keep this (with its current scope) in a single PR, as long as we keep the commit history pretty clean to allow continuing review to happen commit-by-commit. To this end, please make sure add any fixup commits clearly marked (e.g. via a f or fixup prefix in the commit message header) directly under the commit they belong to, so they can cleanly be squashed into the respective feature commits in-between review rounds.

tnull · 2025-03-13T12:54:53Z

Btw, I'm not sure if you're familiar with the previous attempt of implementing LSPS5: #3499

Given this is a clean slate, not sure how much there is to learn, but still might be worth a look. Also not sure if @johncantrell97 would be interested in reviewing this PR, too, as he's familiar with the codebase and LSPS5.

martinsaposnic · 2025-03-14T17:11:07Z

@tnull, ready for the next review round!

Removed reqwest and tokio.
HttpClient is now a generic trait, allowing users to pass any implementation.
Removed std.
Small refactor on client and service to be able to delete some silly / unnecessary code.
Ran formatting on every commit.
All new changes are in commits prefixed with fixup:.

CI is failing because of the usage of the url crate (which I guess does not support rust 1.63?), which I need for validating the webhook URL. A new commit will come addressing that shortly

ldk-reviews-bot · 2025-03-17T00:00:52Z

🔔 1st Reminder

Hey @tnull @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-03-17T00:00:54Z

🔔 1st Reminder

Hey @tnull @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

martinsaposnic · 2025-04-18T18:29:06Z

@tnull thanks a lot for the review!

All review comments have been addressed in fixup commits. Let me know what you think!

Side note. There are a few things I’m still not fully convinced about:

I made the requested changes to url_utils, but now I’m unsure if this util is something we should keep long-term. I have a (not fully working) draft that takes inspiration from the url crate and builds a minimal, no-std-friendly version tailored to our needs. I’d love your opinion on whether that direction makes sense and is worth pursuing, or if the current version is good enough as-is.
I added a property-based test for url_utils as an example (very cool!). I have a few more drafted and will push them once they’re ready.
The UntrustedString wrapper made LSPSAppName and LSPSWebhookUrl more awkward to use (e.g., requiring .0.0 to access the underlying string in some places). I made some adjustments to reduce that friction, but I’m still exploring whether there’s a cleaner solution.
I introduced an LSPS5Error that requires .into() conversions to and from LSPSResponseError when entering and leaving the LSPS5 context. I’m not sure if this is the best approach, but if it is, we might want to apply the same pattern across the other LSPS modules.

ldk-reviews-bot · 2025-04-21T00:00:42Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-04-23T00:01:08Z

🔔 2nd Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

@martinsaposnic This needs a rebase now that #3509 was merged. In particular, you'll need to switch to the new event queue, etc.

Let me know once it's rebased. Will do another full pass then, but should be ready for a second reviewer soon.

- Add 'time' feature flag to allow disabling time-dependent functionality - Include 'time' in default features - Allow users to disable SystemTime::now without disabling all std features - Follows pattern established in other crates (e.g., lightning-transaction-sync) - Improves compatibility with WASM environments

- do new_from_duration_since_epoch (instead of From<Duration>) - Avoid doing ambiguous timestamp types - Add abs_diff function to use on client / service

Adds a new url_utils.rs module that provides: - A lightweight URL parser specialized for LSPS5 webhook validation - An implementation focusing on scheme and host extraction - RFC-compliant scheme validation - Tests for various URL scenarios This implementation allows validating webhook URLs without depending on the external url crate

- Replace String fields with UntrustedString wrapper - Change parse to take owned String and return Err(()) - Simplify error handling, drop free‑form error messages - Add is_https(), url_length(), is_public() helpers - Make url() public, remove scheme()/host() getters - Clean up parsing logic to avoid extra allocations - Update tests to use .to_string() and new API - Add proptest for robust property‑based testing - Remove redundant comments and dead code

- Define LSPS5Request and LSPS5Response enums for webhook registration, listing, and removal. - Implement WebhookNotification and associated helper constructors for different notification types. - Implement serialization/deserialization support with comprehensive tests. - Improve LSPS5 message types, validation, and testing - Replace generic String types with strongly-typed Lsps5AppName and Lsps5WebhookUrl with built-in length and format validation - Restructure imports to follow one-per-line convention - Add constants for notification method strings - Make WebhookNotificationMethod enum more consistent with LSPS5 prefix - Use explicit serde_json::json and serde_json::Value instead of imports - Improve code documentation with proper ticks and references - Add comprehensive test vectors from the BLIP-0055 specification

- Use UntrustedString for LSPS5AppName and LSPS5WebhookUrl - Move URL validation to url_utils for consistency - Use chars().count() for app name length check - Remove jsonrpc field from WebhookNotification struct - Always serialize jsonrpc as 2.0 and Visitor for deserialization - Avoid heap allocations in WebhookNotification deserialization - Update tests for new constructors and error types

- Introduce LSPS5ServiceEvent for LSPS-side webhook events including registration, listing, removal, and notification. - Define LSPS5ClientEvent for handling webhook outcomes on the client (Lightning node) side. - Outline WebhookNotificationParams enum to support notification-specific parameters. - Improve LSPS5 event documentation and field naming - Rename client/lsp fields to counterparty_node_id for consistent terminology - Replace generic String types with more specific Lsps5AppName and Lsps5WebhookUrl - Add comprehensive documentation for all events and fields - Include format specifications (UTF-8, ISO8601) and size constraints - Add request_id field to all relevant events for consistent request tracking - Provide detailed descriptions of error codes and their meanings - Use complete sentences in documentation comments

- Replaces raw error code/message pairs with a structured LSPS5Error type - Changes String timestamp fields to use the proper LSPSDateTime type - Improves documentation with references to MAX_APP_NAME_LENGTH constant

Implements the LSPS5 webhook registration service that allows LSPs to notify clients of important events via webhooks. This service handles webhook registration, listing, removal, and notification delivery according to the LSPS5 specification. Some details: - A generic HttpClient trait is defined so users can provide their own HTTP implementation - A generic TimeProvider trait is defined with a DefaultTimeProvider that uses std functionality - Uses URL utils to validate webhook URLs according to LSPS5 requirements - Uses secure message signing logic from the lightning::util::message_signing module - Works with the events and messages defined in earlier commits - Tests will be provided in a future commit

- Use LSPSDateTime for webhook timestamps for future LDK serialization - Make MIN_WEBHOOK_RETENTION_DAYS and PRUNE_STALE_WEBHOOKS_INTERVAL_DAYS Durations - Remove unnecessary Arc from Mutex fields (webhooks, last_pruning) - Use unwrap() for Mutex locks as per project convention - Move pruning interval to a const - Set last_pruning in prune_stale_webhooks after pruning - Remove unnecessary reassignments and error handling on locks - Use new_with_custom_time_provider for handler construction - Set PROTOCOL_NUMBER to 5 for LSPS5 - Rename _new_with_custom_time_provider to new_with_custom_time_provider - Make notification_cooldown_hours a Duration - Minor doc and formatting improvements

Implements the client-side functionality for LSPS5 webhook registration, allowing Lightning clients to register, list, and remove webhooks with LSPs. This client handler processes responses and verifies webhook notification signatures. Key features: - Full client API for webhook registration operations - Per-peer state tracking for pending requests - Automatic request timeout and cleanup - Security validation for webhook URLs - Notification signature verification - Add store_signature and check_signature to prevent replay attacks - Some tests are provided but more will come in a future commit This implementation pairs with the service-side LSPS5 webhook handler to complete the webhook registration protocol according to the LSPS5 specification.

- Use Duration type consistently for time-related configurations instead of u64 - Replace raw durations with LSPSDateTime for timestamp tracking and comparison - Do unwrap directly on mutex locks - Clean up code organization (imports, visibility modifiers) - Use the new LSPS5Error

Fully integrates the LSPS5 webhook components into the lightning-liquidity framework, enabling usage through the LiquidityManager. It includes - Registering LSPS5 events in the event system - Adding LSPS5 module to the main library exports - Updating LSPS0 serialization to handle LSPS5 messages - Adding LSPS5 configuration options to client and service config structures - Implementing message handling for LSPS5 requests and responses - Adding accessor methods for LSPS5 client and service handlers With this change, LSPS5 webhook functionality can now be accessed through the standard LiquidityManager interface, following the same pattern as other LSPS protocols.

…orrectness of the signing logic

martinsaposnic · 2025-04-28T13:06:04Z

@tnull rebase done!

…derive(Debug)

tnull

Excuse the delay here! I have yet to take another closer look at the tests and parts of the service, but at this point this PR should be ready for another set of eyes.

I'll ping @johncantrell97 as he's familiar with LSPS5 and agreed to review this PR.

Please feel free to squash the fixups.

tnull · 2025-04-29T10:31:20Z

lightning-liquidity/src/events/event_queue.rs

@@ -1,6 +1,5 @@
 use super::LiquidityEvent;
 use crate::sync::{Arc, Mutex};
-


nit: Please leave this line in, it was there for a reason.

tnull · 2025-04-29T10:34:03Z

lightning-liquidity/src/lsps5/service.rs

+
+/// Default time provider using the system clock.
+#[derive(Clone, Debug)]
+#[cfg(feature = "time")]


Hmm, as noted on the TimeProvider PR, I think we should either feature-gate this on all(feature = "std", feature = "time"), or simply have time depend on std in the first place.

tnull · 2025-04-29T10:35:24Z

lightning-liquidity/src/lsps5/service.rs

+	fn default() -> Self {
+		Self {
+			max_webhooks_per_client: DEFAULT_MAX_WEBHOOKS_PER_CLIENT,
+			signing_key: SecretKey::from_slice(&[1; 32]).expect("Static key should be valid"),


I don't think we should provide a default value for signing key. We will probably need to drop the Default implementation alltogether.

tnull · 2025-04-29T10:36:39Z

lightning-liquidity/src/lsps5/service.rs

+	}
+
+	/// Handle a set_webhook request.
+	pub fn handle_set_webhook(


I don't think this should be pub, same with the other internal methods here.

Please also drop the doc comments on the internal methods while you're at it. The comments for the public API methods could be expanded though to include additional context and instructions now how they would be used.

tnull · 2025-04-29T10:43:10Z

lightning-liquidity/src/lsps5/service.rs

+					"Service handler received LSPS5 response message. This should never happen."
+				);
+				Err(LightningError {
+                    err: format!("Service handler received LSPS5 response message from node {:?}. This should never happen.", counterparty_node_id),


nit: Formatting is off here.

tnull · 2025-04-29T10:47:29Z

lightning-liquidity/src/lsps5/client.rs

+					action: ErrorAction::IgnoreAndLog(Level::Error),
+				});
+
+				self.with_peer_state(*counterparty_node_id, |peer_state| {


Phew, can we reduce the indentation here? Maybe at least move the closure to a variable before giving it to with_peer_state? Also, we might be able to avoid the match above by early-aborting in the Request case?

tnull · 2025-04-29T10:49:45Z

lightning-liquidity/src/lsps5/client.rs

+	pub fn verify_notification_signature(
+		&self, counterparty_node_id: PublicKey, signature_timestamp: &LSPSDateTime,
+		signature: &str, notification: &WebhookNotification,
+	) -> Result<bool, LightningError> {


As previously mentioned, I don't think we should expose LightningError as a return value in our public API. Let's rather introduce well-defined error enums instead, probably one for the client and one for the service side.

tnull · 2025-04-29T10:53:13Z

lightning-liquidity/src/manager.rs

+				} else {
+					#[cfg(feature = "time")]
+					{
+						LSPS5ClientHandler::new(


Rather then doing this, let's just go the same way as we did for LSPS5ServiceHandler: add two separate constructors new and new_with_custom_time_provider to LiquidityManager, where the former is gated on time/std and reuses the latter. This would also allow us to require the time_provider field, i.e., drop the Option, no?

tnull · 2025-04-29T10:54:41Z

lightning-liquidity/tests/common/mod.rs

@@ -4,6 +4,7 @@
 #![allow(unused_imports)]
 #![allow(unused_macros)]

+use bitcoin::secp256k1::SecretKey;


nit: Let's move this down to the other bitcoin imports.

tnull · 2025-04-29T11:03:05Z

lightning-liquidity/tests/common/mod.rs

@@ -672,3 +684,52 @@ fn advance_chain(node: &mut Node, num_blocks: u32) {
 		}
 	}
 }
+
+pub(crate) fn get_client_and_service(


This seems specific to LSPS5, so probably should live in lsps5_integration_tests.rs rather than the common mod?

martinsaposnic mentioned this pull request Mar 11, 2025

Add bLIP 55: Webhook Registration (LSPS5) lightning/blips#55

Open

ldk-reviews-bot requested a review from arik-so March 11, 2025 20:22

tnull self-requested a review March 11, 2025 20:37

wpaulino removed the request for review from arik-so March 11, 2025 20:39

tnull reviewed Mar 13, 2025

View reviewed changes

martinsaposnic force-pushed the lsps5 branch from 3aca4fc to 4dbe74d Compare March 14, 2025 14:48

martinsaposnic marked this pull request as draft March 14, 2025 14:49

martinsaposnic force-pushed the lsps5 branch 2 times, most recently from 1d4b47c to edf5346 Compare March 14, 2025 16:31

martinsaposnic marked this pull request as ready for review March 14, 2025 16:31

ldk-reviews-bot requested a review from valentinewallace March 14, 2025 16:32

martinsaposnic requested a review from tnull March 14, 2025 17:12

martinsaposnic force-pushed the lsps5 branch 7 times, most recently from 9809682 to af5929e Compare March 16, 2025 14:24

tnull removed the request for review from valentinewallace March 17, 2025 08:33

martinsaposnic force-pushed the lsps5 branch from af5929e to e018300 Compare March 17, 2025 16:10

martinsaposnic force-pushed the lsps5 branch 3 times, most recently from 29fa657 to 819614e Compare April 18, 2025 15:55

martinsaposnic requested a review from tnull April 18, 2025 18:29

martinsaposnic mentioned this pull request Apr 20, 2025

LSPS2: Add TimeProvider trait for expiry checks in no-std and std builds #3746

Open

tnull reviewed Apr 24, 2025

View reviewed changes

martinsaposnic added 9 commits April 24, 2025 13:10

Prefactor - Improvements on LSPSDateTime

0308483

- do new_from_duration_since_epoch (instead of From<Duration>) - Avoid doing ambiguous timestamp types - Add abs_diff function to use on client / service

fixup: LSPS5 events - Improve type safety and documentation

126c1e8

- Replaces raw error code/message pairs with a structured LSPS5Error type - Changes String timestamp fields to use the proper LSPSDateTime type - Improves documentation with references to MAX_APP_NAME_LENGTH constant

martinsaposnic force-pushed the lsps5 branch from 819614e to 5a165b4 Compare April 24, 2025 16:26

martinsaposnic added 6 commits April 24, 2025 13:51

Add LSPS5 module structure

84c8cd7

Add tests for LSPS5 client and service. Also tests for checking the c…

56d8f26

…orrectness of the signing logic

martinsaposnic force-pushed the lsps5 branch from 5a165b4 to 56d8f26 Compare April 24, 2025 16:51

martinsaposnic requested a review from tnull April 28, 2025 13:05

ldk-node needs both LSPS5ClientConfig and LSPS5ServiceConfig to have …

1b43559

…derive(Debug)

tnull reviewed Apr 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LSPS5 implementation #3662

LSPS5 implementation #3662

martinsaposnic commented Mar 11, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 11, 2025 •

edited

Loading

martinsaposnic commented Mar 11, 2025 •

edited

Loading

codecov bot commented Mar 11, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

ldk-reviews-bot commented Mar 13, 2025

tnull commented Mar 13, 2025 •

edited

Loading

tnull commented Mar 13, 2025

martinsaposnic commented Mar 14, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 17, 2025

ldk-reviews-bot commented Mar 17, 2025

martinsaposnic commented Apr 18, 2025

ldk-reviews-bot commented Apr 21, 2025

ldk-reviews-bot commented Apr 23, 2025

tnull left a comment

martinsaposnic commented Apr 28, 2025

tnull left a comment

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

tnull Apr 29, 2025

		@@ -1,6 +1,5 @@
		use super::LiquidityEvent;
		use crate::sync::{Arc, Mutex};

LSPS5 implementation #3662

Are you sure you want to change the base?

LSPS5 implementation #3662

Conversation

martinsaposnic commented Mar 11, 2025 • edited Loading

ldk-reviews-bot commented Mar 11, 2025 • edited Loading

martinsaposnic commented Mar 11, 2025 • edited Loading

codecov bot commented Mar 11, 2025 • edited Loading

Codecov Report

tnull left a comment • edited Loading

Choose a reason for hiding this comment

ldk-reviews-bot commented Mar 13, 2025

tnull commented Mar 13, 2025 • edited Loading

tnull commented Mar 13, 2025

martinsaposnic commented Mar 14, 2025 • edited Loading

ldk-reviews-bot commented Mar 17, 2025

ldk-reviews-bot commented Mar 17, 2025

martinsaposnic commented Apr 18, 2025

ldk-reviews-bot commented Apr 21, 2025

ldk-reviews-bot commented Apr 23, 2025

tnull left a comment

Choose a reason for hiding this comment

martinsaposnic commented Apr 28, 2025

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinsaposnic commented Mar 11, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 11, 2025 •

edited

Loading

martinsaposnic commented Mar 11, 2025 •

edited

Loading

codecov bot commented Mar 11, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

tnull commented Mar 13, 2025 •

edited

Loading

martinsaposnic commented Mar 14, 2025 •

edited

Loading