-
Notifications
You must be signed in to change notification settings - Fork 371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cleanup PackageTemplate
a bit
#3297
Cleanup PackageTemplate
a bit
#3297
Conversation
1668dbe
to
04c46a5
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3297 +/- ##
==========================================
+ Coverage 89.62% 89.99% +0.37%
==========================================
Files 127 127
Lines 103517 107521 +4004
Branches 103517 107521 +4004
==========================================
+ Hits 92780 96768 +3988
- Misses 8041 8145 +104
+ Partials 2696 2608 -88 ☔ View full report in Codecov by Sentry. |
lightning/src/chain/package.rs
Outdated
/// | ||
/// This is an important limit for aggregation as after this height our counterparty may be | ||
/// able to pin transactions spending this output in the mempool. | ||
pub(crate) fn counterparty_spendable_height(&self) -> u32 { | ||
self.soonest_conf_deadline |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we can rename the variable as well?
I was also confused why it was not an Option
, aren't there cases where the counterparty might be able to claim it immediately? However, the build_package
calls don't seem to discriminate between those.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's always some height at which a counterparty could claim an output, assuming they have the cryptographic material with which to do so, though at least for our outputs maybe we shouldn't set one (because our counterparty can only claim if its a stale output).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But, yes, the variable should be renamed, added two commits to clean more stuff up....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's always some height at which a counterparty could claim an output, assuming they have the cryptographic material with which to do so, though at least for our outputs maybe we shouldn't set one (because our counterparty can only claim if its a stale output).
True, as in 0
or current_height
for outputs without any CSV/CLTV locks? I was wondering if it was clearer to talk about the presence of any of those locks, but it's not really necessary.
I was also confused because the build_package
call for holder HTLC outputs seems to always set counterparty_spendable_height
to htlc.cltv_expiry
, regardless if it was offered/received:
rust-lightning/lightning/src/chain/channelmonitor.rs
Lines 3668 to 3689 in cdd1298
let htlc_output = if htlc.offered { | |
let htlc_output = HolderHTLCOutput::build_offered( | |
htlc.amount_msat, htlc.cltv_expiry, self.onchain_tx_handler.channel_type_features().clone() | |
); | |
htlc_output | |
} else { | |
let payment_preimage = if let Some(preimage) = self.payment_preimages.get(&htlc.payment_hash) { | |
preimage.clone() | |
} else { | |
// We can't build an HTLC-Success transaction without the preimage | |
continue; | |
}; | |
let htlc_output = HolderHTLCOutput::build_accepted( | |
payment_preimage, htlc.amount_msat, self.onchain_tx_handler.channel_type_features().clone() | |
); | |
htlc_output | |
}; | |
let htlc_package = PackageTemplate::build_package( | |
holder_tx.txid, transaction_output_index, | |
PackageSolvingData::HolderHTLCOutput(htlc_output), | |
htlc.cltv_expiry, conf_height | |
); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, as in 0 or current_height for outputs without any CSV/CLTV locks? I was wondering if it was clearer to talk about the presence of any of those locks, but it's not really necessary.
Yea, it might be. I think its really only for our to_self output, and I'm not sure about changing it all for that...
I was also confused because the build_package call for holder HTLC outputs seems to always set counterparty_spendable_height to htlc.cltv_expiry, regardless if it was offered/received:
Ugh, yea, that's a bug too....this code is a mess.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
lightning/src/chain/package.rs
Outdated
/// | ||
/// This is an important limit for aggregation as after this height our counterparty may be | ||
/// able to pin transactions spending this output in the mempool. | ||
pub(crate) fn counterparty_spendable_height(&self) -> u32 { | ||
self.soonest_conf_deadline |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's always some height at which a counterparty could claim an output, assuming they have the cryptographic material with which to do so, though at least for our outputs maybe we shouldn't set one (because our counterparty can only claim if its a stale output).
True, as in 0
or current_height
for outputs without any CSV/CLTV locks? I was wondering if it was clearer to talk about the presence of any of those locks, but it's not really necessary.
I was also confused because the build_package
call for holder HTLC outputs seems to always set counterparty_spendable_height
to htlc.cltv_expiry
, regardless if it was offered/received:
rust-lightning/lightning/src/chain/channelmonitor.rs
Lines 3668 to 3689 in cdd1298
let htlc_output = if htlc.offered { | |
let htlc_output = HolderHTLCOutput::build_offered( | |
htlc.amount_msat, htlc.cltv_expiry, self.onchain_tx_handler.channel_type_features().clone() | |
); | |
htlc_output | |
} else { | |
let payment_preimage = if let Some(preimage) = self.payment_preimages.get(&htlc.payment_hash) { | |
preimage.clone() | |
} else { | |
// We can't build an HTLC-Success transaction without the preimage | |
continue; | |
}; | |
let htlc_output = HolderHTLCOutput::build_accepted( | |
payment_preimage, htlc.amount_msat, self.onchain_tx_handler.channel_type_features().clone() | |
); | |
htlc_output | |
}; | |
let htlc_package = PackageTemplate::build_package( | |
holder_tx.txid, transaction_output_index, | |
PackageSolvingData::HolderHTLCOutput(htlc_output), | |
htlc.cltv_expiry, conf_height | |
); |
fd97a6b
to
e46b1a4
Compare
@@ -3598,7 +3607,7 @@ impl<Signer: EcdsaChannelSigner> ChannelMonitorImpl<Signer> { | |||
self.counterparty_commitment_params.counterparty_htlc_base_key, | |||
htlc.clone(), self.onchain_tx_handler.channel_type_features().clone())) | |||
}; | |||
let counterparty_package = PackageTemplate::build_package(commitment_txid, transaction_output_index, counterparty_htlc_outp, htlc.cltv_expiry, 0); | |||
let counterparty_package = PackageTemplate::build_package(commitment_txid, transaction_output_index, counterparty_htlc_outp, htlc.cltv_expiry); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the counterparty_spendable_height
be adjusted here as well based on whether the htlc is offered or not?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ideally yea, but passing the current height through to here is very annoying, and really it doesn't matter - if its a timeout claim our counterparty can claim it now, but we can't so having a counterparty_claimable_height of when we can both claim it is fine.
commitment_txid, | ||
transaction_output_index, | ||
PackageSolvingData::RevokedHTLCOutput(revk_htlc_outp), | ||
htlc.cltv_expiry, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should the counterparty_spendable_height
be adjusted here as well based on whether the htlc is offered or not?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, sorry, fought with this for a while and think we should do it in a separate PR, it causes a bunch of test changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good to track this feedback somewhere
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
e46b1a4
to
7284666
Compare
let timer_for_target_conf = |target_conf| -> u32 { | ||
if target_conf <= current_height + MIDDLE_FREQUENCY_BUMP_INTERVAL { | ||
current_height + HIGH_FREQUENCY_BUMP_INTERVAL | ||
} else if target_conf <= current_height + LOW_FREQUENCY_BUMP_INTERVAL { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if target_conf is lower than current_height + LOW_FREQUENCY_BUMP_INTERVAL
, isn't it necessarily also lower than current_height + MIDDLE_FREQUENCY_BUMP_INTERVAL
? So can the second clause even ever trigger?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ha, indeed, it cannot. Looks like this is an existing bug.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just to double-check, this is meant to be fixed in a followup, not here, right? If so, on my end the PR looks ready.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, my initial patch was wrong and mislead me into thinking updating tests was easy, but it breaks 9 tests and we should land this PR...The issue dates back to 757bcc2 but also the logic here isn't just broken, its also much too naive, we should rethink it entirely.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added to #3372
7284666
to
544d5ce
Compare
I'm sure you already noticed, but CI's failing :/ |
544d5ce
to
5be3cde
Compare
I had not, thanks. Fixed. |
@@ -3657,7 +3666,7 @@ impl<Signer: EcdsaChannelSigner> ChannelMonitorImpl<Signer> { | |||
// Returns (1) `PackageTemplate`s that can be given to the OnchainTxHandler, so that the handler can | |||
// broadcast transactions claiming holder HTLC commitment outputs and (2) a holder revokable | |||
// script so we can detect whether a holder transaction has been seen on-chain. | |||
fn get_broadcasted_holder_claims(&self, holder_tx: &HolderSignedTx, conf_height: u32) -> (Vec<PackageTemplate>, Option<(ScriptBuf, PublicKey, RevocationKey)>) { | |||
fn get_broadcasted_holder_claims(&self, holder_tx: &HolderSignedTx, _conf_height: u32) -> (Vec<PackageTemplate>, Option<(ScriptBuf, PublicKey, RevocationKey)>) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: are you keeping conf_height
around for use in this function in the future ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, later in my current patchset it is used.
5be3cde
to
fac44b8
Compare
lightning/src/chain/package.rs
Outdated
// Block height at which our counterparty can potentially claim this output as well (assuming | ||
// they have the keys or information required to do so). | ||
// For HTLCs we're claiming with a preimage and revoked outputs, we need to get our spend | ||
// on-chain before this height. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be helpful to retain the reference to OnchainTxHandler::get_height_timer
from the previous comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This value is now not about the fee-bumping, though? get_height_timer
uses this value in a few places because its an easy way to figure out the target, but this value is really about the package merging decision based on if this output is potentially pinnable by our counterparty.
commitment_txid, | ||
transaction_output_index, | ||
PackageSolvingData::RevokedHTLCOutput(revk_htlc_outp), | ||
htlc.cltv_expiry, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be good to track this feedback somewhere
@@ -1038,7 +1038,7 @@ impl PackageTemplate { | |||
soonest_conf_deadline, | |||
aggregable, | |||
feerate_previous: 0, | |||
height_timer: first_bump, | |||
height_timer: 0, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
next_fee_bump_height
seems like a more descriptive name than height_timer
, though if we're setting it to 0 that's a bit inaccurate. It seems like the docs for height_timer
could use a note that it can be 0 as well. It looks like the "timer" phrasing is used in a lot of places right now, though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, there's still more cleanup that should probably happen here, but this PR already kinda got away from me, and there's at least two followups pending...
); | ||
}, | ||
PackageSolvingData::HolderHTLCOutput(outp) if outp.preimage.is_some() => { | ||
// We have the same deadline here as for `CounterpartyOfferedHTLCOutput`. Note |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment is a bit confusing as we seem to calculate the deadline differently for CounterpartyOfferedHTLCOutput
s, i.e. using outp.htlc.cltv_expiry
vs self.soonest_conf_deadline
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The following sentence explains why - outp.cltv_expiry
is actually always 0 due to an unrelated bug, so we can't rely on it :(
let htlc_output = HolderHTLCOutput::build_offered( | ||
htlc.amount_msat, htlc.cltv_expiry, self.onchain_tx_handler.channel_type_features().clone() | ||
); | ||
htlc_output | ||
(htlc_output, conf_height) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing test coverage for this change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, several of the "fix the value we pass" commits don't have test coverage because we don't rely on the value being the new correct one (and can't, old monitors may still be around which don't have the corrected value).
fac44b8
to
630bc52
Compare
630bc52
to
ba0ac67
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm good with squashing this
We don't actually care if a confirmed transaction claimed other outputs, only that it claimed a superset of the outputs in the pending claim we're looking at. Thus, the variable to detect that is renamed `is_claim_subset_of_tx` instead of `are_sets_equal`.
This function was very confusing - its used to determine by when we have to stop aggregating this claim with others as it starts to be at risk of pinning due to the counterparty's ability to spend the output. It is not ever used as a timelock for a transaction, and thus its name is very confusing. Instead we rename it `counterparty_spendable_height`.
ba0ac67
to
58d0ca3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think basically landable on my end, I'm good with continuing to squash
// `CounterpartyReceivedHTLCOutput` | ||
height_timer = cmp::min( | ||
height_timer, | ||
timer_for_target_conf(outp.cltv_expiry + MIN_CLTV_EXPIRY_DELTA as u32), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing test coverage for the added + MIN_CLTV_EXPIRY_DELTA
here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm? If I drop the MIN_CLTV_EXPIRY_DELTA
in the PackageSolvingData::HolderHTLCOutput
(without a preimage
) I see three tests fail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, I meant to leave this comment on PackageSolvingData::CounterpartyReceivedHTLCOutput
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added test coverage.
// We have the same deadline for holder timeout claims as for | ||
// `CounterpartyReceivedHTLCOutput` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible that this isn't a holder timeout claim, but instead a preimage claim where we haven't received the preimage yet (but might in the future)? Wondering if it's possible that outp.cltv_expiry
would be 0 here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By the time we turn a spend into a PackageTemplate
we have some ability to claim it. If the HTLC is inbound (ie its possible to claim with a preimage) we can only end up here with the preimage set.
This has never been used, and its set to a fixed value of zero for HTLCs on local commitment transactions making it impossible to rely on so might as well remove it.
Now that we don't store the confirmation height of the inputs being spent, passing the current height to `PackageTemplate::build_package` is useless - we only use it to set the height at which we should next bump the fee, but we just want it to be "next block", so we might as well use `0` and avoid the extra argument. Further, in one case we were already passing `0`, so passing the argument is just confusing as we can't rely on it being set. Note that this does remove an assertion that we never merge packages that were crated at different heights, and in the future we may wish to do that (as there's no specific reason not to), but we do not currently change the behavior.
`PackageTemplate::get_height_timer` is used to decide when to next bump our feerate on claims which need to make it on chain within some window. It does so by comparing the current height with some deadline and increasing the bump rate as the deadline approaches. However, the deadline used is the `counterparty_spendable_height`, which is the height at which the counterparty might be able to spend the same output, irrespective of why. This doesn't make sense for all output types, for example outbound HTLCs are spendable by our counteraprty immediately (by revealing the preimage), but we don't need to get our HTLC timeout claims confirmed immedaitely, as we actually have `MIN_CLTV_EXPIRY` blocks before the inbound edge of a forwarded HTLC becomes claimable by our (other) counterparty. Thus, here, we adapt `get_height_timer` to look at the type of output being claimed, and adjust the rate at which we bump the fee according to the real deadline.
This renames the field in `PackageTemplate` which describes the height at which a counterparty can make a claim to an output to match its actual use. Previously it had been set based on when a counterparty can claim an output but also used for other purposes. In the previous commit we cleaned up its use for fee-bumping-rate, so here we can rename it as it is now only used as the `counteraprty_spendable_height`.
58d0ca3
to
158005c
Compare
lightning/src/chain/package.rs
Outdated
height_timer = | ||
cmp::min(height_timer, current_height + HIGH_FREQUENCY_BUMP_INTERVAL); | ||
height_timer = current_height + HIGH_FREQUENCY_BUMP_INTERVAL; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This fixup should've been applied to a prior commit
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually, I'm just gonna back this change out. We're iterating a loop and trying to find the minimum, so its weird that sometimes we don't call cmp::min
, even if its the case that we're setting the minimum value in that case.
For outbound HTLCs, the counterparty can spend the output immediately. This fixes the `counterparty_spendable_height` in the `PackageTemplate` claiming outbound HTLCs on local commitment transactions, which was previously spuriously set to the HTLC timeout (at which point *we* can claim the HTLC).
In a previous commit we updated the fee-bump-rate of claims against HTLC timeouts on counterparty commitment transactions so that instead of immediately attempting to bump every block we consider the fact that we actually have at least `MIN_CLTV_EXPIRY_DELTA` blocks to do so, and bumping at the appropriate rate given that. Here we test that by adding an extra check to an existing test that we do not bump in the very next block after the HTLC timeout claim was initially broadcasted.
cb27257
to
20db790
Compare
Digging into better aggregation turned up a bunch of nonsense in
PackageTemplate
that is gonna lead to us breaking things in the future, so since I actually understand how the hell it works I figured I should clean it up a bit so someone else can understand it.