-
Couldn't load subscription status.
- Fork 135
Onion messaging support #68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Summary of Changes
Hello @gijswijs, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request significantly extends the lightning-onion package by integrating full support for onion messaging, a distinct type of onion routing. The changes involve fundamental modifications to how onion packets are structured, created, and processed, enabling greater flexibility in payload sizes and strict adherence to TLV encoding for messages. This enhancement is crucial for future-proofing the library and supporting advanced messaging capabilities within the Lightning Network.
Highlights
- Onion Messaging Support: This pull request introduces comprehensive support for 'onion messaging' within the
lightning-onionpackage, distinguishing it from traditional payment onions. This includes new packet structures, creation, and processing logic tailored for onion messages. - Flexible Routing Info Sizes: The fixed-size
RoutingInfofield inOnionPackethas been replaced with a dynamic slice ([]byte), enabling support for variable payload sizes. New constantsStandardRoutingInfoSize(1300 bytes) andJumboRoutingInfoSize(32768 bytes) are introduced to accommodate different onion message lengths, including those exceeding the previous 1300-byte limit. - Payload Type Enforcement: Onion messages are now strictly enforced to use TLV (Type-Length-Value) payloads, aligning with BOLT 7 specifications. The packet creation and processing logic has been updated to validate this, rejecting legacy payloads for onion messages.
- Updated Packet Creation and Processing: The
NewOnionPacketandProcessOnionPacketfunctions now accept functional options (OnionPacketOption,ProcessOnionOpt) to specify if the packet is an onion message. This allows for specialized handling, such as dynamic routing info sizing and payload type validation, based on the packet's purpose. - New Test Vectors and Comprehensive Testing: New test vectors from the BOLT 4 spec for onion messaging have been integrated. Dedicated tests (
TestBuildOnionMessageBlindedRoute,TestOnionMessageRouteBlinding,TestTLVPayloadMessagePacket) have been added to validate blinded route construction, packet processing, and TLV payload handling for onion messages. Tests for zero-length and larger than 1300-byte payloads are also included. - CLI Tool Enhancement: A new
--onion-messageflag has been added to thecmd/main.goCLI tool, allowing users to generate onion message packets directly. - Dependency Updates: Several Go module dependencies, including
btcd,btcec/v2,stretchr/testify, andgolang.org/x/crypto, have been updated to their latest versions. New indirect dependencies likelnd/tlvhave also been added.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
|
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces comprehensive support for onion messaging. The core logic changes are well-implemented. The review focuses on improving the new tests, identifying areas for improvement such as code clarity and test correctness.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces support for onion messaging, including handling of blinded routes and variable-sized payloads. The changes include new logic for packet creation and processing, and are supported by test vectors and new unit tests. I have a few suggestions to improve code clarity and robustness.
6aa8a12 to
dd68d45
Compare
|
cc: @yyforyongyu @ellemouton for review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice tests! Finish one round, mostly on the code format and tests, will do another round soon to load the context.
| @@ -1,2 +1,3 @@ | |||
| vendor/ | |||
| .idea | |||
| .aider* | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: can move .gitognore and go.mod changes into a new commit
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not resolved yet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Second round done. My main comments are,
- the
legacy vs empty tlv streamcheck, and wonder whether it's needed, and if so, can we do it in a higher layer instead. Also wondering if we could drop the support for legacy payload. - code format needs to be fixed.
- need to make it compile, ran linter and got
> golangci-lint run
cmd/main.go:1: : # github.com/gijswijs/lightning-onion/cmd
cmd/main.go:212:25: undefined: sphinx.OnionPacketOption
cmd/main.go:214:40: undefined: sphinx.WithOnionMessage
cmd/main.go:217:14: cannot use ... in call to non-variadic sphinx.NewOnionPacket (typecheck)
package mainI also noticed that we don't have any CI here, maybe a minimal CI that includes jobs like unit test, lint, and check commits would be a good starting point, and it should be easy to add as we can just copy files from lnd. Another PR tho.
sphinx.go
Outdated
|
|
||
| // If this is an onion message, we only expect TLV | ||
| // payloads. | ||
| if cfg.isOnionMessage && isLegacy { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Think this legacy vs empty tlv kinda increases the complexity a lot - I wonder if we need to check it here, like a higher layer should be able to catch and error out there?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ellemouton made a similar remark here: #68 (comment)
NewOnionPacket is exposed, so the higher layer you reference is outside of this library. Do we want to offload those checks to the ones importing this library?
I think those checks make the library more easy to use, because they give useful error messages.
My $0.02.
|
/gemini review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request introduces comprehensive support for onion messaging, including blinded routes and variable-sized onion packets, which is a significant feature addition. The implementation is well-supported by new tests, including vectors from the BOLT specification.
My review focuses on enhancing the code quality. I've suggested improvements to test robustness by replacing ignored errors with explicit checks, and a simplification of some conditional logic to improve readability and maintainability. These suggestions align with general Go best practices for writing clear and robust code.
sphinx.go
Outdated
| routingInfoLen = StandardRoutingInfoSize | ||
| default: | ||
| routingInfoLen = JumboRoutingInfoSize |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the spec says that the length of the packet for onion messages should be either 1366 or 32834 but not that it must. ie, i think you need to support it being other lengths too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, it says should but why support random lengths? This is what the rationale is in the spec.
lenallows larger messages to be sent than the standard 1300 bytes allowed for an HTLC onion, but this should be used sparingly as it reduces the anonymity set, hence the recommendation that it either looks like an HTLC onion, or if larger, be a fixed size.
Let's nudge people in using this correctly and automatically switch to the fixed large size if they cross the standard routing info size threshold.
sphinx.go
Outdated
| // If this is an onion message, a blinding point must be provided. | ||
| if cfg.isOnionMessage && cfg.blindingPoint == nil { | ||
| return nil, fmt.Errorf("blinding point must be provided for " + | ||
| "onion messages") | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and that assocData is empty. But imo, this is a business logic check that should happen at a higher level
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, not sure if I agree. I would say this is distinct from business logic and qualifies as validation logic. It's not really a business policy that you need to supply a blinding point, it's a fundamental building block of onion messages as such. Also, this is a package that people use, and it's helpful to supply them with meaningful error messages.
But we can probably debate this for hours. I'm ok with it either way. Just say the word.
payload.go
Outdated
| func (hp *HopPayload) Decode(r io.Reader, isMessage ...bool) error { | ||
| // To preserve backwards compatibility, we'll default to isMessage being | ||
| // false if it is not provided. | ||
| isMsg := len(isMessage) > 0 && isMessage[0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same comment as previous commit. i dont think we need to do this. Also adding a variadic param here doesnt make sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
afaik, using a variadic argument is idiomatic Go for maintaining backward compatibility. It doesn't break existing callers.
But I guess we're ok with breaking existing callers, so I'll go ahead and remove this.
0220389 to
e4bf9cc
Compare
6d838d2 to
8b3f9c2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the updates!! looking good!
sphinx.go
Outdated
| return nil, ErrMaxRoutingInfoSizeExceeded | ||
| } | ||
| assocData []byte, pktFiller PacketFiller, | ||
| isOnionMessage bool) (*OnionPacket, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i still think this leaks business logic into this layer.
Saw your previous comment about how we potentially have different definitions for this. All I mean is that logic from the layer above about how this is specifically is being used is leaking into this layer now. Imo, this layer should only be about wrapping & unwrapping an onion and so any new things/options we expose here shoudl just be knobs that above layers can pull in order to tweak how things are being wrapped/unwrapped.
So for example, here: really all we are letting the above caller do is 1) change the default payload size 2) force strong enforcement that any hop payloads have TLV type only.
So i'd probs go for something more abstract that would include adding new functional options like WithMaxPayloadSize(x) and WithTLVOnly() and then if the caller is using NewOnionPacket to construct an onion packet, then they'd make use of these two new types.
That at least keeps OnionPacket usage agnostic.
If we wanted some onion message logic in this repo, we can then simply add a wrapper to OnionPacket which adds these new functional option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
keen to hear what @yyforyongyu thinks too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I think this generalization makes sense. And I remember a previous version had opts ...OnionPacketOption) and built a WithOnionMessage(). Compared to Elle's approach, the old flag (and this bool) passes context down, forcing the onion package to interpret the context and make decisions. By using WithMaxPayloadSize and WithTLVOnly, this package now becomes agnostic; the abstraction is based on what properties OnionPacket provides - for instance in the future we may have another msg that requires the num of onion wrappers then we can just add WithNumWrappers.
That being said, the blindingPoint, or even the legacy payload also fits this context-leaking case, but that's pre-existing.
So yeah agree that would be a better design, but also happy to merge as-is and refactor it in another PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for clarifying your definition of business logic. Now I understand better where you coming from. I've adjusted the code to reflect the abstraction you requested. I went with the functional option WithTLVPayloadOnly() and a variadic argument in NewOnionPacket that allows for passing the size(s) of the final routing info. The sizes passed are then compared to the actual payload size of the
entire path, and the first value that fits the actual payload size will then be used as the size of the routing info. We use this to fix the size of onion messages at 1300 or 32768 bytes as suggested by BOLT-0004
but it can be used to fix the size at any value. If no values are passed the func defaults to MaxRoutingPayloadSize.
As @@yyforyongyu also notes, it's nearly impossible to rid the library of all context-leaking. Apart from his example you could also say that MaxRoutingPayloadSize and MaxOnionMessagePayloadSize are magic numbers that are leaked from the layer above, since wrapping and unwrapping don't require any specific total payload size. I've (kept) those two vars exposed to facilitate easy usage of this library.
But apart from that var name (MaxOnionMessagePayloadSize) onion messaging isn't mentioned in the library code anywhere. It is (obviously) mentioned in the tests tho.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One question about the go.mod file, otherwise LGTM!
sphinx.go
Outdated
| return nil, ErrMaxRoutingInfoSizeExceeded | ||
| } | ||
| assocData []byte, pktFiller PacketFiller, | ||
| isOnionMessage bool) (*OnionPacket, error) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah I think this generalization makes sense. And I remember a previous version had opts ...OnionPacketOption) and built a WithOnionMessage(). Compared to Elle's approach, the old flag (and this bool) passes context down, forcing the onion package to interpret the context and make decisions. By using WithMaxPayloadSize and WithTLVOnly, this package now becomes agnostic; the abstraction is based on what properties OnionPacket provides - for instance in the future we may have another msg that requires the num of onion wrappers then we can just add WithNumWrappers.
That being said, the blindingPoint, or even the legacy payload also fits this context-leaking case, but that's pre-existing.
So yeah agree that would be a better design, but also happy to merge as-is and refactor it in another PR.
8b3f9c2 to
dcb8d6d
Compare
|
|
||
| // Return an error if we couldn't find a suitable payload size. | ||
| if !found { | ||
| return nil, ErrPayloadSizeExceeded |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should either go with WithMaxPayloadSize or revert to the previous version - the current approach creates more complexity than necessary - there shouldn't be more than one payload size being used inside the same onion packet, by taking a list of sizes it also creates a confusing API.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I disagree. See comment: #68 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, you don't need to pass multiple payload sizes, you can pass one if you want, and it would work just fine. You can even pass zero sizes and it defaults to maximum htlc routing payload size. That way you get backwards compatibility for free in the sense that the call sites don't need to be changed anywhere where we are working with htlc routing onions.
You can also pass a "funky" payload size. That way it supports other payload sizes, which was a fair comment made by @ellemouton (#68 (comment))
All in all I thought it to be quite elegant...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From our mighty Gemini,
Summary of NewOnionPacket API Design Review
The core of the discussion revolves around the best way to modify the NewOnionPacket function to support the creation of variable-sized onion packets, a requirement for the new onion messaging feature.
The Author's Approach: Variadic payloadSizes ...int
The author implemented a solution where the function accepts a variadic list of integers representing acceptable payload sizes.
-
Intent: To create a flexible and backward-compatible API. The goal was for the function to intelligently select the most efficient payload size from a list of supported sizes provided by the caller, defaulting to the legacy behavior if no sizes are given. This is seen as an elegant, self-contained solution.
-
Pros:
- Backward Compatibility: Existing function calls do not need to be changed; they will default to the standard
MaxRoutingPayloadSize. - Convenience: The caller can provide multiple potential sizes (e.g., standard and jumbo), and the library automatically handles the logic of picking the smallest one that fits the payload.
- Flexibility: It naturally supports any custom payload size.
- Backward Compatibility: Existing function calls do not need to be changed; they will default to the standard
-
Cons:
- Leaky Abstraction: The internal logic of sorting and selecting a size leaks application-level policy (e.g., "prefer the smallest possible onion") into a lower-level library.
- Confusing API: The signature
...intis ambiguous. An onion packet has only one final size, so it's not intuitive why a user would provide a list. This increases the cognitive load on the developer. - Critical Flaw - Loss of User Intent: The "pick the smallest fit" logic can silently violate the caller's intent. If a caller wants to create a jumbo
MaxOnionMessagePayloadSizepacket for policy reasons (even with a small payload), the library will incorrectly choose the smallerMaxRoutingPayloadSize, leading to unexpected behavior and subtle bugs.
The Reviewer's Proposed Approach: Functional Options
The reviewer proposed refactoring the function to use the functional option pattern, for example, by accepting ...OnionPacketOption and introducing a new option like WithPayloadSize(int).
-
Intent: To create a clear, explicit, and robust API that adheres to the principle of separation of concerns. The library should provide simple, orthogonal "knobs" for the caller to configure, while the caller remains responsible for all policy decisions.
-
Pros:
- Clear and Unambiguous: The caller's intent is stated explicitly (
WithPayloadSize(MaxOnionMessagePayloadSize)). There is no hidden logic or ambiguity. - Preserves User Intent: The library does exactly what it's told. If the caller requests a jumbo packet, it will create a jumbo packet or fail if the payload is too large.
- Prevents Incorrect Usage: The API design makes it impossible to specify multiple, conflicting sizes for a single packet.
- Extensible and Composable: This pattern is clean and easy to extend with new options in the future without adding complexity.
- Clear and Unambiguous: The caller's intent is stated explicitly (
-
Cons:
- Slightly More Verbose: The caller must explicitly add the functional option at the call site rather than relying on default behavior.
Recommendation
While the author's approach is clever, its cons—particularly the critical flaw of silently ignoring the user's intent—introduce a significant risk of hard-to-debug errors. The goal of a library's API should be predictability and clarity.
I strongly recommend adopting the reviewer's proposed approach.
Refactoring to use a functional option like WithPayloadSize(int) will result in a more robust, maintainable, and less error-prone API. It properly separates the policy (decided by the caller) from the mechanism (executed by the library), which is a cornerstone of good software design. The minor cost of verbosity is heavily outweighed by the significant gains in clarity and correctness.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very much agree with YY
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agree with YY's conclusion here as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, this is not a hill I'm willing to die on. I created the functional option WithMaxPayloadSize.
This does mean that when calling into NewOnionPacket the caller has to choose upfront the payload size based on the size of the route. Otherwise when the caller doesn't and tries to encode a huge route it won't fit into the MaxRoutingPayloadSize. You can see this in the test here: https://github.com/gijswijs/lightning-onion/blob/3c7725ddfcf22339650c624f04e75707579e4a35/sphinx_test.go#L278-L286
For now I'll bask in the fuzzy glow of Gemini calling me clever!
While the author's approach is clever
payload.go
Outdated
| if legacyPayload { | ||
| // If the HopPayload isn't guaranteed to be a TLV payload, we check the | ||
| // first byte to see if it is a legacy payload. | ||
| if hp.Type != PayloadTLV && isLegacyPayloadByte(peekByte[0]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think this check changed from the previous iteration? i think previously we were passing in a new variable to Decode but then we agreed that it was something that was inherint to HopPayload and so should be set as a new field on the HopPayload at construction time? Something along the lines of "AllowZeroPayload" or something.
Noting this cause i think the if hp.Type !=PayloadTLV { hp.Type=PayloadLegacy } is strange cause i think it means that that assignment never actually does anything
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting. It has always been a no-op even in 6afc43f. PayloadLegacy is the zero-value for Type and doesn't need to be explicitly set. It just sticks out more because of the changed if-statement.
Great catch.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it still reads strangely to me cause we both check the Type here but then again set it below in the else block?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also don't understand it, why do we also check isLegacyPayloadByte? Don't we trust the Type ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it still reads strangely to me cause we both check the Type here but then again set it below in the else block?
Before onion message support the first byte would be the arbitrator of what type of payload we were dealing with:
- If zero ->
PayloadLegacy - Else ->
PayloadTLV
With onion message support that doesn't hold true, because we now also support zero-length payloads for PayloadTLV.
So we can't always rely anymore on the first byte being the indicator for the type of payload.
So if we know beforehand (in the case of onion_message_packet) that we are dealing with PayloadTLV, we explicitly set the new HopPayload variable to have a type of PayloadTLV before we call into Decode, that way the check on the first byte is skipped. But in the case of an onion_routing_packet at Decode time we still don't know the Type (It's still at it's zero-value: PayloadLegacy), so if the first byte isn't zero, we explicitly have to set the Type to PayloadTLV.
I also don't understand it, why do we also check isLegacyPayloadByte? Don't we trust the Type ?
I think the text above explains that in the case of onion_routing_packet we don't "trust" the type, because we still need to deduce the type and that's where isLegacyPayloadByte comes in.
I did change the code and the comments a bit for further clarification.
|
|
||
| // Return an error if we couldn't find a suitable payload size. | ||
| if !found { | ||
| return nil, ErrPayloadSizeExceeded |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agreed!
payload.go
Outdated
| if legacyPayload { | ||
| // If the HopPayload isn't guaranteed to be a TLV payload, we check the | ||
| // first byte to see if it is a legacy payload. | ||
| if hp.Type != PayloadTLV && isLegacyPayloadByte(peekByte[0]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it still reads strangely to me cause we both check the Type here but then again set it below in the else block?
| hopPayload := HopPayload{} | ||
| if tlvPayloadOnly { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rather create a constructor that takes tlvPayloadOnly as a param?
that way we also explicitly set the Legacy type. We should not depend on the fact that it happens to be the zero value
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 for a constructor, I also think we do not really need this one, and it is not really clear by the name that this only relates to onion_message ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I like the constructor also better. I didn't go for it exactly for what Elle says here: "that way we also explicitly set the Legacy type."
The point is, when unwrapping an onion_routing_packet, we don't know yet if this is explicitly a legacy packet. It might be, but that's what we find out during Decode.
So "explicitly" setting it to legacy here, doesn't mean it actually "is" a legacy payload. We only know that after inspecting the first byte.
Having an extra field in HopPayload could be a solution. Something like ZeroLengthAllowed or TLVGuaranteed.
See also my explanation here: #68 (comment)
not really clear by the name that this only relates to onion_message ?
@ziggie1984 In an earlier phase of reviewing I removed most references to onion messaging, because of a discussion about business logic seeping in: #68 (comment)
|
|
||
| // Return an error if we couldn't find a suitable payload size. | ||
| if !found { | ||
| return nil, ErrPayloadSizeExceeded |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very much agree with YY
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking good 💪 !
I think we need to be very strict with the wording of the spec. otherwise it will be very hard to follow the logic. The wording in general should use the wording of the BOLT04.
Let's bring this one over the finish line.
| hopPayload := HopPayload{} | ||
| if tlvPayloadOnly { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 for a constructor, I also think we do not really need this one, and it is not really clear by the name that this only relates to onion_message ?
|
|
||
| // Return an error if we couldn't find a suitable payload size. | ||
| if !found { | ||
| return nil, ErrPayloadSizeExceeded |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
agree with YY's conclusion here as well
|
@gijswijs, remember to re-request review from reviewers when ready |
This commit removes an unused var and changes bytes.Compare to the idiomatic bytes.Equal.
This commit adds the spec test vector for blinded onion messages. It also adds a test that tests BuildBlindedRoute, decryptBlindedHopData and NextEphemeral against this vector.
We add TestOnionMessageRouteBlinding which verifies that the onion message packet from the test vector can be processed correctly by the nodes in a blinded route.
TestTLVPayloadMessagePacket creates a onion message with payload and the blinded route from the test vector. It then checks if the onion packet we create is equal to the one provided in the test vector.
Since the onion message payload can be zero-length, we need to decode it correctly. This commit adds a boolean flag to the HopPayload Decode that tells whether the payload is an onion message payload or not. If it is, the payload is decoded as a tlv payload also if the first byte is 0x00. sphinx_test: Add zero-length payload om test
Onion messages allow for payloads that exceed 1300 bytes, in which case the payload should become 32768 bytes. This commit introduces support for those custom size packets and the tests for this feature. NewOnionPacket now allows for a final variadic argument payloadSizes. The sizes passed are then compared to the actual payload size of the entire path, and the first value that fits the actual payload size will then be used as the size of the routing info. We use this to fix the size of onion messages at 1300 or 32768 bytes as suggested by BOLT-0004 but it can be used to fix the size at any value. If no values are passed the func defaults to MaxRoutingPayloadSize. MaxRoutingPayloadSize and MaxOnionMessagePayloadSize are exposed to facilitate easy usage of this library. sphinx_test now has a helper function to create onion messages of a specified length. This helper is then used to test the handling of packets larger than 1300 bytes specifically for onion messages.
3c7725d to
75b1451
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made a bunch of changes. All review comments are either addressed in code or have a reply/answer from me.
I kept all changes in fixup-commits so that it's easier to see what has changed since the last round of reviews. Obviously I'll squash those in a final rebase.
There's just one thing that I'm not fully happy with and that's the use of hopPayload.Type to convey that a payload is guaranteed to be tlv. I think an extra field there would be better, but I would like to hear your thoughts on that @ellemouton and @ziggie1984.
| NextNodeID string `json:"next_node_id"` | ||
| NextPathKeyOverride string `json:"next_path_key_override"` | ||
| PathKeyOverrideSecret string `json:"path_key_override_secret"` | ||
| PathID string `json:"path_id"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's intentional. We don't want to pull in the tlv package into this library and to fully test those fields you would need that package to extract those from the tlv stream.
That being said, they are effectively being tested:
"tlvs": {
"next_node_id": "027f31ebc5462c1fdce1b737ecff52d37d75dea43ce11c74d25aa297165faa2007",
"unknown_tag_561": "123456"
},
"encrypted_data_tlv": "0421027f31ebc5462c1fdce1b737ecff52d37d75dea43ce11c74d25aa297165faa2007fd023103123456",
...
"tlvs": {
"padding": "",
"path_id": "deadbeefbadc0ffeedeadbeefbadc0ffeedeadbeefbadc0ffeedeadbeefbadc0",
"unknown_tag_65535": "06c1"
},
"encrypted_data_tlv": "01000620deadbeefbadc0ffeedeadbeefbadc0ffeedeadbeefbadc0ffeedeadbeefbadc0fdffff0206c1",
You can see those fields sitting in the encrypted_data_tlv and we encrypt and decrypt them and see if they come back as expected. The higher level handling of it all will be tested in lnd when we pull in this version of lightning-onion.
Pull Request Test Coverage Report for Build 18812626550Details
💛 - Coveralls |
This PR adds support for onion messaging to the
lightning-onionpackage. It does the following things:TestBuildOnionMessageBlindedRoutethat tests BuildBlindedRoute, decryptBlindedHopData and NextEphemeral against this vector.TestOnionMessageRouteBlindingwhich verifies that the onion message packet from the test vector can be processed correctly by the nodes in a blinded route.TestTLVPayloadMessagePacketthat creates a onion message with payload and the blinded route from the test vector.It also adds support for zero-length payloads and longer than 1300 bytes payloads that are specifically supported by the spec, but not tested by the test vectors supplied. Both features are accompanied by their own unit tests.