Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send hashes in the echo round instead of the full messages #90

Merged
merged 2 commits into from
Feb 10, 2025

Conversation

fjarri
Copy link
Member

@fjarri fjarri commented Feb 1, 2025

Some echo broadcasts in CGGMP'24 include large structures that scale linearly with the number of nodes, so the corresponding echo round messages would scale quadratically. This PR replaces sending full payloads the second time with just sending payload hashes. The trade-offs are:

  • every message needs two hash applications instead of one (hash the payload, then hash (payload hash, metadata))
  • more information to keep in the Evidence (need to keep the echo messages themselves and the signed hashes)

Also fixes a bug with the TestHasher OutputSize and buffer.len() mismatch. Before this PR the created digest was actually ignored when creating TestSignature, so the bug didn't fire.

This approach hashes in two stages when it's technically not necessary (that is, for non-echo messages), but I don't think it leads to any noticeable performance decrease — with most hashes it's the update part that's slow, not the finalize. So the only additional cost is an extra tag pushed into the digest.

@fjarri fjarri self-assigned this Feb 1, 2025
@fjarri fjarri marked this pull request as draft February 1, 2025 00:30
@coveralls
Copy link

coveralls commented Feb 1, 2025

Pull Request Test Coverage Report for Build 13193469924

Details

  • 144 of 158 (91.14%) changed or added relevant lines in 7 files are covered.
  • 3 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+2.7%) to 71.202%

Changes Missing Coverage Covered Lines Changed/Added Lines %
manul/src/protocol/message.rs 17 18 94.44%
manul/src/session/transcript.rs 17 18 94.44%
manul/src/session/evidence.rs 33 39 84.62%
manul/src/session/message.rs 61 67 91.04%
Files with Coverage Reduction New Missed Lines %
manul/src/session/evidence.rs 1 53.0%
manul/src/session/message.rs 1 93.91%
manul/src/protocol/message.rs 1 80.43%
Totals Coverage Status
Change from base Build 13062498128: 2.7%
Covered Lines: 1978
Relevant Lines: 2778

💛 - Coveralls

@fjarri fjarri force-pushed the optimize-echo-broadcast branch 2 times, most recently from 3b6d310 to 6ed1729 Compare February 2, 2025 08:21
@fjarri fjarri marked this pull request as ready for review February 2, 2025 23:08
@fjarri fjarri requested a review from dvdplm February 4, 2025 19:48
Copy link
Contributor

@dvdplm dvdplm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

EDIT: Although I have a question. This optimization is CGGMP-specific and I wonder how well it jives with manul's goal of being a general purpose "round manager" library. I'm fine merging CGGMP specific stuff but it's worth keeping track of where we stray from the general-purpose goal.

manul/src/dev/session_parameters.rs Show resolved Hide resolved
manul/src/session/message.rs Outdated Show resolved Hide resolved
manul/src/session/message.rs Outdated Show resolved Hide resolved
@fjarri
Copy link
Member Author

fjarri commented Feb 7, 2025

This optimization is CGGMP-specific

I think it's a pretty general thing, we don't want to send whole messages the second time. Even without quadratic scaling echos with full messages can get pretty large.

@fjarri
Copy link
Member Author

fjarri commented Feb 7, 2025

In addition to the requested changes, also added the forgotten length prefix when hashing variable-length bytestrings. Tend to forget this kind of stuff because in synedrion it happens automatically.

@fjarri fjarri force-pushed the optimize-echo-broadcast branch from d67250d to 60fec79 Compare February 10, 2025 19:26
@fjarri fjarri merged commit 68d70f8 into entropyxyz:master Feb 10, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants