Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: use fmt.Fprintf to avoid unnecessary string+args + WriteString #3434

Conversation

odeke-em
Copy link
Contributor

@odeke-em odeke-em commented Jan 1, 2025

In a bid to remove unnecessary CPU and memory bloat for the gnovm which takes the order of minutes to run certain code, I noticed the pattern:

io.StringWriter.WriteString(fmt.Sprintf(...))

in which fmt.Sprintf(...) has to create a string by inserting all arguments into the format specifiers then pass that into .WriteString which defeats the entire purpose of io.StringWriter.WriteString that *bytes.Buffer and *strings.Builder implement.

Just from picking a single benchmark that already exists results in improvements in all dimensions:

name               old time/op    new time/op    delta
StringLargeData-8    8.87ms ± 1%    8.28ms ± 3%   -6.68%  (p=0.000 n=17+19)

name               old alloc/op   new alloc/op   delta
StringLargeData-8    8.44MB ± 0%    7.78MB ± 0%   -7.75%  (p=0.000 n=20+19)

name               old allocs/op  new allocs/op  delta
StringLargeData-8     94.1k ± 0%     70.1k ± 0%  -25.51%  (p=0.000 n=15+20)

for heavily used code this is going to reduce on garbage collection cycles too.

Fixes #3433

@github-actions github-actions bot added 📦 🤖 gnovm Issues or PRs gnovm related 📦 🌐 tendermint v2 Issues or PRs tm2 related labels Jan 1, 2025
@Gno2D2
Copy link
Collaborator

Gno2D2 commented Jan 1, 2025

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • IGNORE the bot requirements for this PR (force green CI check)
  • The pull request description provides enough details (checked by @n2p5)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info)

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 The pull request was created from a fork (head branch repo: odeke-em/gno)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission
The pull request description provides enough details

If

🟢 Condition met
└── 🟢 Not (🔴 Pull request author is a member of the team: core-contributors)

Can be checked by

  • team core-contributors

Copy link

codecov bot commented Jan 1, 2025

Codecov Report

Attention: Patch coverage is 90.62500% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
gnovm/pkg/gnolang/machine.go 85.71% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

Copy link
Member

@notJoon notJoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more cleaner than previous version. I left some comments

gnovm/pkg/gnolang/machine.go Outdated Show resolved Hide resolved
gnovm/pkg/gnolang/machine.go Show resolved Hide resolved
@odeke-em odeke-em force-pushed the all-use-fmt.Fprintf-to-avoid-raw-string-concatentation+io.StringWriter.WriteString branch from ff8f2c3 to 932a525 Compare January 2, 2025 03:04
@odeke-em
Copy link
Contributor Author

odeke-em commented Jan 2, 2025

Thank you for the code review and discourse @notJoon, I've made an update, kindly please help me take another stab.

@odeke-em odeke-em requested a review from notJoon January 2, 2025 03:06
Copy link
Member

@notJoon notJoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 💯

@odeke-em odeke-em force-pushed the all-use-fmt.Fprintf-to-avoid-raw-string-concatentation+io.StringWriter.WriteString branch 2 times, most recently from 902ba09 to 6cd03c2 Compare January 2, 2025 03:14
Copy link
Contributor

@n2p5 n2p5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed over video call with @odeke-em , LGTM.

@odeke-em odeke-em force-pushed the all-use-fmt.Fprintf-to-avoid-raw-string-concatentation+io.StringWriter.WriteString branch 6 times, most recently from bfb7245 to f981e89 Compare January 9, 2025 13:44
In a bid to remove unnecessary CPU and memory bloat for the gnovm which
takes the order of minutes to run certain code, I noticed the pattern:

    io.StringWriter.WriteString(fmt.Sprintf(...))

in which fmt.Sprintf(...) has to create a string by inserting all
arguments into the format specifiers then pass that into .WriteString
which defeats the entire purpose of io.StringWriter.WriteString
that *bytes.Buffer and *strings.Builder implement.

Just from picking a single benchmark that already exists results in
improvements in all dimensions:

```shell
$ benchstat before.txt after.txt
name               old time/op    new time/op    delta
StringLargeData-8    8.87ms ± 1%    8.28ms ± 3%   -6.68%  (p=0.000 n=17+19)

name               old alloc/op   new alloc/op   delta
StringLargeData-8    8.44MB ± 0%    7.78MB ± 0%   -7.75%  (p=0.000 n=20+19)

name               old allocs/op  new allocs/op  delta
StringLargeData-8     94.1k ± 0%     70.1k ± 0%  -25.51%  (p=0.000 n=15+20)
```

for heavily used code this is going to reduce on garbage collection
cycles too.

Fixes gnolang#3433
@odeke-em odeke-em force-pushed the all-use-fmt.Fprintf-to-avoid-raw-string-concatentation+io.StringWriter.WriteString branch from f981e89 to b4f6bbb Compare January 9, 2025 14:06
@n2p5
Copy link
Contributor

n2p5 commented Jan 9, 2025

I gave feedback to @odeke-em out-of-band. He's going to sort out the last of the failing tests and we can merge.

@thehowl thehowl merged commit 817e71f into gnolang:master Jan 9, 2025
62 checks passed
@odeke-em odeke-em deleted the all-use-fmt.Fprintf-to-avoid-raw-string-concatentation+io.StringWriter.WriteString branch January 9, 2025 20:19
albttx pushed a commit that referenced this pull request Jan 10, 2025
…3434)

In a bid to remove unnecessary CPU and memory bloat for the gnovm which
takes the order of minutes to run certain code, I noticed the pattern:

    io.StringWriter.WriteString(fmt.Sprintf(...))

in which fmt.Sprintf(...) has to create a string by inserting all
arguments into the format specifiers then pass that into .WriteString
which defeats the entire purpose of io.StringWriter.WriteString that
*bytes.Buffer and *strings.Builder implement.

Just from picking a single benchmark that already exists results in
improvements in all dimensions:

```shell
name               old time/op    new time/op    delta
StringLargeData-8    8.87ms ± 1%    8.28ms ± 3%   -6.68%  (p=0.000 n=17+19)

name               old alloc/op   new alloc/op   delta
StringLargeData-8    8.44MB ± 0%    7.78MB ± 0%   -7.75%  (p=0.000 n=20+19)

name               old allocs/op  new allocs/op  delta
StringLargeData-8     94.1k ± 0%     70.1k ± 0%  -25.51%  (p=0.000 n=15+20)
```

for heavily used code this is going to reduce on garbage collection
cycles too.

Fixes #3433

---------

Co-authored-by: Nathan Toups <[email protected]>
Co-authored-by: Morgan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 🌐 tendermint v2 Issues or PRs tm2 related 📦 🤖 gnovm Issues or PRs gnovm related
Projects
5 participants