Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark memory #808

Merged
merged 5 commits into from
Jan 22, 2025
Merged

Benchmark memory #808

merged 5 commits into from
Jan 22, 2025

Conversation

jprendes
Copy link
Collaborator

@jprendes jprendes commented Jan 21, 2025

This PR adds a memory usage benchmark to CI.

Fixes #802 , at least for the single container use case.

The PR is based on #775, I'll rebase once that is merged.
Marking as draft until then.

Signed-off-by: Jorge Prendes <[email protected]>
Signed-off-by: Jorge Prendes <[email protected]>
Signed-off-by: Jorge Prendes <[email protected]>
@jprendes jprendes marked this pull request as ready for review January 21, 2025 22:34
@jprendes jprendes requested review from Mossaka and andreiltd January 21, 2025 22:34
@jprendes
Copy link
Collaborator Author

Here's how the benchmark shows in CI
https://github.com/containerd/runwasi/actions/runs/12904819293/job/35982768369?pr=808#step:7:61

github-action-benchmark was run successfully! 
Data: {
  commit: {
    author: { name: 'jprendes', username: 'jprendes' },
    committer: { name: 'jprendes', username: 'jprendes' },
    id: '4a89cedbb35496fbab8cae5fdf2d239cf6e1aeaf',
    message: 'Benchmark memory',
    timestamp: '2025-01-07T09:23:28Z',
    url: 'https://github.com/containerd/runwasi/pull/808/commits/4a89cedbb35496fbab8cae5fdf2d239cf6e1aeaf'
  },
  date: 1737537142444,
  tool: 'customSmallerIsBetter',
  benches: [
    {
      name: 'wamr/memory-usage',
      value: 16044,
      unit: 'kB',
      range: undefined,
      extra: 'shim: 10624 kB\nzygote: 5420 kB'
    },
    {
      name: 'wasmedge/memory-usage',
      value: 67004,
      unit: 'kB',
      range: undefined,
      extra: 'shim: 51968 kB\nzygote: 15036 kB'
    },
    {
      name: 'wasmer/memory-usage',
      value: 19060,
      unit: 'kB',
      range: undefined,
      extra: 'shim: 13440 kB\nzygote: 5620 kB'
    },
    {
      name: 'wasmtime/memory-usage',
      value: 22712,
      unit: 'kB',
      range: undefined,
      extra: 'shim: 16256 kB\nzygote: 6456 kB'
    }
  ]
}

@andreiltd
Copy link
Member

Nice! At what point in the process’s lifetime we take a snapshot of memory usage? IIUC, those values represent the peak usage during the process’s runtime, so ideally we should capture the snapshot just before terminating the process?

@jprendes
Copy link
Collaborator Author

Indeed, the snapshot is taken after shutting down the shim server, just before exiting the main function.

@jprendes
Copy link
Collaborator Author

And yes, they represent peak usage as reported by the kernel.

@Mossaka Mossaka merged commit 827fbd4 into containerd:main Jan 22, 2025
73 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Benchmark shim's memory usage
3 participants