Add iai_callgrind benchmark for measuring memory #64569

arlyon · 2024-04-16T16:23:24Z

No description provided.

arlyon · 2024-04-16T16:23:35Z

add CI config #64572
Add iai_callgrind benchmark for measuring memory #64569 👈
add next-build-test binary / library #64121 : 2 other dependent PRs (#64122 , #64717 )
canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @arlyon and the rest of your teammates on Graphite

ijjk · 2024-04-16T16:23:47Z

Allow CI Workflow Run

approve CI run for commit: ee00dad

Note: this should only be enabled once the PR is ready to go and can only be enabled by a maintainer

github-actions · 2024-04-16T17:07:10Z

All broken links are now fixed, thank you!

bgw · 2024-04-17T01:50:18Z

packages/next-swc/crates/next-build-test/benches/bench.rs

+
+fn get_container() -> (turbo_tasks::Vc<ProjectContainer>, tempdir::TempDir) {
+    let (options, dir) = init_bench(
+        "https://github.com/shadcn-ui/ui.git",


If we want a reproducible benchmark, this should have a commit hash associated with it. If you don't want to hardcode it, maybe take the commit hash in as an environment variable?

bgw · 2024-04-17T01:52:42Z

packages/next-swc/crates/next-build-test/benches/bench.rs

+        "https://github.com/shadcn-ui/ui.git",
+        "pnpm i",
+        Some("apps/www"),
+        "{\n  \"env\": {},\n  \"webpack\": {},\n  \"eslint\": {\n    \"ignoreDuringBuilds\": false\n  },\n  \"typescript\": {\n    \"ignoreBuildErrors\": false,\n    \"tsconfigPath\": \"tsconfig.json\"\n  },\n  \"distDir\": \".next\",\n  \"cleanDistDir\": true,\n  \"assetPrefix\": \"\",\n  \"cacheMaxMemorySize\": 52428800,\n  \"configOrigin\": \"next.config.mjs\",\n  \"useFileSystemPublicRoutes\": true,\n  \"generateBuildId\": null,\n  \"generateEtags\": true,\n  \"pageExtensions\": [\n    \"tsx\",\n    \"ts\",\n    \"jsx\",\n    \"js\"\n  ],\n  \"poweredByHeader\": true,\n  \"compress\": true,\n  \"analyticsId\": \"\",\n  \"images\": {\n    \"deviceSizes\": [\n      640,\n      750,\n      828,\n      1080,\n      1200,\n      1920,\n      2048,\n      3840\n    ],\n    \"imageSizes\": [\n      16,\n      32,\n      48,\n      64,\n      96,\n      128,\n      256,\n      384\n    ],\n    \"path\": \"/_next/image\",\n    \"loader\": \"default\",\n    \"loaderFile\": \"\",\n    \"domains\": [],\n    \"disableStaticImages\": false,\n    \"minimumCacheTTL\": 60,\n    \"formats\": [\n      \"image/webp\"\n    ],\n    \"dangerouslyAllowSVG\": false,\n    \"contentSecurityPolicy\": \"script-src 'none'; frame-src 'none'; sandbox;\",\n    \"contentDispositionType\": \"inline\",\n    \"remotePatterns\": [\n      {\n        \"protocol\": \"https\",\n        \"hostname\": \"avatars.githubusercontent.com\"\n      },\n      {\n        \"protocol\": \"https\",\n        \"hostname\": \"images.unsplash.com\"\n      }\n    ],\n    \"unoptimized\": false\n  },\n  \"devIndicators\": {\n    \"buildActivity\": true,\n    \"buildActivityPosition\": \"bottom-right\"\n  },\n  \"onDemandEntries\": {\n    \"maxInactiveAge\": 3600000,\n    \"pagesBufferLength\": 5\n  },\n  \"amp\": {\n    \"canonicalBase\": \"\"\n  },\n  \"basePath\": \"\",\n  \"sassOptions\": {},\n  \"trailingSlash\": false,\n  \"i18n\": null,\n  \"productionBrowserSourceMaps\": false,\n  \"optimizeFonts\": true,\n  \"excludeDefaultMomentLocales\": true,\n  \"serverRuntimeConfig\": {},\n  \"publicRuntimeConfig\": {},\n  \"reactProductionProfiling\": false,\n  \"reactStrictMode\": true,\n  \"httpAgentOptions\": {\n    \"keepAlive\": true\n  },\n  \"outputFileTracing\": true,\n  \"staticPageGenerationTimeout\": 60,\n  \"swcMinify\": true,\n  \"modularizeImports\": {\n    \"@mui/icons-material\": {\n      \"transform\": \"@mui/icons-material/{{member}}\"\n    },\n    \"lodash\": {\n      \"transform\": \"lodash/{{member}}\"\n    }\n  },\n  \"experimental\": {\n    \"prerenderEarlyExit\": false,\n    \"serverMinification\": true,\n    \"serverSourceMaps\": false,\n    \"linkNoTouchStart\": false,\n    \"caseSensitiveRoutes\": false,\n    \"clientRouterFilter\": true,\n    \"clientRouterFilterRedirects\": false,\n    \"fetchCacheKeyPrefix\": \"\",\n    \"middlewarePrefetch\": \"flexible\",\n    \"optimisticClientCache\": true,\n    \"manualClientBasePath\": false,\n    \"cpus\": 9,\n    \"memoryBasedWorkersCount\": false,\n    \"isrFlushToDisk\": true,\n    \"workerThreads\": false,\n    \"optimizeCss\": false,\n    \"nextScriptWorkers\": false,\n    \"scrollRestoration\": false,\n    \"externalDir\": false,\n    \"disableOptimizedLoading\": false,\n    \"gzipSize\": true,\n    \"craCompat\": false,\n    \"esmExternals\": true,\n    \"fullySpecified\": false,\n    \"outputFileTracingRoot\": \"/Users/arlyon/Programming/ui\",\n    \"swcTraceProfiling\": false,\n    \"forceSwcTransforms\": false,\n    \"largePageDataBytes\": 128000,\n    \"adjustFontFallbacks\": false,\n    \"adjustFontFallbacksWithSizeAdjust\": false,\n    \"typedRoutes\": false,\n    \"instrumentationHook\": false,\n    \"bundlePagesExternals\": false,\n    \"parallelServerCompiles\": false,\n    \"parallelServerBuildTraces\": false,\n    \"ppr\": false,\n    \"missingSuspenseWithCSRBailout\": true,\n    \"optimizeServerReact\": true,\n    \"useEarlyImport\": false,\n    \"staleTimes\": {\n      \"dynamic\": 30,\n      \"static\": 300\n    },\n    \"optimizePackageImports\": [\n      \"lucide-react\",\n      \"date-fns\",\n      \"lodash-es\",\n      \"ramda\",\n      \"antd\",\n      \"react-bootstrap\",\n      \"ahooks\",\n      \"@ant-design/icons\",\n      \"@headlessui/react\",\n      \"@headlessui-float/react\",\n      \"@heroicons/react/20/solid\",\n      \"@heroicons/react/24/solid\",\n      \"@heroicons/react/24/outline\",\n      \"@visx/visx\",\n      \"@tremor/react\",\n      \"rxjs\",\n      \"@mui/material\",\n      \"@mui/icons-material\",\n      \"recharts\",\n      \"react-use\",\n      \"@material-ui/core\",\n      \"@material-ui/icons\",\n      \"@tabler/icons-react\",\n      \"mui-core\",\n      \"react-icons/ai\",\n      \"react-icons/bi\",\n      \"react-icons/bs\",\n      \"react-icons/cg\",\n      \"react-icons/ci\",\n      \"react-icons/di\",\n      \"react-icons/fa\",\n      \"react-icons/fa6\",\n      \"react-icons/fc\",\n      \"react-icons/fi\",\n      \"react-icons/gi\",\n      \"react-icons/go\",\n      \"react-icons/gr\",\n      \"react-icons/hi\",\n      \"react-icons/hi2\",\n      \"react-icons/im\",\n      \"react-icons/io\",\n      \"react-icons/io5\",\n      \"react-icons/lia\",\n      \"react-icons/lib\",\n      \"react-icons/lu\",\n      \"react-icons/md\",\n      \"react-icons/pi\",\n      \"react-icons/ri\",\n      \"react-icons/rx\",\n      \"react-icons/si\",\n      \"react-icons/sl\",\n      \"react-icons/tb\",\n      \"react-icons/tfi\",\n      \"react-icons/ti\",\n      \"react-icons/vsc\",\n      \"react-icons/wi\"\n    ]\n  },\n  \"configFile\": \"/Users/arlyon/Programming/ui/apps/www/next.config.mjs\",\n  \"configFileName\": \"next.config.mjs\",\n  \"_originalRedirects\": [\n    {\n      \"source\": \"/components\",\n      \"destination\": \"/docs/components/accordion\",\n      \"permanent\": true\n    },\n    {\n      \"source\": \"/docs/components\",\n      \"destination\": \"/docs/components/accordion\",\n      \"permanent\": true\n    },\n    {\n      \"source\": \"/examples\",\n      \"destination\": \"/examples/mail\",\n      \"permanent\": false\n    },\n    {\n      \"source\": \"/docs/primitives/:path*\",\n      \"destination\": \"/docs/components/:path*\",\n      \"permanent\": true\n    },\n    {\n      \"source\": \"/figma\",\n      \"destination\": \"/docs/figma\",\n      \"permanent\": true\n    },\n    {\n      \"source\": \"/docs/forms\",\n      \"destination\": \"/docs/components/form\",\n      \"permanent\": false\n    },\n    {\n      \"source\": \"/docs/forms/react-hook-form\",\n      \"destination\": \"/docs/components/form\",\n      \"permanent\": false\n    }\n  ],\n  \"exportPathMap\": {}\n}"


This is illegible, and seems like it might be difficult to update. Can we move these long strings into separate json files and pull them in with include_str!()?

bgw · 2024-04-17T01:59:06Z

packages/next-swc/crates/next-build-test/benches/bench.rs

+}
+
+fn do_in_runtime() -> impl FnOnce(
+    Pin<


I think this type can be cleaned up a bit so that it doesn't get split into so many lines:

import Future at the top of the file?

Consider using the BoxFuture type from the futures_core or futures crates? https://docs.rs/futures-core/latest/futures_core/future/type.BoxFuture.html

Use anyhow::Result<()> instead of std::result::Result<(), anyhow::Error>

It looks like we're re-using this type later. Maybe it should have a type alias?

bgw · 2024-04-17T02:03:12Z

packages/next-swc/crates/next-build-test/benches/bench.rs

+
+    let rt = tokio::runtime::Builder::new_multi_thread()
+        .enable_all()
+        .on_thread_stop(|| {


Can't this just be

.on_thread_stop(TurboMalloc::thread_stop)

(and shouldn't clippy detect this? maybe I'm missing something?)

Yes. There is another place in the codebase where it is initialized without eta reduction so I just kept it consistent.

bgw · 2024-04-17T02:03:43Z

packages/next-swc/crates/next-build-test/benches/bench.rs

+        let entrypoints = container.entrypoints().await.unwrap();
+
+        let routes = entrypoints.routes.clone().into_iter().limit(pages);
+        // .filter(|route| SELECTED_PAGES.contains(&route.0.as_str()));


Leftover comment?

bgw · 2024-04-17T02:07:41Z

packages/next-swc/crates/next-build-test/benches/bench.rs

+        .unwrap();
+
+    let project_path = package_path
+        .map(|pp| tmp.path().join(pp))


map_or_else should combine this with the .unwrap_or_else below it.

https://doc.rust-lang.org/std/option/enum.Option.html#method.map_or_else

I thought there was a clippy lint for this?

I don' think clippy runs on benchmark code (or examples).

arlyon

Closing this as the overhead in recording is too much to be run in CI

ijjk added Turbopack Related to Turbopack with Next.js. type: next labels Apr 16, 2024

arlyon mentioned this pull request Apr 16, 2024

add next-build-test binary / library #64121

Merged

This was referenced Apr 16, 2024

add criterion benches #64122

Closed

add basic GB + secs measurement #64123

Closed

arlyon changed the title ~~add benches~~ Summary for Pull Request Apr 16, 2024

arlyon changed the title ~~Summary for Pull Request~~ Add iai_callgrind benchmark for measuring memory Apr 16, 2024

arlyon marked this pull request as ready for review April 16, 2024 16:25

arlyon requested review from timneutkens, ijjk, shuding, huozhi and a team as code owners April 16, 2024 16:25

arlyon force-pushed the arlyon/memory-benchmark branch from e413e17 to d40c089 Compare April 16, 2024 17:06

arlyon force-pushed the arlyon/memory-benchmark-iai-callgrind branch from 442b6bc to 778aee9 Compare April 16, 2024 17:06

arlyon mentioned this pull request Apr 16, 2024

add CI config #64572

Closed

arlyon force-pushed the arlyon/memory-benchmark branch from d40c089 to 9ac59e7 Compare April 16, 2024 17:19

arlyon force-pushed the arlyon/memory-benchmark-iai-callgrind branch from 778aee9 to 9d06c47 Compare April 16, 2024 17:19

bgw reviewed Apr 17, 2024

View reviewed changes

add benches

ee00dad

arlyon force-pushed the arlyon/memory-benchmark branch from 9ac59e7 to 9c35371 Compare April 17, 2024 12:56

arlyon force-pushed the arlyon/memory-benchmark-iai-callgrind branch from 9d06c47 to ee00dad Compare April 17, 2024 12:57

arlyon closed this Apr 17, 2024

arlyon commented Apr 17, 2024

View reviewed changes

This was referenced Apr 18, 2024

add bash script to run a benchmark #64717

Merged

add CI script for benchmarks #64718

Closed

github-actions bot added the locked label May 2, 2024

github-actions bot locked as resolved and limited conversation to collaborators May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add iai_callgrind benchmark for measuring memory #64569

Add iai_callgrind benchmark for measuring memory #64569

arlyon commented Apr 16, 2024 •

edited

Loading

arlyon commented Apr 16, 2024 •

edited

Loading

ijjk commented Apr 16, 2024 •

edited

Loading

github-actions bot commented Apr 16, 2024 •

edited

Loading

bgw Apr 17, 2024

bgw Apr 17, 2024

bgw Apr 17, 2024

bgw Apr 17, 2024

arlyon Apr 17, 2024

bgw Apr 17, 2024

bgw Apr 17, 2024

arlyon Apr 17, 2024

arlyon left a comment

Add iai_callgrind benchmark for measuring memory #64569

Add iai_callgrind benchmark for measuring memory #64569

Conversation

arlyon commented Apr 16, 2024 • edited Loading

arlyon commented Apr 16, 2024 • edited Loading

ijjk commented Apr 16, 2024 • edited Loading

github-actions bot commented Apr 16, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arlyon left a comment

Choose a reason for hiding this comment

arlyon commented Apr 16, 2024 •

edited

Loading

arlyon commented Apr 16, 2024 •

edited

Loading

ijjk commented Apr 16, 2024 •

edited

Loading

github-actions bot commented Apr 16, 2024 •

edited

Loading