Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(benchmarks/tailwind): Add a proper README and rename benchmarks to be clearer #1083

Merged
merged 7 commits into from
Dec 7, 2023
50 changes: 50 additions & 0 deletions benchmarks/tailwind-component/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Bencharmks for the Tailwind component

This is a collection of `tinybench` benchmarks that we've written with the purposes of scientifically
determining the performance hits that the Tailwind component causes to try improving it.

## Structure

```
├── package.json
├── src
| ├── emails
| ├── benchmark-0.0.12-vs-local-version.ts
| ├── benchmark-with-vs-without.ts
| └── tailwind-render.ts
├── tailwind.config.js
└── tsconfig.json
```

Each direct descendant of `./src` is a benchmark we have for a specific purpose.

The only exception for this is the `./src/tailwind-render.ts` as it is used for making a
flamegraph on the rendering process of the Tailwind component.

The `emails` folder contains examples to be used across different benchmarks.

## Running benchmarks

To avoid ESM problems, these benchmarks need to be compiled using `tsup`,
which can be done by running `pnpm compile`, and then using `node` directly.
Something like the following if you want to run the `with-vs-without` benchmark:

```sh
pnpm compile && node ./dist/benchmark-with-vs-without.js
```

They are each compiled into a different entry on the `./dist` folder with their respective names.

We have scripts for each benchmark on our `./package.json` that you can try running:

```json
"scripts": {
"with-vs-without": "pnpm compile && node ./dist/benchmark-with-vs-without.js",
"before-perf-vs-after-perf": "pnpm compile && node ./dist/benchmark-0.0.12-vs-local-version",

"flamegraph-render-tailwind": "pnpm compile && node --prof ./dist/tailwind-render && node --prof-process --preprocess -j isolate*.log | flamebearer",

"compile": "tsup src/*.ts",
"lint": "eslint ."
},
```
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[{"totalTime":12038.617523986846,"min":103.67929299920797,"max":159.12310099974275,"hz":8.306601634344709,"period":120.38617523986846,"samples":[103.67929299920797,105.21537199988961,105.42615700140595,106.07665099948645,106.37340800091624,106.42210000008345,106.94859899953008,107.04294800013304,107.76807000115514,107.96288700029254,108.07823400199413,108.35426300019026,108.59506599977612,109.02817900106311,109.05898800119758,109.12921999767423,109.15932599827647,109.55055199936032,109.60058600082994,109.77846100181341,109.85651699826121,110.3762039989233,111.15072999894619,111.17128900066018,111.18644699826837,111.29167499765754,112.28759799897671,112.5253240019083,112.5420360006392,113.1479649990797,113.18393300101161,113.26294099912047,113.3146480023861,113.57648099958897,113.6285380013287,113.77276999875903,113.8291859999299,113.87677500024438,113.91487699747086,114.08545700088143,114.15173299983144,114.5101259984076,114.66707099974155,115.09689899906516,115.28191700205207,115.33678100258112,115.51951399818063,115.75755199790001,115.8184460029006,115.9816830009222,116.02089600265026,116.29580299928784,117.04034299775958,117.16970700025558,117.42955499887466,117.81526900082827,117.9530789963901,118.26566699892282,118.28102499991655,118.81687299907207,119.09300199896097,119.1513409987092,120.37130599841475,120.67375399917364,120.86141699925065,121.37692600116134,121.50347400084138,122.36241899803281,122.70810800045729,122.89613199979067,123.11405200138688,123.61617599800229,124.1475649997592,124.34458500146866,126.06378800049424,126.18667799979448,126.64983899891376,126.96603399887681,127.51752000302076,127.85983400046825,128.67044800147414,132.14266699925065,134.0649299994111,135.84123999997973,137.67898400127888,137.87513300031424,138.75194199755788,138.7647969983518,139.1910180002451,139.48946899920702,140.04651600122452,140.52569700032473,141.95115599781275,145.6930919997394,145.84002799913287,148.3565689995885,148.63709700107574,149.61825200170279,156.45174799859524,159.12310099974275],"mean":120.38617523986846,"variance":157.94430766393114,"sd":12.567589572544575,"sem":1.2567589572544575,"df":99,"critical":1.9843,"moe":2.49378679888002,"rme":2.0714893499284037,"p75":126.06378800049424,"p99":156.45174799859524,"p995":159.12310099974275,"p999":159.12310099974275},{"totalTime":332638.3692750111,"min":3204.766656998545,"max":3745.5639889985323,"hz":0.30062677440955193,"period":3326.383692750111,"samples":[3204.766656998545,3214.9845269992948,3219.0279789976776,3220.6312830001116,3220.9293829984963,3222.649466998875,3223.5032120011747,3224.314356997609,3229.5999650023878,3232.916138999164,3234.773299999535,3235.755518000573,3236.6217459999025,3237.2919760011137,3237.8091380000114,3238.249605998397,3239.034701999277,3239.291525002569,3241.345287002623,3242.547295998782,3243.7745740003884,3244.297297000885,3244.8451160006225,3245.592660997063,3245.81346699968,3246.8237159997225,3247.1744250021875,3247.8810040019453,3249.4279730021954,3249.4329419992864,3249.9084559977055,3249.9610540010035,3250.1241609975696,3252.3451669998467,3253.2294390015304,3254.0084140002728,3256.0808099992573,3256.6045839972794,3257.4555049985647,3257.6531870029867,3258.5201559998095,3259.964090999216,3261.0969609990716,3261.9362290017307,3261.9993070028722,3262.276126999408,3264.3573299981654,3267.538520999253,3267.785675998777,3268.367760002613,3272.181471001357,3273.4954000003636,3276.806907001883,3276.8203620016575,3276.8539750017226,3277.237254999578,3279.130855999887,3280.7546079978347,3281.3124570026994,3281.7080099992454,3283.6706300005317,3286.4964330010116,3289.6075330004096,3290.887568999082,3294.2855079993606,3295.766261000186,3300.2926409989595,3302.5084969997406,3312.179950002581,3315.1587700024247,3331.7442319989204,3339.1331029981375,3354.7064530029893,3357.92853000015,3385.8360170014203,3398.4634180031717,3420.3473610021174,3426.1686350032687,3434.8194900006056,3438.820301000029,3441.3845020011067,3452.21779999882,3455.2742060013115,3456.0998480021954,3467.608545001596,3487.191211000085,3487.5488440021873,3493.309683997184,3495.012366000563,3522.858517996967,3530.621022000909,3534.4392809979618,3538.773800998926,3543.9955069981515,3599.68636899814,3600.567795999348,3617.627930998802,3619.995852999389,3681.150396000594,3745.5639889985323],"mean":3326.383692750111,"variance":14763.04411505073,"sd":121.50326791922402,"sem":12.150326791922401,"df":99,"critical":1.9843,"moe":24.10989345321162,"rme":0.7248079500196983,"p75":3385.8360170014203,"p99":3681.150396000594,"p995":3745.5639889985323,"p999":3745.5639889985323}]
4 changes: 2 additions & 2 deletions benchmarks/tailwind-component/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
"private": true,
"main": "dist/benchmark.js",
"scripts": {
"with-vs-without": "pnpm compile && node dist/benchmark-with-without",
"current-vs-latest": "pnpm compile && node dist/benchmark-current-vs-latest",
"with-vs-without": "pnpm compile && node ./dist/benchmark-with-vs-without.js",
"before-perf-vs-after-perf": "pnpm compile && node ./dist/benchmark-0.0.12-vs-local-version",
"flamegraph-render-tailwind": "pnpm compile && node --prof ./dist/tailwind-render && node --prof-process --preprocess -j isolate*.log | flamebearer",

"compile": "tsup src/*.ts",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,21 @@
import { render } from "@react-email/render";
import { writeFileSync } from "fs";
import { Bench } from "tinybench";
import EmailWithTailwind from "./emails/with-tailwind.js";
import { Tailwind as CurrentTailwind } from "../../../packages/tailwind/dist";
import { Tailwind as LatestTailwind } from "@react-email/tailwind";
import { Tailwind as VersionTwelveTailwind } from "@react-email/tailwind";

const main = async () => {
const bench = new Bench();
const bench = new Bench({
iterations: 100,
});

bench
.add("current", () => {
render(EmailWithTailwind({ Tailwind: CurrentTailwind }));
})
.add("latest", () => {
render(EmailWithTailwind({ Tailwind: LatestTailwind }));
render(EmailWithTailwind({ Tailwind: VersionTwelveTailwind }));
});

await bench.run();
Expand All @@ -22,6 +25,11 @@ const main = async () => {

main()
.then((bench) => {
writeFileSync(
"bench-results-100-iterations.json",
JSON.stringify(bench.results),
"utf-8",
);
console.table(bench.table());
})
.catch(console.error);
3 changes: 2 additions & 1 deletion benchmarks/tailwind-component/src/tailwind-render.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { render } from "@react-email/render";
import { Tailwind as CurrentTailwind } from "../../../packages/tailwind/dist";
import EmailWithTailwind from "./emails/with-tailwind.js";

render(EmailWithTailwind());
render(EmailWithTailwind({ Tailwind: CurrentTailwind }));