Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimize scan with a lookup table to reduce branching #59106

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Optimize scan with a lookup table to reduce branching #59106

wants to merge 13 commits into from
+226 −150

Conversation

DanielRosenwasser
Copy link
Member

@DanielRosenwasser DanielRosenwasser commented Jul 2, 2024 •
edited
Loading

This change introduces two lookup tables to the lexical scanner/tokenizer. One is a simple array-based lookup for entities in the ASCII range, and the other is a Map for wider codepoints.

These tables assign categories to codepoints to indicate how they should be handled. For example, all line breaks, single-line whitespace, and single-character tokens can be handled generically in their own respective fashion. Doing so allows us to perform fewer individual checks to jump into or skip over a given branch.

One additional optimization is the way in which simple single-character tokens are handled is that the token itself is encoded in the lower 8 bits the category value. This allows us to avoid any branching for such cases.

This all comes at the expense of some memory overhead, along with some overhead from fetching from the table. To avoid hitting the table too often, spaces, tabs, and carriage returns, and line feeds are currently handled separately.

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jul 2, 2024
@DanielRosenwasser
Copy link
Member Author

It's probably faster to actually check the common case of spaces, tabs, and newlines, but the current draft always does the lookup.

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,967k (± 1.03%) 192,827k (± 0.77%) ~ 192,123k 195,863k p=0.810 n=6
Parse Time 1.57s (± 1.04%) 1.54s (± 0.76%) -0.03s (- 2.12%) 1.53s 1.56s p=0.010 n=6
Bind Time 0.85s (± 0.48%) 0.85s (± 0.96%) ~ 0.83s 0.85s p=1.000 n=6
Check Time 11.16s (± 0.73%) 11.22s (± 0.58%) ~ 11.17s 11.35s p=0.261 n=6
Emit Time 3.25s (± 1.13%) 3.26s (± 0.54%) ~ 3.23s 3.28s p=0.802 n=6
Total Time 16.84s (± 0.48%) 16.86s (± 0.44%) ~ 16.80s 17.01s p=0.574 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,360k (± 0.00%) 1,218,350k (± 0.00%) ~ 1,218,273k 1,218,452k p=0.575 n=6
Parse Time 7.97s (± 0.63%) 7.92s (± 0.48%) ~ 7.86s 7.97s p=0.064 n=6
Bind Time 2.22s (± 0.18%) 2.23s (± 0.87%) ~ 2.20s 2.25s p=0.473 n=6
Check Time 35.84s (± 0.39%) 35.73s (± 0.59%) ~ 35.51s 36.09s p=0.173 n=6
Emit Time 16.22s (± 0.31%) 16.14s (± 0.59%) ~ 16.01s 16.28s p=0.148 n=6
Total Time 62.25s (± 0.24%) 62.03s (± 0.40%) ~ 61.76s 62.39s p=0.148 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,767k (± 0.00%) 2,117,708k (± 0.01%) ~ 2,117,559k 2,117,943k p=0.298 n=6
Parse Time 9.71s (± 0.16%) 9.63s (± 0.63%) ~ 9.55s 9.73s p=0.063 n=6
Bind Time 3.39s (± 0.58%) 3.40s (± 1.27%) ~ 3.34s 3.45s p=0.571 n=6
Check Time 102.35s (± 1.25%) 102.33s (± 0.99%) ~ 100.38s 103.20s p=0.936 n=6
Emit Time 0.19s (± 4.29%) 0.19s (± 2.81%) ~ 0.19s 0.20s p=0.855 n=6
Total Time 115.64s (± 1.09%) 115.55s (± 0.87%) ~ 113.64s 116.47s p=0.423 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,226,737 +1,456 (+ 0.12%) ~ ~ p=0.001 n=6
Types 261,342 261,620 +278 (+ 0.11%) ~ ~ p=0.001 n=6
Memory used 2,341,362k (± 0.05%) 2,341,478k (± 0.06%) ~ 2,339,978k 2,343,685k p=1.000 n=6
Parse Time 7.52s (± 0.57%) 7.47s (± 1.18%) ~ 7.38s 7.59s p=0.378 n=6
Bind Time 2.78s (± 0.87%) 2.80s (± 0.69%) ~ 2.78s 2.83s p=0.191 n=6
Check Time 49.65s (± 0.65%) 49.63s (± 0.54%) ~ 49.39s 50.16s p=1.000 n=6
Emit Time 3.90s (± 5.28%) 4.07s (± 6.58%) ~ 3.75s 4.38s p=0.298 n=6
Total Time 63.88s (± 0.45%) 63.98s (± 0.59%) ~ 63.47s 64.49s p=0.689 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,226,737 +1,456 (+ 0.12%) ~ ~ p=0.001 n=6
Types 261,342 261,620 +278 (+ 0.11%) ~ ~ p=0.001 n=6
Memory used 2,424,311k (± 1.02%) 2,415,497k (± 0.05%) ~ 2,414,486k 2,417,424k p=0.471 n=6
Parse Time 6.28s (± 0.93%) 6.25s (± 0.84%) ~ 6.20s 6.35s p=0.298 n=6
Bind Time 2.02s (± 0.74%) 2.02s (± 0.60%) ~ 2.00s 2.03s p=0.517 n=6
Check Time 40.72s (± 0.37%) 40.95s (± 0.63%) ~ 40.66s 41.23s p=0.230 n=6
Emit Time 3.20s (± 6.13%) 3.22s (± 5.34%) ~ 3.04s 3.53s p=0.688 n=6
Total Time 52.23s (± 0.46%) 52.44s (± 0.54%) ~ 52.07s 52.79s p=0.173 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 259,650 +1,456 (+ 0.56%) ~ ~ p=0.001 n=6
Types 104,737 105,015 +278 (+ 0.27%) ~ ~ p=0.001 n=6
Memory used 427,489k (± 0.01%) 428,192k (± 0.01%) +704k (+ 0.16%) 428,150k 428,249k p=0.005 n=6
Parse Time 3.29s (± 0.57%) 3.27s (± 0.32%) ~ 3.26s 3.29s p=0.370 n=6
Bind Time 1.33s (± 0.31%) 1.32s (± 0.96%) ~ 1.30s 1.33s p=0.214 n=6
Check Time 17.86s (± 0.42%) 17.82s (± 0.31%) ~ 17.77s 17.92s p=0.335 n=6
Emit Time 1.25s (± 1.40%) 1.26s (± 0.96%) ~ 1.25s 1.28s p=0.317 n=6
Total Time 23.72s (± 0.41%) 23.68s (± 0.20%) ~ 23.62s 23.75s p=0.195 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,514k (± 0.04%) 369,435k (± 0.02%) ~ 369,334k 369,557k p=0.298 n=6
Parse Time 2.76s (± 0.67%) 2.76s (± 1.11%) ~ 2.72s 2.79s p=0.747 n=6
Bind Time 1.58s (± 0.52%) 1.59s (± 1.17%) ~ 1.57s 1.61s p=0.445 n=6
Check Time 15.44s (± 0.24%) 15.41s (± 0.44%) ~ 15.31s 15.49s p=0.573 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.77s (± 0.27%) 19.75s (± 0.44%) ~ 19.65s 19.88s p=0.936 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,975 2,879,975 ~ ~ ~ p=1.000 n=6
Types 975,327 975,327 ~ ~ ~ p=1.000 n=6
Memory used 3,042,839k (± 0.00%) 3,042,871k (± 0.00%) ~ 3,042,771k 3,042,936k p=0.298 n=6
Parse Time 13.73s (± 0.35%) 13.73s (± 0.40%) ~ 13.64s 13.78s p=1.000 n=6
Bind Time 4.24s (± 1.91%) 4.26s (± 2.49%) ~ 4.18s 4.40s p=0.935 n=6
Check Time 74.09s (± 2.34%) 73.73s (± 0.43%) ~ 73.21s 74.19s p=0.471 n=6
Emit Time 23.67s (± 6.50%) 24.12s (± 1.92%) ~ 23.22s 24.50s p=0.936 n=6
Total Time 115.72s (± 0.26%) 115.84s (± 0.36%) ~ 115.22s 116.41s p=0.810 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,511k (± 0.01%) 411,543k (± 0.01%) ~ 411,475k 411,623k p=0.378 n=6
Parse Time 3.18s (± 0.59%) 3.15s (± 0.53%) ~ 3.13s 3.17s p=0.062 n=6
Bind Time 1.41s (± 1.22%) 1.41s (± 0.59%) ~ 1.41s 1.43s p=0.934 n=6
Check Time 14.16s (± 0.24%) 14.19s (± 0.33%) ~ 14.14s 14.25s p=0.688 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.75s (± 0.23%) 18.75s (± 0.25%) ~ 18.69s 18.82s p=0.936 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,921k (± 0.07%) 462,714k (± 0.08%) ~ 462,378k 463,125k p=0.471 n=6
Parse Time 3.18s (± 0.37%) 3.08s (± 0.67%) -0.09s (- 2.94%) 3.06s 3.12s p=0.005 n=6
Bind Time 1.17s 1.17s (± 0.76%) ~ 1.16s 1.18s p=1.000 n=6
Check Time 17.89s (± 0.36%) 17.95s (± 0.71%) ~ 17.82s 18.18s p=0.470 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.24s (± 0.29%) 22.20s (± 0.64%) ~ 22.04s 22.46s p=0.378 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,294ms (± 0.67%) 2,287ms (± 0.64%) ~ 2,265ms 2,310ms p=0.298 n=6
Req 2 - geterr 5,059ms (± 0.89%) 5,059ms (± 0.55%) ~ 5,008ms 5,087ms p=1.000 n=6
Req 3 - references 264ms (± 0.24%) 264ms (± 0.21%) ~ 263ms 264ms p=0.201 n=6
Req 4 - navto 228ms (± 0.18%) 227ms (± 0.90%) ~ 223ms 228ms p=0.218 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 89ms (± 0.94%) 90ms (± 3.58%) ~ 88ms 96ms p=1.000 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,424ms (± 0.56%) 2,397ms (± 0.69%) -27ms (- 1.11%) 2,382ms 2,418ms p=0.030 n=6
Req 2 - geterr 3,791ms (± 0.52%) 3,794ms (± 0.28%) ~ 3,778ms 3,810ms p=1.000 n=6
Req 3 - references 278ms (± 0.20%) 278ms (± 0.63%) ~ 275ms 279ms p=0.508 n=6
Req 4 - navto 226ms (± 0.28%) 230ms (± 2.60%) +4ms (+ 1.70%) 227ms 242ms p=0.008 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 71ms (± 3.83%) 83ms (± 5.31%) 🔻+12ms (+16.12%) 74ms 85ms p=0.007 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,184ms (± 0.87%) 5,140ms (± 0.30%) -44ms (- 0.85%) 5,120ms 5,156ms p=0.030 n=6
Req 2 - geterr 1,133ms (± 1.61%) 1,125ms (± 1.33%) ~ 1,111ms 1,153ms p=0.423 n=6
Req 3 - references 78ms (± 4.99%) 77ms (± 4.67%) ~ 75ms 82ms p=0.344 n=6
Req 4 - navto 457ms (± 0.23%) 458ms (± 0.93%) ~ 456ms 467ms p=0.605 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 832ms (± 2.15%) 842ms (± 1.25%) ~ 830ms 857ms p=0.297 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 156.97ms (± 0.16%) 156.76ms (± 0.21%) -0.20ms (- 0.13%) 155.79ms 162.86ms p=0.000 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.76ms (± 0.13%) 230.73ms (± 0.13%) ~ 229.50ms 233.81ms p=0.615 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 225.94ms (± 0.14%) 226.15ms (± 0.13%) +0.21ms (+ 0.09%) 224.77ms 229.96ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.80ms (± 0.16%) 225.89ms (± 0.13%) +0.08ms (+ 0.04%) 224.41ms 227.84ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,968k (± 1.03%) 193,382k (± 0.97%) ~ 192,131k 195,906k p=0.936 n=6
Parse Time 1.58s (± 0.40%) 1.55s (± 1.35%) -0.03s (- 1.90%) 1.51s 1.57s p=0.005 n=6
Bind Time 0.85s (± 0.60%) 0.86s (± 1.47%) ~ 0.85s 0.88s p=0.418 n=6
Check Time 11.17s (± 0.39%) 11.17s (± 0.45%) ~ 11.10s 11.25s p=1.000 n=6
Emit Time 3.29s (± 0.17%) 3.27s (± 0.59%) ~ 3.25s 3.30s p=0.100 n=6
Total Time 16.89s (± 0.31%) 16.84s (± 0.28%) ~ 16.80s 16.92s p=0.170 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,354k (± 0.00%) 1,218,334k (± 0.00%) ~ 1,218,307k 1,218,364k p=0.471 n=6
Parse Time 7.96s (± 1.13%) 7.86s (± 0.61%) -0.09s (- 1.19%) 7.78s 7.92s p=0.043 n=6
Bind Time 2.23s (± 0.52%) 2.22s (± 0.68%) ~ 2.21s 2.25s p=0.401 n=6
Check Time 35.74s (± 0.30%) 35.75s (± 0.36%) ~ 35.59s 35.97s p=0.936 n=6
Emit Time 16.16s (± 0.44%) 16.12s (± 0.29%) ~ 16.05s 16.18s p=0.377 n=6
Total Time 62.08s (± 0.21%) 61.95s (± 0.17%) ~ 61.81s 62.14s p=0.065 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,728k (± 0.01%) 2,117,653k (± 0.01%) ~ 2,117,562k 2,117,860k p=0.378 n=6
Parse Time 7.89s (± 0.19%) 7.77s (± 0.29%) -0.12s (- 1.52%) 7.75s 7.81s p=0.005 n=6
Bind Time 2.76s (± 0.37%) 2.76s (± 0.70%) ~ 2.73s 2.79s p=1.000 n=6
Check Time 83.71s (± 0.25%) 83.61s (± 0.39%) ~ 83.29s 84.12s p=0.470 n=6
Emit Time 0.16s (± 4.65%) 0.16s (± 3.95%) ~ 0.15s 0.17s p=0.718 n=6
Total Time 94.52s (± 0.22%) 94.31s (± 0.34%) ~ 93.98s 94.78s p=0.336 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,410 +129 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,443 +101 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,350,713k (± 1.06%) 2,341,270k (± 0.04%) ~ 2,339,826k 2,342,288k p=0.378 n=6
Parse Time 7.52s (± 0.49%) 7.42s (± 0.98%) -0.09s (- 1.26%) 7.34s 7.55s p=0.045 n=6
Bind Time 2.78s (± 0.49%) 2.79s (± 0.77%) ~ 2.75s 2.81s p=0.569 n=6
Check Time 49.78s (± 0.71%) 49.77s (± 0.38%) ~ 49.51s 50.03s p=1.000 n=6
Emit Time 3.94s (± 5.23%) 3.85s (± 3.21%) ~ 3.78s 4.10s p=0.688 n=6
Total Time 64.05s (± 0.74%) 63.84s (± 0.55%) ~ 63.48s 64.46s p=0.689 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,410 +129 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,443 +101 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,415,021k (± 0.04%) 2,416,113k (± 0.05%) ~ 2,415,052k 2,417,912k p=0.128 n=6
Parse Time 7.80s (± 0.99%) 7.66s (± 1.02%) -0.13s (- 1.69%) 7.52s 7.73s p=0.020 n=6
Bind Time 2.51s (± 0.39%) 2.52s (± 1.05%) ~ 2.48s 2.55s p=0.870 n=6
Check Time 50.40s (± 0.64%) 50.43s (± 0.51%) ~ 50.20s 50.86s p=1.000 n=6
Emit Time 3.99s (± 4.97%) 3.85s (± 2.53%) ~ 3.73s 3.99s p=0.336 n=6
Total Time 64.71s (± 0.41%) 64.47s (± 0.37%) ~ 64.26s 64.89s p=0.173 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,323 +129 (+ 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,838 +101 (+ 0.10%) ~ ~ p=0.001 n=6
Memory used 427,490k (± 0.01%) 427,609k (± 0.01%) +119k (+ 0.03%) 427,547k 427,695k p=0.005 n=6
Parse Time 3.30s (± 0.70%) 3.23s (± 0.54%) -0.07s (- 1.97%) 3.21s 3.26s p=0.005 n=6
Bind Time 1.32s (± 0.91%) 1.31s (± 1.42%) ~ 1.28s 1.33s p=0.096 n=6
Check Time 17.86s (± 0.30%) 17.84s (± 0.29%) ~ 17.78s 17.91s p=0.471 n=6
Emit Time 1.26s (± 0.83%) 1.26s (± 0.82%) ~ 1.24s 1.27s p=0.801 n=6
Total Time 23.74s (± 0.27%) 23.63s (± 0.22%) -0.11s (- 0.46%) 23.58s 23.69s p=0.037 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,602k (± 0.05%) 369,472k (± 0.01%) ~ 369,409k 369,567k p=0.298 n=6
Parse Time 2.78s (± 1.50%) 2.72s (± 0.72%) -0.05s (- 1.92%) 2.70s 2.75s p=0.036 n=6
Bind Time 1.57s (± 0.40%) 1.59s (± 1.11%) ~ 1.56s 1.61s p=0.081 n=6
Check Time 15.44s (± 0.14%) 15.45s (± 0.19%) ~ 15.42s 15.49s p=0.625 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.79s (± 0.22%) 19.76s (± 0.18%) ~ 19.71s 19.80s p=0.147 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,975 2,879,975 ~ ~ ~ p=1.000 n=6
Types 975,327 975,327 ~ ~ ~ p=1.000 n=6
Memory used 3,042,832k (± 0.00%) 3,042,846k (± 0.00%) ~ 3,042,776k 3,042,921k p=0.872 n=6
Parse Time 13.70s (± 0.25%) 13.63s (± 0.29%) -0.07s (- 0.51%) 13.57s 13.68s p=0.020 n=6
Bind Time 4.24s (± 2.07%) 4.20s (± 0.18%) ~ 4.19s 4.21s p=0.210 n=6
Check Time 73.44s (± 0.29%) 74.72s (± 2.03%) +1.28s (+ 1.75%) 73.36s 77.27s p=0.045 n=6
Emit Time 24.17s (± 1.22%) 23.06s (± 6.83%) ~ 20.45s 24.52s p=0.335 n=6
Total Time 115.55s (± 0.40%) 115.61s (± 0.22%) ~ 115.30s 116.02s p=1.000 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,574k (± 0.02%) 411,568k (± 0.01%) ~ 411,519k 411,630k p=1.000 n=6
Parse Time 3.81s (± 0.59%) 3.75s (± 0.31%) -0.07s (- 1.75%) 3.73s 3.76s p=0.005 n=6
Bind Time 1.70s (± 0.62%) 1.69s (± 0.58%) ~ 1.68s 1.71s p=0.547 n=6
Check Time 16.74s (± 0.28%) 16.76s (± 0.25%) ~ 16.70s 16.83s p=0.421 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.25s (± 0.22%) 22.19s (± 0.25%) ~ 22.11s 22.28s p=0.106 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,752k (± 0.09%) 462,922k (± 0.07%) ~ 462,313k 463,152k p=0.936 n=6
Parse Time 2.65s (± 0.82%) 2.56s (± 0.40%) 🟩-0.08s (- 3.15%) 2.55s 2.57s p=0.004 n=6
Bind Time 0.98s (± 0.42%) 0.97s (± 0.42%) -0.01s (- 0.68%) 0.97s 0.98s p=0.034 n=6
Check Time 15.18s (± 0.33%) 15.19s (± 0.33%) ~ 15.10s 15.24s p=0.748 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.81s (± 0.20%) 18.72s (± 0.30%) -0.08s (- 0.43%) 18.62s 18.78s p=0.013 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,299ms (± 0.49%) 2,280ms (± 0.70%) -19ms (- 0.84%) 2,253ms 2,295ms p=0.045 n=6
Req 2 - geterr 5,060ms (± 0.47%) 5,056ms (± 0.55%) ~ 5,012ms 5,085ms p=0.936 n=6
Req 3 - references 264ms (± 0.28%) 264ms (± 0.62%) ~ 261ms 265ms p=0.800 n=6
Req 4 - navto 227ms (± 1.31%) 229ms (± 0.33%) ~ 228ms 230ms p=0.081 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 89ms (± 0.84%) 86ms (± 7.46%) ~ 78ms 93ms p=0.564 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,409ms (± 0.50%) 2,375ms (± 0.37%) -34ms (- 1.40%) 2,366ms 2,391ms p=0.005 n=6
Req 2 - geterr 3,796ms (± 0.43%) 3,801ms (± 0.20%) ~ 3,794ms 3,814ms p=0.173 n=6
Req 3 - references 276ms (± 0.42%) 276ms (± 0.54%) ~ 275ms 279ms p=0.804 n=6
Req 4 - navto 227ms (± 0.18%) 227ms (± 0.18%) ~ 226ms 227ms p=1.000 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 73ms (± 7.73%) 76ms (± 8.76%) ~ 69ms 85ms p=0.372 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,204ms (± 0.84%) 5,137ms (± 0.78%) -68ms (- 1.30%) 5,086ms 5,200ms p=0.031 n=6
Req 2 - geterr 1,128ms (± 1.34%) 1,130ms (± 0.68%) ~ 1,123ms 1,143ms p=0.936 n=6
Req 3 - references 78ms (± 4.93%) 77ms (± 5.21%) ~ 74ms 83ms p=0.681 n=6
Req 4 - navto 459ms (± 0.79%) 457ms (± 0.29%) ~ 455ms 459ms p=0.246 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 841ms (± 2.00%) 835ms (± 1.56%) ~ 823ms 851ms p=0.689 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 156.99ms (± 0.17%) 156.89ms (± 0.17%) -0.10ms (- 0.07%) 155.87ms 159.53ms p=0.001 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.67ms (± 0.13%) 230.82ms (± 0.14%) +0.16ms (+ 0.07%) 229.42ms 235.56ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.45ms (± 0.13%) 226.53ms (± 0.13%) +0.08ms (+ 0.04%) 225.23ms 229.04ms p=0.009 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.73ms (± 0.13%) 225.97ms (± 0.13%) +0.25ms (+ 0.11%) 224.25ms 228.07ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,962k (± 1.02%) 193,453k (± 0.92%) ~ 192,161k 195,765k p=0.936 n=6
Parse Time 1.57s (± 0.74%) 1.52s (± 0.77%) -0.05s (- 2.98%) 1.50s 1.53s p=0.005 n=6
Bind Time 0.85s (± 0.89%) 0.85s (± 0.60%) ~ 0.85s 0.86s p=0.241 n=6
Check Time 11.23s (± 0.43%) 11.21s (± 0.34%) ~ 11.17s 11.27s p=0.519 n=6
Emit Time 3.27s (± 1.09%) 3.28s (± 0.89%) ~ 3.24s 3.33s p=0.519 n=6
Total Time 16.92s (± 0.43%) 16.86s (± 0.21%) ~ 16.83s 16.93s p=0.171 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,327k (± 0.00%) 1,218,367k (± 0.00%) ~ 1,218,289k 1,218,410k p=0.066 n=6
Parse Time 7.96s (± 0.47%) 7.88s (± 0.33%) -0.08s (- 0.96%) 7.85s 7.91s p=0.008 n=6
Bind Time 2.23s (± 0.54%) 2.23s (± 0.54%) ~ 2.22s 2.25s p=1.000 n=6
Check Time 35.79s (± 0.58%) 35.84s (± 0.21%) ~ 35.75s 35.92s p=0.198 n=6
Emit Time 16.18s (± 0.29%) 16.23s (± 0.73%) ~ 16.11s 16.43s p=0.520 n=6
Total Time 62.16s (± 0.35%) 62.18s (± 0.20%) ~ 62.04s 62.35s p=0.574 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,610k (± 0.00%) 2,117,706k (± 0.01%) ~ 2,117,502k 2,117,823k p=0.093 n=6
Parse Time 9.71s (± 0.57%) 9.54s (± 0.35%) -0.17s (- 1.73%) 9.51s 9.59s p=0.005 n=6
Bind Time 3.38s (± 0.76%) 3.37s (± 0.50%) ~ 3.35s 3.39s p=0.568 n=6
Check Time 102.45s (± 0.35%) 101.87s (± 0.99%) ~ 99.94s 102.67s p=0.261 n=6
Emit Time 0.19s (± 4.22%) 0.20s (± 4.15%) ~ 0.19s 0.21s p=0.389 n=6
Total Time 115.73s (± 0.29%) 114.98s (± 0.85%) ~ 113.12s 115.78s p=0.078 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,412 +131 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,447 +105 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,341,897k (± 0.06%) 2,342,141k (± 0.03%) ~ 2,341,329k 2,343,133k p=0.936 n=6
Parse Time 6.05s (± 0.90%) 5.93s (± 0.86%) -0.12s (- 1.96%) 5.89s 6.02s p=0.013 n=6
Bind Time 2.27s (± 0.66%) 2.25s (± 0.61%) ~ 2.24s 2.27s p=0.197 n=6
Check Time 40.19s (± 0.46%) 40.33s (± 0.44%) ~ 40.18s 40.56s p=0.128 n=6
Emit Time 3.04s (± 5.65%) 3.03s (± 2.33%) ~ 2.95s 3.15s p=0.471 n=6
Total Time 51.57s (± 0.61%) 51.55s (± 0.50%) ~ 51.33s 51.89s p=0.810 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,412 +131 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,447 +105 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,415,583k (± 0.03%) 2,415,755k (± 0.01%) ~ 2,415,449k 2,416,417k p=0.575 n=6
Parse Time 7.76s (± 0.37%) 7.59s (± 1.04%) -0.17s (- 2.19%) 7.46s 7.68s p=0.005 n=6
Bind Time 2.52s (± 0.89%) 2.52s (± 1.54%) ~ 2.47s 2.57s p=0.809 n=6
Check Time 50.21s (± 0.47%) 50.38s (± 0.29%) ~ 50.19s 50.58s p=0.173 n=6
Emit Time 3.93s (± 4.63%) 3.92s (± 1.48%) ~ 3.82s 4.00s p=0.470 n=6
Total Time 64.41s (± 0.49%) 64.42s (± 0.25%) ~ 64.15s 64.60s p=0.810 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,325 +131 (+ 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,842 +105 (+ 0.10%) ~ ~ p=0.001 n=6
Memory used 427,480k (± 0.00%) 427,683k (± 0.01%) +203k (+ 0.05%) 427,630k 427,729k p=0.005 n=6
Parse Time 3.28s (± 0.66%) 3.23s (± 0.73%) -0.05s (- 1.42%) 3.20s 3.26s p=0.013 n=6
Bind Time 1.31s (± 0.96%) 1.30s (± 1.41%) ~ 1.29s 1.33s p=0.397 n=6
Check Time 17.79s (± 0.29%) 17.82s (± 0.38%) ~ 17.76s 17.94s p=0.870 n=6
Emit Time 1.25s (± 0.71%) 1.25s (± 1.01%) ~ 1.23s 1.27s p=1.000 n=6
Total Time 23.63s (± 0.18%) 23.60s (± 0.21%) ~ 23.54s 23.68s p=0.377 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,562k (± 0.03%) 369,530k (± 0.04%) ~ 369,434k 369,814k p=0.575 n=6
Parse Time 2.78s (± 0.71%) 2.73s (± 0.46%) -0.05s (- 1.68%) 2.71s 2.74s p=0.005 n=6
Bind Time 1.58s (± 0.98%) 1.59s (± 0.94%) ~ 1.58s 1.62s p=0.084 n=6
Check Time 15.48s (± 0.34%) 15.45s (± 0.42%) ~ 15.34s 15.53s p=0.422 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.84s (± 0.39%) 19.78s (± 0.45%) ~ 19.64s 19.89s p=0.336 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,975 2,879,975 ~ ~ ~ p=1.000 n=6
Types 975,327 975,327 ~ ~ ~ p=1.000 n=6
Memory used 3,042,831k (± 0.00%) 3,042,773k (± 0.00%) ~ 3,042,693k 3,042,884k p=0.093 n=6
Parse Time 13.74s (± 0.52%) 13.54s (± 0.26%) -0.20s (- 1.44%) 13.50s 13.60s p=0.005 n=6
Bind Time 4.24s (± 1.92%) 4.22s (± 1.96%) ~ 4.17s 4.39s p=0.373 n=6
Check Time 73.51s (± 0.35%) 74.09s (± 1.81%) ~ 73.20s 76.16s p=0.810 n=6
Emit Time 24.23s (± 0.42%) 23.60s (± 4.01%) ~ 22.20s 24.27s p=0.128 n=6
Total Time 115.72s (± 0.20%) 115.46s (± 0.33%) ~ 115.07s 116.08s p=0.228 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,528k (± 0.01%) 411,534k (± 0.02%) ~ 411,456k 411,618k p=0.936 n=6
Parse Time 3.18s (± 0.54%) 3.13s (± 0.90%) -0.05s (- 1.63%) 3.10s 3.16s p=0.013 n=6
Bind Time 1.41s (± 0.53%) 1.41s (± 0.94%) ~ 1.39s 1.43s p=0.548 n=6
Check Time 14.20s (± 0.34%) 14.20s (± 0.24%) ~ 14.15s 14.25s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.79s (± 0.28%) 18.74s (± 0.25%) ~ 18.66s 18.80s p=0.172 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,528k (± 0.06%) 462,641k (± 0.08%) ~ 462,297k 463,096k p=0.575 n=6
Parse Time 3.18s (± 0.32%) 3.08s (± 0.45%) 🟩-0.10s (- 3.20%) 3.06s 3.10s p=0.004 n=6
Bind Time 1.17s (± 0.47%) 1.17s (± 0.35%) ~ 1.16s 1.17s p=0.282 n=6
Check Time 17.88s (± 0.45%) 17.88s (± 0.18%) ~ 17.84s 17.91s p=0.468 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.22s (± 0.39%) 22.12s (± 0.17%) ~ 22.07s 22.17s p=0.066 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,291ms (± 0.18%) 2,242ms (± 0.62%) -49ms (- 2.14%) 2,227ms 2,262ms p=0.005 n=6
Req 2 - geterr 5,057ms (± 0.58%) 5,071ms (± 0.31%) ~ 5,045ms 5,087ms p=0.471 n=6
Req 3 - references 265ms (± 0.15%) 264ms (± 0.20%) -1ms (- 0.44%) 263ms 264ms p=0.007 n=6
Req 4 - navto 228ms 228ms (± 0.37%) ~ 226ms 228ms p=0.176 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 88ms (± 0.46%) 88ms (± 0.46%) ~ 88ms 89ms p=1.000 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,419ms (± 0.30%) 2,386ms (± 1.10%) ~ 2,366ms 2,425ms p=0.065 n=6
Req 2 - geterr 3,793ms (± 0.38%) 3,792ms (± 0.34%) ~ 3,777ms 3,812ms p=0.748 n=6
Req 3 - references 276ms (± 0.27%) 277ms (± 0.48%) ~ 275ms 279ms p=0.131 n=6
Req 4 - navto 226ms (± 0.28%) 226ms (± 0.23%) ~ 226ms 227ms p=0.386 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 72ms (± 3.83%) 72ms (± 8.71%) ~ 67ms 84ms p=0.412 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,182ms (± 0.45%) 5,126ms (± 0.19%) -56ms (- 1.08%) 5,112ms 5,137ms p=0.005 n=6
Req 2 - geterr 1,120ms (± 1.44%) 1,137ms (± 1.43%) ~ 1,118ms 1,162ms p=0.148 n=6
Req 3 - references 77ms (± 2.95%) 79ms (± 4.88%) ~ 75ms 82ms p=0.673 n=6
Req 4 - navto 459ms (± 0.70%) 459ms (± 1.26%) ~ 456ms 471ms p=0.505 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 833ms (± 1.53%) 841ms (± 1.18%) ~ 830ms 851ms p=0.469 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.08ms (± 0.17%) 157.15ms (± 0.17%) +0.08ms (+ 0.05%) 156.02ms 160.72ms p=0.004 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.51ms (± 0.14%) 230.61ms (± 0.19%) +0.10ms (+ 0.04%) 229.34ms 246.30ms p=0.008 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.09ms (± 0.13%) 226.31ms (± 0.17%) +0.22ms (+ 0.10%) 225.05ms 236.63ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.82ms (± 0.16%) 225.89ms (± 0.17%) +0.07ms (+ 0.03%) 224.42ms 235.68ms p=0.034 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,433k (± 0.93%) 194,534k (± 0.94%) ~ 192,152k 195,805k p=0.810 n=6
Parse Time 1.58s (± 1.09%) 1.50s (± 0.73%) 🟩-0.08s (- 4.96%) 1.49s 1.52s p=0.005 n=6
Bind Time 0.85s (± 1.21%) 0.86s (± 1.36%) ~ 0.84s 0.87s p=0.456 n=6
Check Time 11.17s (± 0.37%) 11.22s (± 0.45%) ~ 11.16s 11.28s p=0.196 n=6
Emit Time 3.29s (± 0.85%) 3.28s (± 0.45%) ~ 3.26s 3.30s p=0.935 n=6
Total Time 16.89s (± 0.17%) 16.86s (± 0.26%) ~ 16.80s 16.91s p=0.295 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,400k (± 0.00%) 1,218,372k (± 0.00%) ~ 1,218,309k 1,218,440k p=0.230 n=6
Parse Time 6.67s (± 0.92%) 6.57s (± 0.54%) -0.10s (- 1.47%) 6.50s 6.60s p=0.016 n=6
Bind Time 1.87s (± 0.44%) 1.87s (± 0.22%) ~ 1.87s 1.88s p=0.206 n=6
Check Time 30.60s (± 0.36%) 30.62s (± 0.67%) ~ 30.43s 31.02s p=1.000 n=6
Emit Time 13.58s (± 0.40%) 13.58s (± 0.50%) ~ 13.51s 13.69s p=1.000 n=6
Total Time 52.72s (± 0.38%) 52.65s (± 0.51%) ~ 52.42s 53.15s p=0.470 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,980 2,136,980 ~ ~ ~ p=1.000 n=6
Types 929,117 929,117 ~ ~ ~ p=1.000 n=6
Memory used 2,117,784k (± 0.01%) 2,117,749k (± 0.00%) ~ 2,117,569k 2,117,854k p=0.810 n=6
Parse Time 7.88s (± 0.25%) 7.74s (± 0.28%) -0.14s (- 1.76%) 7.72s 7.78s p=0.005 n=6
Bind Time 2.73s (± 0.43%) 2.74s (± 0.68%) ~ 2.71s 2.76s p=0.101 n=6
Check Time 83.32s (± 0.38%) 83.25s (± 0.24%) ~ 82.99s 83.49s p=0.471 n=6
Emit Time 0.15s (± 3.53%) 0.16s (± 3.29%) ~ 0.15s 0.16s p=0.640 n=6
Total Time 94.08s (± 0.35%) 93.89s (± 0.20%) ~ 93.66s 94.12s p=0.336 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,412 +131 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,448 +106 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,342,656k (± 0.02%) 2,342,024k (± 0.04%) ~ 2,340,664k 2,343,136k p=0.378 n=6
Parse Time 6.03s (± 0.77%) 5.98s (± 0.93%) ~ 5.90s 6.05s p=0.297 n=6
Bind Time 2.26s (± 0.91%) 2.26s (± 0.54%) ~ 2.24s 2.27s p=0.625 n=6
Check Time 40.29s (± 0.56%) 40.33s (± 0.13%) ~ 40.24s 40.38s p=0.936 n=6
Emit Time 2.95s (± 1.41%) 3.04s (± 2.52%) +0.09s (+ 2.99%) 2.96s 3.17s p=0.045 n=6
Total Time 51.54s (± 0.54%) 51.63s (± 0.17%) ~ 51.51s 51.72s p=0.936 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,412 +131 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,448 +106 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,415,148k (± 0.05%) 2,415,902k (± 0.04%) ~ 2,414,963k 2,417,577k p=0.128 n=6
Parse Time 7.74s (± 0.76%) 7.63s (± 1.08%) ~ 7.56s 7.73s p=0.066 n=6
Bind Time 2.52s (± 1.29%) 2.53s (± 0.65%) ~ 2.51s 2.56s p=0.574 n=6
Check Time 50.30s (± 0.52%) 50.54s (± 0.79%) ~ 50.18s 51.24s p=0.575 n=6
Emit Time 3.92s (± 3.49%) 3.87s (± 1.94%) ~ 3.73s 3.94s p=0.689 n=6
Total Time 64.49s (± 0.56%) 64.59s (± 0.79%) ~ 64.12s 65.49s p=1.000 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,325 +131 (+ 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,843 +106 (+ 0.10%) ~ ~ p=0.001 n=6
Memory used 427,524k (± 0.03%) 427,685k (± 0.01%) ~ 427,642k 427,717k p=0.065 n=6
Parse Time 3.30s (± 0.56%) 3.22s (± 0.47%) -0.07s (- 2.17%) 3.21s 3.25s p=0.005 n=6
Bind Time 1.33s (± 0.61%) 1.31s (± 1.79%) ~ 1.28s 1.34s p=0.171 n=6
Check Time 17.89s (± 0.30%) 17.84s (± 0.35%) ~ 17.77s 17.91s p=0.230 n=6
Emit Time 1.26s (± 1.30%) 1.25s (± 1.20%) ~ 1.24s 1.28s p=0.287 n=6
Total Time 23.77s (± 0.21%) 23.62s (± 0.27%) -0.15s (- 0.63%) 23.51s 23.67s p=0.005 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,547k (± 0.05%) 369,476k (± 0.02%) ~ 369,393k 369,583k p=0.810 n=6
Parse Time 2.75s (± 1.00%) 2.73s (± 1.06%) ~ 2.69s 2.76s p=0.225 n=6
Bind Time 1.57s (± 0.48%) 1.58s (± 0.67%) ~ 1.56s 1.59s p=0.611 n=6
Check Time 15.44s (± 0.33%) 15.45s (± 0.22%) ~ 15.42s 15.51s p=0.871 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.76s (± 0.31%) 19.74s (± 0.22%) ~ 19.70s 19.81s p=0.686 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,880,017 2,880,017 ~ ~ ~ p=1.000 n=6
Types 975,336 975,336 ~ ~ ~ p=1.000 n=6
Memory used 3,042,842k (± 0.00%) 3,042,791k (± 0.00%) ~ 3,042,683k 3,042,922k p=0.229 n=6
Parse Time 11.45s (± 0.38%) 11.32s (± 0.51%) -0.13s (- 1.12%) 11.27s 11.43s p=0.010 n=6
Bind Time 3.50s (± 0.33%) 3.51s (± 0.43%) ~ 3.49s 3.53s p=0.618 n=6
Check Time 64.76s (± 1.98%) 65.23s (± 1.96%) ~ 63.50s 66.91s p=0.575 n=6
Emit Time 19.25s (± 6.63%) 18.70s (± 6.37%) ~ 17.27s 20.15s p=0.471 n=6
Total Time 98.97s (± 0.40%) 98.75s (± 0.22%) ~ 98.48s 99.02s p=0.173 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,571k (± 0.02%) 411,585k (± 0.01%) ~ 411,507k 411,672k p=0.575 n=6
Parse Time 3.83s (± 0.64%) 3.75s (± 0.62%) -0.08s (- 1.96%) 3.72s 3.78s p=0.005 n=6
Bind Time 1.70s (± 0.71%) 1.69s (± 0.32%) ~ 1.69s 1.70s p=0.865 n=6
Check Time 16.78s (± 0.29%) 16.75s (± 0.30%) ~ 16.67s 16.80s p=0.469 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.31s (± 0.26%) 22.20s (± 0.30%) -0.11s (- 0.48%) 22.11s 22.26s p=0.016 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,791k (± 0.08%) 462,975k (± 0.02%) ~ 462,906k 463,149k p=0.423 n=6
Parse Time 2.65s (± 0.31%) 2.56s (± 0.59%) 🟩-0.08s (- 3.15%) 2.54s 2.58s p=0.004 n=6
Bind Time 0.98s (± 0.53%) 0.98s (± 1.12%) ~ 0.97s 1.00s p=0.784 n=6
Check Time 15.19s (± 0.16%) 15.24s (± 0.48%) ~ 15.13s 15.30s p=0.335 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.82s (± 0.15%) 18.78s (± 0.41%) ~ 18.67s 18.85s p=0.688 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,301ms (± 0.29%) 2,242ms (± 0.71%) -59ms (- 2.56%) 2,221ms 2,259ms p=0.005 n=6
Req 2 - geterr 5,060ms (± 0.37%) 5,075ms (± 0.60%) ~ 5,023ms 5,107ms p=0.335 n=6
Req 3 - references 264ms (± 0.15%) 266ms (± 1.18%) ~ 264ms 272ms p=0.248 n=6
Req 4 - navto 227ms (± 1.48%) 226ms (± 1.40%) ~ 222ms 229ms p=0.607 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 89ms (± 2.30%) 89ms (± 6.61%) ~ 78ms 94ms p=0.531 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,419ms (± 0.55%) 2,360ms (± 0.65%) -60ms (- 2.46%) 2,345ms 2,384ms p=0.005 n=6
Req 2 - geterr 3,797ms (± 0.41%) 3,802ms (± 0.21%) ~ 3,792ms 3,815ms p=0.378 n=6
Req 3 - references 277ms (± 0.30%) 277ms (± 0.15%) ~ 276ms 277ms p=0.285 n=6
Req 4 - navto 226ms (± 0.33%) 226ms (± 0.53%) ~ 224ms 227ms p=0.485 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 73ms (± 8.26%) 73ms (± 8.07%) ~ 69ms 84ms p=1.000 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,217ms (± 0.43%) 6,249ms (± 4.61%) ~ 6,101ms 6,836ms p=0.066 n=6
Req 2 - geterr 1,577ms (±10.42%) 1,582ms (±11.03%) ~ 1,353ms 1,710ms p=0.936 n=6
Req 3 - references 118ms (± 4.38%) 112ms (± 7.33%) ~ 99ms 122ms p=0.213 n=6
Req 4 - navto 606ms (± 2.37%) 599ms (± 2.44%) ~ 580ms 620ms p=0.378 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,268ms (± 2.05%) 1,255ms (± 1.67%) ~ 1,234ms 1,284ms p=0.377 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 188.04ms (± 0.17%) 188.07ms (± 0.17%) ~ 186.53ms 192.34ms p=0.188 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.59ms (± 0.12%) 230.73ms (± 0.12%) +0.14ms (+ 0.06%) 229.43ms 233.17ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 225.92ms (± 0.13%) 226.09ms (± 0.15%) +0.16ms (+ 0.07%) 224.82ms 232.69ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 226.00ms (± 0.13%) 226.46ms (± 0.14%) +0.46ms (+ 0.20%) 224.95ms 231.90ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser
Copy link
Member Author

I'm actually not sure if that's much better or not, it could just be a better run.

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162560/artifacts?artifactName=tgz&fileId=4AA9667F9E1F7B8C6212CF52192D9CDF3B283A368A7D0AD742ABB748FCC2402602&fileName=/typescript-5.6.0-insiders.20240702.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@DanielRosenwasser DanielRosenwasser changed the title Experiments with token scanning Optimize scan with a lookup table to reduce branching Jul 2, 2024
@DanielRosenwasser
Copy link
Member Author

DanielRosenwasser commented Jul 2, 2024 •
edited
Loading

@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
pack this ✅ Started ✅ Results

@DanielRosenwasser
Copy link
Member Author

@typescript-bot test top400
@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top400 ✅ Started ✅ Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 2, 2024 •
edited
Loading

Hey @DanielRosenwasser, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162561/artifacts?artifactName=tgz&fileId=E35843EEA10E387D43B75BB02FEF6B9CF5F7BA42782FB52C26B1840A976DBB6102&fileName=/typescript-5.6.0-insiders.20240702.tgz"
    }
}

and then running npm install.

There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/[email protected]".;

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 193,427k (± 0.92%) 192,159k (± 0.02%) ~ 192,113k 192,244k p=0.298 n=6
Parse Time 1.31s (± 0.75%) 1.24s (± 0.83%) 🟩-0.07s (- 5.21%) 1.23s 1.26s p=0.004 n=6
Bind Time 0.71s 0.71s ~ ~ ~ p=1.000 n=6
Check Time 9.43s (± 0.61%) 9.45s (± 0.26%) ~ 9.40s 9.47s p=0.519 n=6
Emit Time 2.75s (± 0.88%) 2.73s (± 1.37%) ~ 2.66s 2.76s p=0.255 n=6
Total Time 14.20s (± 0.45%) 14.13s (± 0.34%) ~ 14.05s 14.19s p=0.064 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,317k (± 0.00%) 1,218,356k (± 0.00%) ~ 1,218,316k 1,218,402k p=0.092 n=6
Parse Time 7.99s (± 0.52%) 7.88s (± 0.69%) -0.11s (- 1.34%) 7.81s 7.95s p=0.006 n=6
Bind Time 2.22s (± 0.34%) 2.22s (± 0.55%) ~ 2.20s 2.23s p=0.503 n=6
Check Time 35.89s (± 0.71%) 35.81s (± 0.49%) ~ 35.55s 36.04s p=0.688 n=6
Emit Time 16.19s (± 0.34%) 16.19s (± 0.36%) ~ 16.08s 16.24s p=0.809 n=6
Total Time 62.29s (± 0.45%) 62.10s (± 0.28%) ~ 61.88s 62.36s p=0.230 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,136,939 2,136,939 ~ ~ ~ p=1.000 n=6
Types 929,123 929,123 ~ ~ ~ p=1.000 n=6
Memory used 2,117,686k (± 0.01%) 2,117,782k (± 0.00%) ~ 2,117,617k 2,117,884k p=0.230 n=6
Parse Time 9.70s (± 0.34%) 9.54s (± 0.38%) -0.16s (- 1.60%) 9.50s 9.59s p=0.005 n=6
Bind Time 3.38s (± 0.51%) 3.40s (± 0.37%) +0.02s (+ 0.64%) 3.39s 3.42s p=0.035 n=6
Check Time 102.88s (± 0.57%) 101.50s (± 1.84%) ~ 99.72s 103.42s p=0.378 n=6
Emit Time 0.20s (± 4.15%) 0.20s (± 4.47%) ~ 0.19s 0.21s p=0.550 n=6
Total Time 116.15s (± 0.51%) 114.64s (± 1.60%) ~ 112.87s 116.51s p=0.298 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,412 +131 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,448 +106 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,341,955k (± 0.05%) 2,341,458k (± 0.03%) ~ 2,340,732k 2,342,696k p=0.230 n=6
Parse Time 6.01s (± 0.91%) 5.93s (± 0.81%) ~ 5.89s 5.99s p=0.061 n=6
Bind Time 2.26s (± 0.93%) 2.25s (± 0.76%) ~ 2.23s 2.27s p=0.809 n=6
Check Time 40.21s (± 0.31%) 40.29s (± 0.78%) ~ 40.08s 40.92s p=0.936 n=6
Emit Time 3.06s (± 4.86%) 3.06s (± 2.89%) ~ 2.88s 3.11s p=0.574 n=6
Total Time 51.57s (± 0.44%) 51.56s (± 0.67%) ~ 51.22s 52.21s p=0.689 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,412 +131 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,448 +106 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,415,657k (± 0.02%) 2,415,756k (± 0.04%) ~ 2,414,302k 2,417,226k p=0.810 n=6
Parse Time 7.74s (± 0.38%) 7.60s (± 0.79%) -0.14s (- 1.77%) 7.54s 7.68s p=0.005 n=6
Bind Time 2.51s (± 0.70%) 2.51s (± 0.91%) ~ 2.47s 2.53s p=0.628 n=6
Check Time 50.15s (± 0.31%) 50.46s (± 0.63%) +0.31s (+ 0.62%) 50.09s 51.05s p=0.045 n=6
Emit Time 3.82s (± 2.13%) 3.90s (± 2.95%) ~ 3.73s 4.08s p=0.149 n=6
Total Time 64.23s (± 0.21%) 64.48s (± 0.63%) ~ 63.95s 65.17s p=0.109 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,325 +131 (+ 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,843 +106 (+ 0.10%) ~ ~ p=0.001 n=6
Memory used 427,519k (± 0.00%) 427,684k (± 0.01%) +164k (+ 0.04%) 427,650k 427,708k p=0.005 n=6
Parse Time 3.31s (± 1.05%) 3.21s (± 0.65%) 🟩-0.10s (- 3.02%) 3.19s 3.24s p=0.005 n=6
Bind Time 1.32s (± 0.31%) 1.29s (± 1.16%) -0.03s (- 2.14%) 1.28s 1.32s p=0.010 n=6
Check Time 17.82s (± 0.39%) 17.84s (± 0.39%) ~ 17.73s 17.91s p=0.810 n=6
Emit Time 1.25s (± 1.17%) 1.25s (± 0.94%) ~ 1.24s 1.27s p=0.680 n=6
Total Time 23.71s (± 0.33%) 23.60s (± 0.33%) -0.11s (- 0.46%) 23.48s 23.68s p=0.045 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,504k (± 0.02%) 369,545k (± 0.03%) ~ 369,396k 369,689k p=0.575 n=6
Parse Time 2.77s (± 1.64%) 2.71s (± 0.77%) -0.06s (- 2.11%) 2.67s 2.73s p=0.029 n=6
Bind Time 1.58s (± 0.96%) 1.58s (± 0.33%) ~ 1.57s 1.58s p=0.351 n=6
Check Time 15.48s (± 0.24%) 15.48s (± 0.25%) ~ 15.41s 15.51s p=1.000 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.83s (± 0.22%) 19.77s (± 0.28%) -0.07s (- 0.34%) 19.66s 19.82s p=0.037 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,780 2,879,780 ~ ~ ~ p=1.000 n=6
Types 975,260 975,260 ~ ~ ~ p=1.000 n=6
Memory used 3,042,725k (± 0.00%) 3,042,710k (± 0.00%) ~ 3,042,618k 3,042,860k p=0.810 n=6
Parse Time 13.69s (± 0.56%) 13.54s (± 0.36%) -0.16s (- 1.16%) 13.47s 13.62s p=0.008 n=6
Bind Time 4.19s (± 0.25%) 4.26s (± 2.08%) +0.07s (+ 1.71%) 4.18s 4.41s p=0.037 n=6
Check Time 73.60s (± 0.73%) 74.17s (± 1.36%) ~ 73.00s 75.93s p=0.298 n=6
Emit Time 24.11s (± 2.28%) 23.59s (± 4.16%) ~ 21.91s 24.44s p=0.687 n=6
Total Time 115.59s (± 0.33%) 115.56s (± 0.24%) ~ 115.18s 115.86s p=0.810 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,550k (± 0.02%) 411,617k (± 0.02%) ~ 411,549k 411,731k p=0.093 n=6
Parse Time 3.18s (± 0.37%) 3.12s (± 0.43%) -0.06s (- 1.78%) 3.10s 3.14s p=0.004 n=6
Bind Time 1.41s (± 0.37%) 1.41s (± 0.59%) ~ 1.39s 1.41s p=0.070 n=6
Check Time 14.21s (± 0.25%) 14.21s (± 0.26%) ~ 14.14s 14.24s p=0.936 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.81s (± 0.22%) 18.73s (± 0.26%) -0.08s (- 0.40%) 18.64s 18.78s p=0.009 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 463,001k (± 0.07%) 463,009k (± 0.05%) ~ 462,622k 463,281k p=0.810 n=6
Parse Time 2.66s (± 0.39%) 2.56s (± 0.67%) 🟩-0.09s (- 3.52%) 2.54s 2.58s p=0.005 n=6
Bind Time 0.98s (± 0.77%) 0.97s (± 0.86%) ~ 0.97s 0.99s p=0.172 n=6
Check Time 15.19s (± 0.26%) 15.25s (± 0.37%) ~ 15.20s 15.34s p=0.126 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.82s (± 0.23%) 18.79s (± 0.34%) ~ 18.74s 18.91s p=0.334 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,294ms (± 0.34%) 2,235ms (± 0.76%) -60ms (- 2.59%) 2,223ms 2,265ms p=0.005 n=6
Req 2 - geterr 5,061ms (± 0.50%) 5,077ms (± 0.46%) ~ 5,050ms 5,116ms p=0.298 n=6
Req 3 - references 265ms (± 1.25%) 264ms (± 0.28%) ~ 263ms 265ms p=0.931 n=6
Req 4 - navto 227ms (± 0.53%) 227ms (± 1.18%) ~ 222ms 230ms p=0.931 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 87ms (± 3.88%) 88ms (± 6.22%) ~ 78ms 93ms p=0.391 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,431ms (± 0.56%) 2,364ms (± 0.24%) -67ms (- 2.77%) 2,357ms 2,372ms p=0.005 n=6
Req 2 - geterr 3,798ms (± 0.18%) 3,791ms (± 0.42%) ~ 3,774ms 3,816ms p=0.378 n=6
Req 3 - references 277ms (± 0.29%) 276ms (± 0.46%) ~ 275ms 278ms p=0.082 n=6
Req 4 - navto 227ms (± 0.28%) 229ms (± 2.83%) ~ 225ms 242ms p=0.438 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 79ms (± 9.11%) 78ms (± 8.10%) ~ 69ms 85ms p=0.677 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 6,342ms (± 5.02%) 6,109ms (± 0.55%) 🟩-233ms (- 3.67%) 6,056ms 6,160ms p=0.005 n=6
Req 2 - geterr 1,597ms (±10.84%) 1,585ms (±10.42%) ~ 1,364ms 1,721ms p=0.689 n=6
Req 3 - references 113ms (± 4.02%) 114ms (± 4.48%) ~ 110ms 121ms p=0.561 n=6
Req 4 - navto 597ms (± 2.78%) 596ms (± 1.51%) ~ 586ms 609ms p=0.810 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,251ms (± 1.60%) 1,254ms (± 1.32%) ~ 1,234ms 1,274ms p=0.873 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.17ms (± 0.20%) 157.21ms (± 0.16%) +0.03ms (+ 0.02%) 156.22ms 160.96ms p=0.047 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.32ms (± 0.21%) 230.43ms (± 0.14%) +0.11ms (+ 0.05%) 229.08ms 236.21ms p=0.000 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.63ms (± 0.14%) 226.87ms (± 0.14%) +0.24ms (+ 0.11%) 225.59ms 231.38ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 275.26ms (± 0.28%) 276.27ms (± 0.73%) +1.01ms (+ 0.37%) 269.29ms 308.09ms p=0.000 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@DanielRosenwasser DanielRosenwasser marked this pull request as ready for review July 2, 2024 20:30
@typescript-bot
Copy link
Collaborator

@DanielRosenwasser Here are the results of running the top 400 repos with tsc comparing main and refs/pull/59106/merge:

Everything looks good!

@DanielRosenwasser
Copy link
Member Author

@dragomirtitian maybe this balances out the parse-time hit from #58928. 😄

@DanielRosenwasser
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jul 3, 2024 •
edited
Loading

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this ✅ Started 👀 Results

@typescript-bot
Copy link
Collaborator

@DanielRosenwasser
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 192,850k (± 0.77%) 193,503k (± 0.93%) ~ 192,149k 195,866k p=0.261 n=6
Parse Time 1.58s (± 0.48%) 1.50s (± 1.09%) 🟩-0.08s (- 4.85%) 1.49s 1.53s p=0.005 n=6
Bind Time 0.86s (± 1.41%) 0.85s (± 1.16%) ~ 0.84s 0.86s p=0.246 n=6
Check Time 11.18s (± 0.60%) 11.21s (± 0.17%) ~ 11.18s 11.23s p=0.520 n=6
Emit Time 3.28s (± 0.76%) 3.29s (± 0.95%) ~ 3.25s 3.32s p=0.808 n=6
Total Time 16.90s (± 0.19%) 16.85s (± 0.19%) ~ 16.82s 16.89s p=0.065 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,114 944,114 ~ ~ ~ p=1.000 n=6
Types 407,050 407,050 ~ ~ ~ p=1.000 n=6
Memory used 1,218,371k (± 0.00%) 1,218,388k (± 0.00%) ~ 1,218,311k 1,218,446k p=0.575 n=6
Parse Time 7.96s (± 0.64%) 7.90s (± 0.49%) -0.07s (- 0.82%) 7.84s 7.93s p=0.030 n=6
Bind Time 2.24s (± 1.08%) 2.22s (± 0.66%) ~ 2.21s 2.25s p=0.145 n=6
Check Time 35.73s (± 0.40%) 35.87s (± 0.43%) ~ 35.65s 36.13s p=0.171 n=6
Emit Time 16.11s (± 0.32%) 16.18s (± 0.22%) ~ 16.14s 16.23s p=0.052 n=6
Total Time 62.04s (± 0.33%) 62.17s (± 0.27%) ~ 61.90s 62.43s p=0.229 n=6
mui-docs - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,154,949 2,154,949 ~ ~ ~ p=1.000 n=6
Types 931,473 931,473 ~ ~ ~ p=1.000 n=6
Memory used 2,125,930k (± 0.00%) 2,126,089k (± 0.00%) +159k (+ 0.01%) 2,126,001k 2,126,145k p=0.005 n=6
Parse Time 9.77s (± 0.34%) 9.56s (± 0.33%) -0.21s (- 2.20%) 9.52s 9.61s p=0.005 n=6
Bind Time 3.39s (± 0.37%) 3.38s (± 0.51%) ~ 3.36s 3.40s p=0.415 n=6
Check Time 103.96s (± 0.62%) 103.99s (± 0.33%) ~ 103.33s 104.29s p=1.000 n=6
Emit Time 0.19s (± 2.13%) 0.20s 🔻+0.01s (+ 4.35%) ~ ~ p=0.007 n=6
Total Time 117.31s (± 0.55%) 117.12s (± 0.31%) ~ 116.41s 117.43s p=0.297 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,413 +132 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,448 +106 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,342,013k (± 0.03%) 2,341,774k (± 0.04%) ~ 2,340,479k 2,343,198k p=0.471 n=6
Parse Time 6.03s (± 1.12%) 5.99s (± 1.09%) ~ 5.91s 6.09s p=0.336 n=6
Bind Time 2.26s (± 0.87%) 2.27s (± 1.02%) ~ 2.23s 2.29s p=0.686 n=6
Check Time 40.22s (± 0.60%) 40.42s (± 0.49%) ~ 40.11s 40.63s p=0.128 n=6
Emit Time 3.05s (± 1.85%) 3.11s (± 6.04%) ~ 2.93s 3.47s p=0.873 n=6
Total Time 51.56s (± 0.52%) 51.79s (± 0.47%) ~ 51.37s 52.11s p=0.128 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,225,281 1,225,413 +132 (+ 0.01%) ~ ~ p=0.001 n=6
Types 261,342 261,448 +106 (+ 0.04%) ~ ~ p=0.001 n=6
Memory used 2,414,808k (± 0.05%) 2,415,265k (± 0.03%) ~ 2,414,227k 2,416,053k p=0.689 n=6
Parse Time 7.79s (± 0.85%) 7.64s (± 0.36%) -0.15s (- 1.95%) 7.61s 7.69s p=0.008 n=6
Bind Time 2.51s (± 0.48%) 2.52s (± 1.03%) ~ 2.48s 2.55s p=0.518 n=6
Check Time 50.32s (± 0.43%) 50.45s (± 0.31%) ~ 50.17s 50.62s p=0.298 n=6
Emit Time 3.94s (± 2.65%) 3.99s (± 4.57%) ~ 3.89s 4.36s p=0.298 n=6
Total Time 64.58s (± 0.34%) 64.61s (± 0.40%) ~ 64.20s 64.99s p=0.936 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,194 258,326 +132 (+ 0.05%) ~ ~ p=0.001 n=6
Types 104,737 104,843 +106 (+ 0.10%) ~ ~ p=0.001 n=6
Memory used 427,488k (± 0.01%) 427,701k (± 0.01%) +213k (+ 0.05%) 427,650k 427,748k p=0.005 n=6
Parse Time 3.28s (± 0.60%) 3.26s (± 0.19%) -0.02s (- 0.71%) 3.25s 3.27s p=0.048 n=6
Bind Time 1.31s (± 1.12%) 1.32s (± 1.82%) ~ 1.29s 1.35s p=0.514 n=6
Check Time 17.76s (± 0.29%) 17.77s (± 0.35%) ~ 17.69s 17.87s p=1.000 n=6
Emit Time 1.25s (± 1.31%) 1.25s (± 1.20%) ~ 1.24s 1.27s p=0.458 n=6
Total Time 23.60s (± 0.30%) 23.60s (± 0.37%) ~ 23.51s 23.75s p=0.810 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 224,565 ~ ~ ~ p=1.000 n=6
Types 93,734 93,734 ~ ~ ~ p=1.000 n=6
Memory used 369,475k (± 0.04%) 369,514k (± 0.04%) ~ 369,393k 369,750k p=0.298 n=6
Parse Time 2.76s (± 0.88%) 2.74s (± 1.41%) ~ 2.68s 2.79s p=0.376 n=6
Bind Time 1.59s (± 0.94%) 1.59s (± 1.30%) ~ 1.58s 1.62s p=0.666 n=6
Check Time 15.46s (± 0.26%) 15.49s (± 0.36%) ~ 15.42s 15.56s p=0.469 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 19.81s (± 0.18%) 19.83s (± 0.30%) ~ 19.78s 19.93s p=0.747 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,879,791 2,879,791 ~ ~ ~ p=1.000 n=6
Types 975,122 975,122 ~ ~ ~ p=1.000 n=6
Memory used 3,043,017k (± 0.00%) 3,042,990k (± 0.00%) ~ 3,042,960k 3,043,023k p=0.298 n=6
Parse Time 13.71s (± 0.75%) 13.56s (± 0.41%) -0.14s (- 1.06%) 13.47s 13.62s p=0.016 n=6
Bind Time 4.30s (± 2.90%) 4.19s (± 0.30%) ~ 4.18s 4.21s p=0.369 n=6
Check Time 73.88s (± 2.34%) 74.11s (± 2.25%) ~ 73.21s 77.49s p=0.092 n=6
Emit Time 23.67s (± 6.92%) 23.66s (± 6.60%) ~ 20.50s 24.48s p=0.810 n=6
Total Time 115.55s (± 0.26%) 115.52s (± 0.34%) ~ 114.87s 115.91s p=1.000 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 267,117 267,117 ~ ~ ~ p=1.000 n=6
Types 108,775 108,775 ~ ~ ~ p=1.000 n=6
Memory used 411,562k (± 0.02%) 411,552k (± 0.01%) ~ 411,496k 411,643k p=0.936 n=6
Parse Time 3.18s (± 0.67%) 3.14s (± 0.20%) -0.04s (- 1.21%) 3.13s 3.15s p=0.004 n=6
Bind Time 1.41s 1.42s (± 0.39%) ~ 1.41s 1.42s p=0.071 n=6
Check Time 14.20s (± 0.31%) 14.16s (± 0.31%) ~ 14.09s 14.22s p=0.127 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 18.79s (± 0.27%) 18.71s (± 0.26%) -0.08s (- 0.41%) 18.64s 18.78s p=0.037 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 525,251 525,251 ~ ~ ~ p=1.000 n=6
Types 178,574 178,574 ~ ~ ~ p=1.000 n=6
Memory used 462,855k (± 0.08%) 462,947k (± 0.04%) ~ 462,673k 463,208k p=0.810 n=6
Parse Time 3.93s (± 0.57%) 3.83s (± 0.26%) -0.10s (- 2.58%) 3.82s 3.84s p=0.004 n=6
Bind Time 1.46s (± 1.28%) 1.46s (± 0.71%) ~ 1.44s 1.47s p=1.000 n=6
Check Time 22.15s (± 0.28%) 22.17s (± 0.28%) ~ 22.09s 22.27s p=0.463 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.54s (± 0.24%) 27.47s (± 0.26%) ~ 27.37s 27.57s p=0.127 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,295ms (± 0.54%) 2,244ms (± 0.54%) -50ms (- 2.19%) 2,230ms 2,259ms p=0.005 n=6
Req 2 - geterr 5,068ms (± 0.37%) 5,072ms (± 0.32%) ~ 5,056ms 5,100ms p=0.689 n=6
Req 3 - references 263ms (± 0.44%) 264ms (± 0.50%) ~ 262ms 266ms p=0.216 n=6
Req 4 - navto 229ms (± 0.81%) 227ms (± 0.72%) ~ 224ms 228ms p=0.177 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 84ms (± 8.24%) 87ms (± 5.91%) ~ 78ms 94ms p=0.432 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 2,414ms (± 0.41%) 2,387ms (± 0.59%) -27ms (- 1.13%) 2,370ms 2,406ms p=0.008 n=6
Req 2 - geterr 3,794ms (± 0.37%) 3,795ms (± 0.33%) ~ 3,780ms 3,809ms p=1.000 n=6
Req 3 - references 277ms (± 0.29%) 277ms (± 0.35%) ~ 275ms 278ms p=0.604 n=6
Req 4 - navto 226ms (± 0.33%) 227ms (± 0.18%) ~ 226ms 227ms p=0.100 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 75ms (± 7.21%) 77ms (±10.31%) ~ 69ms 85ms p=1.000 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 5,182ms (± 0.20%) 5,116ms (± 0.28%) -66ms (- 1.28%) 5,100ms 5,135ms p=0.005 n=6
Req 2 - geterr 1,129ms (± 0.88%) 1,123ms (± 0.18%) ~ 1,120ms 1,125ms p=0.196 n=6
Req 3 - references 77ms (± 4.67%) 75ms (± 0.54%) ~ 74ms 75ms p=0.114 n=6
Req 4 - navto 460ms (± 1.04%) 457ms (± 0.11%) ~ 457ms 458ms p=1.000 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 834ms (± 1.52%) 832ms (± 1.05%) ~ 823ms 848ms p=0.748 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 157.43ms (± 0.16%) 157.38ms (± 0.17%) ~ 156.21ms 159.84ms p=0.074 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 230.50ms (± 0.15%) 230.58ms (± 0.15%) +0.08ms (+ 0.04%) 229.33ms 236.14ms p=0.025 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 226.02ms (± 0.15%) 226.16ms (± 0.17%) +0.14ms (+ 0.06%) 224.63ms 232.34ms p=0.000 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 225.25ms (± 0.15%) 225.33ms (± 0.13%) +0.08ms (+ 0.04%) 224.05ms 227.31ms p=0.003 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@DanielRosenwasser DanielRosenwasser

Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@DanielRosenwasser @typescript-bot