Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significant performance regression in 1.5.29/ 1.5.31 #61

Closed
Vintagemotors opened this issue Jan 11, 2024 · 9 comments
Closed

Significant performance regression in 1.5.29/ 1.5.31 #61

Vintagemotors opened this issue Jan 11, 2024 · 9 comments

Comments

@Vintagemotors
Copy link
Collaborator

1.5.29 tested 5 times avg 73.5

image

1.5.28 tested avg 80

image

Site background is also being improperly recolored in 1.5.29 but I will open a separate issue for that.

@ThomazPom
Copy link
Owner

ThomazPom commented Jan 13, 2024

I've just released two versions of UltimaDark: 1.5.31 and 1.5.32.

Both address a portion of the performance drop, with 1.5.32 fully resolving the issue but presenting notable drawbacks.

In 1.5.31, I disabled the UltimaDark "Direct window export system," opting for the code to leverage the native Firefox export system, which is 60% slower in website post-load operations. The install time remains 2.0566037735849 miliseconds (with a range of 1ms to 6ms).

In 1.5.32, I reinstated the UltimaDark "Direct window export system," despite reservations due to its too much aggressive nature (acknowledging that UltimaDark philosophy embraces aggressive techniques). However, this approach may encounter issues when websites use polyfills with the same name as a function employed by UltimaDark, potentially causing breaks until I explicitly forbid such polyfills. (I've identified one instance on commons.wikimedia.org, but there could be numerous possibilities.) The install time is 2.0566037735849 miliseconds (ranging from 1ms to 5ms). After identifying and forbidding a polyfill, it needs to be injected preemptively across all websites, proving useful very rarely.

The Speedometer2.1 score from https://browserbench.org/Speedometer2.1/ aims to measure page load speed but also encompasses post-load operations. The use of the native Firefox export system results in a 60% slowdown in these operations, causing an 8% drop in the overall score.

It's crucial to note that these post-load operation delays, to the best of my knowledge, are imperceptible to UltimaDark users as they are non-blocking and have no impact on webpage loading.

Visible demonstration

This slowdown becomes evident in a specialized test: https://jsben.ch/GP26t

1.5.31 20% speed against common comparator

image

1.5.32 : 50% speed against common comparator

image

The question arises whether an 8% speed decrease, potentially imperceptible to users, justifies increased code complexity and some rare websites not being directly covered.

For now, I will maintain the native Firefox export system. If the consequences of this decision become too perceptible, I have made the transition back to the UltimaDark "Direct window export system" very straightforward

#53 (comment)

@Vintagemotors
Copy link
Collaborator Author

Vintagemotors commented Jan 13, 2024

1.5.31 has the same performance regression as 1.5.29 avg 73
image
image

@Vintagemotors Vintagemotors changed the title Significant performance regression in 1.5.29 Significant performance regression in 1.5.29/ 1.5.31 Jan 13, 2024
@Vintagemotors
Copy link
Collaborator Author

This is a run immediately after with 1.5.28
image

@ThomazPom
Copy link
Owner

ThomazPom commented Jan 13, 2024

1.5.30 was refused by Mozilla because i've published 1.5.31 while 1.5.30 was still under review.
I've published 1.5.32 and adjusted the comment text #61 (comment)

@Vintagemotors
Copy link
Collaborator Author

Performance regression largely reversed by 1.5.32 (1.5.30) I am seeing an average of 79 R/M which can be explained by background processes.

@ThomazPom
Copy link
Owner

ThomazPom commented Jan 13, 2024

Although the performance drop is quantifiable, ranging from 5% to 20% according to browserbench metrics, my understanding on page loading leads me to believe that this decline is non blocking for webpage , making it probably non perceptible to users.

Interestingly, I occasionally observe comparable scores between 1.5.31 & 1.5.32 on browserbench, and I haven't identified a clear explanation for this.

@Vintagemotors
Copy link
Collaborator Author

Vintagemotors commented Jan 13, 2024

Speedometer 2.0 and 2.1 differ in that 2.0 had a delay timer added in some cases which 2.1 removes and 2.1 uses newer versions of the frameworks used in the test which changed their performance characteristics. Generally speaking 2.1 should be slightly faster than 2.0 but I use 2.0 for testing because I have a larger sample size in it and the results are generally consistent between runs.

@Vintagemotors
Copy link
Collaborator Author

Vintagemotors commented Jan 13, 2024

Interestingly, I occasionally observe comparable scores between 1.5.31 & 1.5.32

I have not encountered this.

@Vintagemotors
Copy link
Collaborator Author

Within margin of error on 1.5.33

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

2 participants