Optimise will read frequently

[neopheus]

Motivation

I am proposing this change to optimize the performance of getImageData when multiple read operations are being performed. This optimization is particularly beneficial when frequently using filters, where the creation of helper layers can be avoided at each filter step.

Description

I have added the willReadFrequently: true option when retrieving the 2D context of the canvas. This signals to the browser that we plan to read the canvas data frequently, allowing the browser to optimize read operations.

var ctx = targetCanvas.getContext('2d', { willReadFrequently: true });

[asturur]

I think this could be fine.
The reason why i didn't bother with it before is that they do not explain exactly what this optimization is and why is not always on.

this target canvas will be the base for our filtered image as soon as the filtering ends.
How does it performs then in subsequent draw on a non optimized canvas?

Can you measure both the improvements you get from this change and how the drawing of the image then performs later?

The slow part of filtering with cpu is looping over image data.
We do read it once and then loop over it more than once, how much is the speed benefit of reading here?

[neopheus]

Source : https://github.com/fserb/canvas2D/blob/master/spec/will-read-frequently.md

There's an important use case for apps/games that are very reliant on getImageData. This operation is particularly slow when Canvas2D is being backed by accelerated graphics (GPU), that speed up considerably most other use cases. Historically, browsers have tried heuristics to decide if a canvas is being read frequently or not, to switch code paths. This is unreliable, complex and brittle. To prevent this, we propose a hint option that will allow developers to tell the browser that this is a canvas that rely on getImageData performance.

When the user sets willReadFrequently to true, the UA can optimize for read access, usually by not using the GPU for rendering.

[ShaMan123]

I agree it should be optional somehow wherever it is needed

[asturur]

Source : https://github.com/fserb/canvas2D/blob/master/spec/will-read-frequently.md

There's an important use case for apps/games that are very reliant on getImageData. This operation is particularly slow when Canvas2D is being backed by accelerated graphics (GPU), that speed up considerably most other use cases. Historically, browsers have tried heuristics to decide if a canvas is being read frequently or not, to switch code paths. This is unreliable, complex and brittle. To prevent this, we propose a hint option that will allow developers to tell the browser that this is a canvas that rely on getImageData performance.

When the user sets willReadFrequently to true, the UA can optimize for read access, usually by not using the GPU for rendering.

Sorry for not getting back to you sooner.
What i asked was entirely different.
The place where you added that option is when we read the imageData once per applyFilter without webgl.
All the slowness of filtering without webgl comes from the loops over the imageData, that happens in the cpu domain with or without the optimization

On the other side that canvas is now optimized for read from CPU but then it gets used again in a gpu context for rendering on the canvas. applyFilters() happens once, rendering on the canvas will happen at 60fps ( hopefully ).

What i was asking was, can you actually measure performance changes with this code change? Can you show a flamegraph and point out with an arrow showing before / after that this is actually improving something and not damaging something else?

I see this optimization useful only if you do a tileset and you need to read pieces constantly because for some reason you can't just read them all at once and keep them in memory, or when you need to update a drawing often and pass the content across a postMessage, or pass it to some tensor library.

To be even more clear, i suspect this won't improve anything, and has a slight chance to slow down something else, but since i don't have time to dig, i m asking you to do it

[asturur]

On the contrary this could be useful on the transparent pixel detection.
(still needs a proof of concept since last optimizations changed how it works and now more drawing happens rather than reads )