[Flock][Feature] - Mutate bitmap data directly from Dart

[matthew-carroll]

Flutter Issue Ticket Link:
flutter/flutter#37180

Why didn't Flutter implement this?
The team said combinations of "it's hard", "it's ambiguous", "it's impossible".

Why does this belong in Flock?
The goal requires adding or altering the boundary between the framework and the engine, which can't be done in a package.

Describe the solution you'd like
I don't have strong opinions about APIs, or even which problem is solved. There are at least a couple places that a solution could be formulated:

• Make it possible to change pixels on an Image object (or create a MutableImage). Currently, one has to copy the pixels from the engine into Dart, modify the pixels, then send the pixels back to the engine.
• Introduce a canvas method for drawing pixels.

Additional context
This limitation was the most impactful blocker when working on flutter_processing in terms of being able to alter bitmap data in real time. Many Processing APIs are bitmap-centric. You set individual pixel values. This makes it possible for visual effects where the next frame is applied to the previous frame.

These effects can generally be accomplished with shaders. However, shaders require knowledge of a shader language, and involve a completely separate development and integration pathway. It would be nice if Flutter developers could leverage their Dart and Flutter abilities to at least create proofs of concept before worrying about shader languages.

Flutter Processing: https://github.com/matthew-carroll/flutter_processing

[jezell]

This is kind of why the Texture stuff is there, but as pointed out in the issue, the challenge with Texture is that the assumption (at the moment) is that you are always mutating the texture from native code, and there aren't hooks for interacting with the textures directly. Maybe some higher level texture support? I've requested it multiple times, starting with just basic support for Texture on web:

flutter/flutter#119649

[jezell]

Also related, though maybe not directly, support for video textures is really something that would be nice out of the box:

flutter/flutter#150592

Same type of challenge, that you need better hooks to have any chance of it performing well.

[matthew-carroll]

@jezell can you file a ticket for the video texture, along with a motivating use-case, and ideally provide any breadcrumbs for a contributor to start looking in the right parts of the engine?

[jezell]

@matthew-carroll sure

[jezell]

#45 and #46 added which could be related to this.