[iOS] composition items memory issue

[jinookoo]

in FramesExtrator, Exporter
initialize 'VideoCompositionItemDecoder' at once
So, If a lot of items make OOM crash.
This problem can be solved by using decoder pool only for required items
Then I can preview a lot of composition items.

However, the frame generation loop in the exporter still has memory issues.
If you are using 'Instructions' for monitoring memory allocation.
It continues to increase until it is finished.
so short composition items will be fine but long & heavy video have same problem.

CVPixelBufferCreate return kCVReturnAllocationFailed
create MTLTexture return null

But It already been called

CVPixelBufferRelease(pixelBuffer);
[cpuAccessibleTexture setPurgeableState:MTLPurgeableStateEmpty];

Can anyone help me with this?

[jinookoo]

@fdecampredon 🥹

[fdecampredon]

For preview, the problem with using VideoCompositionItemDecoder on demand is for live composition playing. for example let's say you need to play a video at frame 60, if we don't pre-create the decoder it might not be ready in time.

For export, we'll try to investigate memory issue, but in general most of the memory allocated is done by AVAssetReader, which we don't have the control over.
However, we could use lazy VideoCompositionItemDecoder creation.

Just for info are you using the resolution field in your composition item ? Downscaling the resolution has a great impact on memory issues.

[jinookoo]

Oh, I'm missing 'resolution' in composition item. It pretty good for better preview.

But it still have many composition items will load at once.

How about initialize items will renderer by compositionStartTime, startTime, duration.
Here is PR

and exporting memory allocation fixed PR