Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[media] Limit max number of buffers to write with audio write ahead #3271

Merged
merged 1 commit into from
Jun 11, 2024

Conversation

borongc
Copy link
Contributor

@borongc borongc commented May 16, 2024

Limit the maximum number of reads from DemuxerStream in the range of [1, max_number_of_buffers_to_write]. Cobalt can support writing of multiple samples to speed up playback start and seek (b/227837774). However, to avoid writing excessive audio data to SbPlayer, Cobalt estimates the maximum number of writes with audio write ahead |audio_write_duration_|. The number of writes is the minimum value of the estimated value and |max_number_of_buffers_to_write| if Cobalt allows batched sample write via H5VCC flag.

For audio preroll:

  1. When audio codec is aac, opus, or e/ac3, Cobalt uses default numbers as maximum number of writes, which are computed by commonly used sampling rate and frames per DecoderBuffer.
  2. For other audio codecs, Cobalt uses the time difference between input buffers to estimate how many input buffers to write with |audio_write_duration_|.
  3. The maximum number of sample writes for IAMF is pending implementation in b/41486346.

After audio preroll, Cobalt estimates the number of writes with the difference between |audio_write_duration_| and |time_ahead_of_playback|.

b/342484772

Test-On-Device: true

@borongc borongc force-pushed the media_multiple_write branch 4 times, most recently from 6920aa9 to 9f19098 Compare May 17, 2024 21:20
cobalt/media/base/sbplayer_pipeline.cc Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
@borongc borongc force-pushed the media_multiple_write branch 5 times, most recently from 0bbd48c to 6391e17 Compare May 21, 2024 18:57
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.h Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
@borongc borongc force-pushed the media_multiple_write branch 4 times, most recently from ffea18a to 6e1fd97 Compare May 22, 2024 00:05
cobalt/media/base/sbplayer_pipeline.h Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
@borongc borongc force-pushed the media_multiple_write branch 5 times, most recently from b2180f7 to 188b88f Compare May 24, 2024 18:23
@borongc borongc requested a review from xiaomings May 24, 2024 18:23
@borongc borongc force-pushed the media_multiple_write branch 3 times, most recently from 3001978 to 85342ba Compare May 29, 2024 16:19
@borongc borongc requested a review from xiaomings June 10, 2024 23:36
borongc added a commit that referenced this pull request Jun 10, 2024
Refer to the original PR: #3513

1. Refactor for PR (#3271) that
limits the maximum number of samples per write to SbPlayer with audio
write ahead.
2. This PR doesn't have any functional change.

b/342484772

Co-authored-by: Bo-Rong Chen <[email protected]>
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
cobalt/media/base/sbplayer_pipeline.cc Outdated Show resolved Hide resolved
@borongc borongc force-pushed the media_multiple_write branch 2 times, most recently from a61d111 to a0fb1af Compare June 11, 2024 17:23
@borongc borongc requested a review from xiaomings June 11, 2024 17:27
@borongc borongc force-pushed the media_multiple_write branch 5 times, most recently from b47f9e4 to c8031ca Compare June 11, 2024 18:50
@borongc borongc merged commit 147a2ed into youtube:main Jun 11, 2024
356 of 366 checks passed
@borongc borongc deleted the media_multiple_write branch June 11, 2024 22:16
@borongc borongc added the cp-25.lts.1+ Cherry Pick to the 25.lts.1+ branch label Jun 11, 2024
cobalt-github-releaser-bot pushed a commit that referenced this pull request Jun 11, 2024
…3271)

Limit the maximum number of reads from DemuxerStream in the range of [1,
`max_number_of_buffers_to_write`]. Cobalt can support writing of
multiple samples to speed up playback start and seek (b/227837774).
However, to avoid writing excessive audio data to SbPlayer, Cobalt
estimates the maximum number of writes with audio write ahead
|audio_write_duration_|. The number of writes is the minimum value of
the estimated value and |max_number_of_buffers_to_write| if Cobalt
allows batched sample write via H5VCC flag.

For audio preroll:
1. When audio codec is aac, opus, or e/ac3, Cobalt uses default numbers
as maximum number of writes, which are computed by commonly used
sampling rate and frames per DecoderBuffer.
2. For other audio codecs, Cobalt uses the time difference between input
buffers to estimate how many input buffers to write with
|audio_write_duration_|.
3. The maximum number of sample writes for IAMF is pending
implementation in b/41486346.

After audio preroll, Cobalt estimates the number of writes with the
difference between |audio_write_duration_| and |time_ahead_of_playback|.

b/342484772

Test-On-Device: true
(cherry picked from commit 147a2ed)
borongc added a commit that referenced this pull request Jun 11, 2024
…th audio write ahead (#3525)

Refer to the original PR: #3271

Limit the maximum number of reads from DemuxerStream in the range of [1,
`max_number_of_buffers_to_write`]. Cobalt can support writing of
multiple samples to speed up playback start and seek (b/227837774).
However, to avoid writing excessive audio data to SbPlayer, Cobalt
estimates the maximum number of writes with audio write ahead
|audio_write_duration_|. The number of writes is the minimum value of
the estimated value and |max_number_of_buffers_to_write| if Cobalt
allows batched sample write via H5VCC flag.

For audio preroll:
1. When audio codec is aac, opus, or e/ac3, Cobalt uses default numbers
as maximum number of writes, which are computed by commonly used
sampling rate and frames per DecoderBuffer.
2. For other audio codecs, Cobalt uses the time difference between input
buffers to estimate how many input buffers to write with
|audio_write_duration_|.
3. The maximum number of sample writes for IAMF is pending
implementation in b/41486346.

After audio preroll, Cobalt estimates the number of writes with the
difference between |audio_write_duration_| and |time_ahead_of_playback|.

b/342484772

Test-On-Device: true

Co-authored-by: Bo-Rong Chen <[email protected]>
borongc added a commit that referenced this pull request Jun 13, 2024
…SbPlayer from webapp (#3325)

The PR (#3271) limits the maximum
number of samples per write to SbPlayer with audio write ahead. This PR
refactors the existing H5VCC flag from "EnableBatchedSampleWrite" to
"AudioBatchedSampleWrite" that allows configuring maximum number of
samples per write to SbPlayer from webapp.

1. When "AudioBatchedSampleWrite=1" or not set, Cobalt uses the default
number of samples per write (i.e., 1) to SbPlayer.
2. When "AudioBatchedSampleWrite=n", where n > 1, Cobalt uses the number
of samples per write estimated by GetEstimatedMaxBuffers() to SbPlayer.

This PR also changes SbPlayerGetMaximumNumberOfSamplesPerWrite() to 256
on Android TV. Note in case 2, the number of samples per write is in the
range of [1, min(SbPlayerGetMaximumNumberOfSamplesPerWrite(),
AudioBatchedSampleWrite)]. For other platforms except Android,
SbPlayerGetMaximumNumberOfSamplesPerWrite should return 1.

b/227837774

Test-On-Device: true
cobalt-github-releaser-bot pushed a commit that referenced this pull request Jun 13, 2024
…SbPlayer from webapp (#3325)

The PR (#3271) limits the maximum
number of samples per write to SbPlayer with audio write ahead. This PR
refactors the existing H5VCC flag from "EnableBatchedSampleWrite" to
"AudioBatchedSampleWrite" that allows configuring maximum number of
samples per write to SbPlayer from webapp.

1. When "AudioBatchedSampleWrite=1" or not set, Cobalt uses the default
number of samples per write (i.e., 1) to SbPlayer.
2. When "AudioBatchedSampleWrite=n", where n > 1, Cobalt uses the number
of samples per write estimated by GetEstimatedMaxBuffers() to SbPlayer.

This PR also changes SbPlayerGetMaximumNumberOfSamplesPerWrite() to 256
on Android TV. Note in case 2, the number of samples per write is in the
range of [1, min(SbPlayerGetMaximumNumberOfSamplesPerWrite(),
AudioBatchedSampleWrite)]. For other platforms except Android,
SbPlayerGetMaximumNumberOfSamplesPerWrite should return 1.

b/227837774

Test-On-Device: true
(cherry picked from commit ed62b35)
borongc added a commit that referenced this pull request Jun 14, 2024
…SbPlayer from webapp (#3325)

The PR (#3271) limits the maximum
number of samples per write to SbPlayer with audio write ahead. This PR
refactors the existing H5VCC flag from "EnableBatchedSampleWrite" to
"AudioBatchedSampleWrite" that allows configuring maximum number of
samples per write to SbPlayer from webapp.

1. When "AudioBatchedSampleWrite=1" or not set, Cobalt uses the default
number of samples per write (i.e., 1) to SbPlayer.
2. When "AudioBatchedSampleWrite=n", where n > 1, Cobalt uses the number
of samples per write estimated by GetEstimatedMaxBuffers() to SbPlayer.

This PR also changes SbPlayerGetMaximumNumberOfSamplesPerWrite() to 256
on Android TV. Note in case 2, the number of samples per write is in the
range of [1, min(SbPlayerGetMaximumNumberOfSamplesPerWrite(),
AudioBatchedSampleWrite)]. For other platforms except Android,
SbPlayerGetMaximumNumberOfSamplesPerWrite should return 1.

b/227837774

Test-On-Device: true
(cherry picked from commit ed62b35)
kaidokert pushed a commit that referenced this pull request Jun 14, 2024
… samples per write to SbPlayer from webapp (#3561)

Refer to the original PR: #3325

The PR (#3271) limits the maximum
number of samples per write to SbPlayer with audio write ahead. This PR
refactors the existing H5VCC flag from "EnableBatchedSampleWrite" to
"AudioBatchedSampleWrite" that allows configuring maximum number of
samples per write to SbPlayer from webapp.

1. When "AudioBatchedSampleWrite=1" or not set, Cobalt uses the default
number of samples per write (i.e., 1) to SbPlayer.
2. When "AudioBatchedSampleWrite=n", where n > 1, Cobalt uses the number
of samples per write estimated by GetEstimatedMaxBuffers() to SbPlayer.

This PR also changes SbPlayerGetMaximumNumberOfSamplesPerWrite() to 256
on Android TV. Note in case 2, the number of samples per write is in the
range of [1, min(SbPlayerGetMaximumNumberOfSamplesPerWrite(),
AudioBatchedSampleWrite)]. For other platforms except Android,
SbPlayerGetMaximumNumberOfSamplesPerWrite should return 1.

b/227837774

Test-On-Device: true

Co-authored-by: Bo-Rong Chen <[email protected]>
borongc added a commit that referenced this pull request Sep 30, 2024
Adding additional Media Telemetry to cover SbPlayerWriteSamples.

The rough range of each function:
1. When batched write is enabled (Media.MaxAudioSamplesPerWrite=256,
#3271):
    1. SbPlayerWriteSamples()/audio: 1us to 98us
    2. SbPlayerWriteSamples()/video: 1us to 3.4ms
2. When batched write is disabled (Media.MaxAudioSamplesPerWrite=1,
#3271):
    1. SbPlayerWriteSamples()/audio: 9us to 65.47ms
    2. SbPlayerWriteSamples()/video: 8us to 28.07ms

b/303328238
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cp-25.lts.1+ Cherry Pick to the 25.lts.1+ branch on_device
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants