Replies: 8 comments 19 replies
-
|
I recall @Keukhan may have answered this. The thumbnail publisher currently decodes every frame whether it is creating a thumbnail or not. Your approach is more efficient although I wonder if there's a cost to spawning many short-lived FFmpeg processes at once. You guys operate at scale so I guess it's OK? Anyways, +1 for a more efficient thumbnail publisher. My application relies on "live" thumbnails and I definitely notice the impact on CPU. |
Beta Was this translation helpful? Give feedback.
-
|
We make screenshots and thumbnails every ten seconds with ffmpeg for up to a hundred streams and don't run into any real problems no. If we were worried about that we could just reduce the frequency of updates |
Beta Was this translation helpful? Give feedback.
-
|
For reference, our ffmpeg call that is run every 10s
|
Beta Was this translation helpful? Give feedback.
-
|
Left hand side is OME with Thumbnail publisher disabled, blue line is Ruby Thumbnailer CPU usage, Right hand side is OME with Thumbnailer published enabeld, you can see it uses more CPU than the left side, it goes down as streams started to go offline |
Beta Was this translation helpful? Give feedback.
-
|
This is in our backlog. We will check to see if we can improve. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @naanlizard, I'm curious... how does that scale for you? Is there a way to keep the same instance going and pipe multiple streams in/out? We generally support 5-25 live streams at each customer site. I'd be weary of spinning up that many FFmpegs every 2 seconds for a 1-frame output. But if it works... :) |
Beta Was this translation helpful? Give feedback.
-
|
5-50 streams, typically. We update our screenshots every 10s, which is enough for our purposes, but I don't see why 2s would be unreasonable if you have the CPU for it I'm not aware of a way to have ffmpeg connect and spit out a frame every X seconds, but it may exist, I'll google a bit We have a docker container that every 10s runs an ffmpeg command IIRC. |
Beta Was this translation helpful? Give feedback.
-
|
Looks like we discussed this last year in this very thread 😂 This is a cut down version of the command you're using It works but the 16+ seconds latency on the output thumbnail makes me think FFmpeg is only reading traditional HLS. Any idea how to make it read LL-HLS? |
Beta Was this translation helpful? Give feedback.
-
|
Let me test but I'm pretty sure if you feed it an llhls link it'll be low ish latency |
Beta Was this translation helpful? Give feedback.
-
|
Appreciate it! I only tested the macOS homebrew version of FFmpeg 7.0 so far. Could be that this particular build is missing a piece too. |
Beta Was this translation helpful? Give feedback.
-
|
Hmm, at least my version of ffmpeg isn't doing any better than yours, even
after playing with a bunch of flags. I can document more later but gotta
run for now
…On Fri, Aug 9, 2024 at 3:57 PM bchah ***@***.***> wrote:
Appreciate it! I only tested the macOS homebrew version of FFmpeg 7.0 so
far. Could be that this particular build is missing a piece too.
—
Reply to this email directly, view it on GitHub
<#1447 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV4BWJMJ2MWEPH74ELF4QLZQUNMJAVCNFSM6AAAAABJOM63GOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRZGEYDQNI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you for your patience. An option to decode only keyframes has been added to significantly reduce CPU usage when only the thumbnail publisher is used. It has been added to the master branch, but it has not been released yet. Please test it and let us know if you have any suggestions for improvements. Please refer to the link below for usage instructions. Gitbook : https://airensoft.gitbook.io/ovenmediaengine/dev/transcoding#keyframe-decoding-only Below are the results from my testing Before After The feature to configure the number of decoding threads you requested has also been added Gitbook : https://airensoft.gitbook.io/ovenmediaengine/dev/transcoding#decoding-thread-configuration |
Beta Was this translation helpful? Give feedback.
-
|
Some initial thoughts: With KFI=2 for my input stream and the following config, I'm getting about 16s latency between live and the screenshot. Does that sound correct? Is there any way to improve that latency? It's not a big deal, but worth asking. Is there a performant way to serve two sizes of the same screenshot with OME? Or should I just use something like imgproxy? Is there a way to save these screenshots to the filesystem with OME directly? Or again would I need a third party tool? <Application>
<Name>live</Name>
<!-- Application type (live/vod) -->
<Type>live</Type>
<OutputProfiles>
<!--
Common setting for decoders. Decodes is optional.
-->
<Decodes>
<KeyframeOnlyIfNeed>true</KeyframeOnlyIfNeed>
</Decodes>
<!-- Enable this configuration if you want to hardware acceleration using GPU -->
<HardwareAcceleration>false</HardwareAcceleration>
<OutputProfile>
<Name>bypass_stream</Name>
<OutputStreamName>${OriginStreamName}</OutputStreamName>
<Encodes>
<Audio>
<Bypass>true</Bypass>
</Audio>
<Video>
<Bypass>true</Bypass>
</Video>
<Audio>
<Codec>opus</Codec>
<Bitrate>128000</Bitrate>
<Samplerate>48000</Samplerate>
<Channel>2</Channel>
</Audio>
<Image>
<Codec>jpeg</Codec>
<Framerate>0.1</Framerate>
</Image>
</Encodes>
</OutputProfile> |
Beta Was this translation helpful? Give feedback.
-
|
Wow, perfect timing! We'll try it asap
I was just fighting with our current thumbnail setup
…On Mon, Nov 11, 2024 at 23:29 Keukhan Kwon ***@***.***> wrote:
@hernanrz <https://github.com/hernanrz> @bchah <https://github.com/bchah>
@naanlizard <https://github.com/naanlizard>
Thank you for your patience. An option to decode only keyframes has been
added to significantly reduce CPU usage when only the thumbnail publisher
is used. It has been added to the master branch, but it has not been
released yet. Please test it and let us know if you have any suggestions
for improvements.
Please refer to the link below for usage instructions.
Gitbook :
https://airensoft.gitbook.io/ovenmediaengine/dev/transcoding#keyframe-decoding-only
Github : 4dff75a
<4dff75a>
Below are the results from my testing
*Before*
image.png (view on web)
<https://github.com/user-attachments/assets/d933633a-4cc8-47a5-b911-9f0b984d8f3b>
*After*
image.png (view on web)
<https://github.com/user-attachments/assets/749cc28d-1084-4e4b-bf52-9d4dead4846f>
—
Reply to this email directly, view it on GitHub
<#1447 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV4BWPB4USQMXE4C5J6EJT2AF7ZXAVCNFSM6AAAAABJOM63GOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRSGEZDEMQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
A question about the implementation:
When the option is enabled and ome only decodes key frames, how does the
screenshot output profile framerate setting interact?
If my stream has a keyframe interval of 1s and the target output is 1s, I
assume it works as expected and the image updates every 1s
If the KFI is 10s and the framerate is set to 1s, I think it would update
every 10s?
Similarly, KFI=1s, framerate 10s, image updates every 10s?
I think the flag name is confusing and will think of a better one.
KeyframeOnlyImages? Something like that
I will try a pull request with maybe a better name and also update the
gitbook
…On Mon, Nov 11, 2024 at 23:29 Keukhan Kwon ***@***.***> wrote:
@hernanrz <https://github.com/hernanrz> @bchah <https://github.com/bchah>
@naanlizard <https://github.com/naanlizard>
Thank you for your patience. An option to decode only keyframes has been
added to significantly reduce CPU usage when only the thumbnail publisher
is used. It has been added to the master branch, but it has not been
released yet. Please test it and let us know if you have any suggestions
for improvements.
Please refer to the link below for usage instructions.
Gitbook :
https://airensoft.gitbook.io/ovenmediaengine/dev/transcoding#keyframe-decoding-only
Github : 4dff75a
<4dff75a>
Below are the results from my testing
*Before*
image.png (view on web)
<https://github.com/user-attachments/assets/d933633a-4cc8-47a5-b911-9f0b984d8f3b>
*After*
image.png (view on web)
<https://github.com/user-attachments/assets/749cc28d-1084-4e4b-bf52-9d4dead4846f>
—
Reply to this email directly, view it on GitHub
<#1447 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABV4BWPB4USQMXE4C5J6EJT2AF7ZXAVCNFSM6AAAAABJOM63GOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRSGEZDEMQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***
com>
|
Beta Was this translation helpful? Give feedback.
-
|
In my stream, when the Keyframe Interval (KFI) is set to 10 and the image framerate is 1, the image updates only once every 10 seconds, which makes it hard to get accurate keyframes. When KFI is set to 1 and the framerate is 10, decoding happens 10 times, but the image still only updates every 10 seconds. It would be possible to improve this by skipping 9 of the decodes, but that hasn't been developed yet. If you have any good name ideas, please share them with us. There have been many comments internally that the term isn’t intuitive. :) |
Beta Was this translation helpful? Give feedback.
-
|
Thank you Keukhan! How about PreferKeyframeOnlyDecoding? Or, since it is already part of the |
Beta Was this translation helpful? Give feedback.
-
|
I think "Prefer" is too ambiguous - as I understand the feature, it ONLY decodes keyframes, so there's no preference involved (as opposed to say, preferring vanilla, but if vanilla isn't available, you'll eat chocolate)
|
Beta Was this translation helpful? Give feedback.
-
|
You all know my english isn't very good. so, i don't know feeling of words. If you all pick one name and let me know, I’ll change the option to that. Even if you pick something like < BchahAndNaanlizard >, I’ll use it. Currently, < Decodes > . < KeyframeOnlyIfNeed > is set, meaning it will only decode keyframes when it's a good situation to do so. If the video encoding is in OutputProfile, this option will automatically turn off. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks Keukhan. You're the best. All I really care is that you keep drinking water! 😀 |
Beta Was this translation helpful? Give feedback.
-
|
Lastly, and I'll probably open a new ticket, but are PNG and jpeg the only
options for image output now? Any hope of webp?
…On Tue, Nov 12, 2024 at 00:19 Garrett B ***@***.***> wrote:
A question about the implementation:
When the option is enabled and ome only decodes key frames, how does the
screenshot output profile framerate setting interact?
If my stream has a keyframe interval of 1s and the target output is 1s, I
assume it works as expected and the image updates every 1s
If the KFI is 10s and the framerate is set to 1s, I think it would update
every 10s?
Similarly, KFI=1s, framerate 10s, image updates every 10s?
I think the flag name is confusing and will think of a better one.
KeyframeOnlyImages? Something like that
I will try a pull request with maybe a better name and also update the
gitbook
On Mon, Nov 11, 2024 at 23:29 Keukhan Kwon ***@***.***>
wrote:
> @hernanrz <https://github.com/hernanrz> @bchah <https://github.com/bchah>
> @naanlizard <https://github.com/naanlizard>
>
> Thank you for your patience. An option to decode only keyframes has been
> added to significantly reduce CPU usage when only the thumbnail publisher
> is used. It has been added to the master branch, but it has not been
> released yet. Please test it and let us know if you have any suggestions
> for improvements.
>
> Please refer to the link below for usage instructions.
>
> Gitbook :
> https://airensoft.gitbook.io/ovenmediaengine/dev/transcoding#keyframe-decoding-only
> Github : 4dff75a
> <4dff75a>
>
> Below are the results from my testing
>
> *Before*
> image.png (view on web)
> <https://github.com/user-attachments/assets/d933633a-4cc8-47a5-b911-9f0b984d8f3b>
>
> *After*
> image.png (view on web)
> <https://github.com/user-attachments/assets/749cc28d-1084-4e4b-bf52-9d4dead4846f>
>
> —
> Reply to this email directly, view it on GitHub
> <#1447 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABV4BWPB4USQMXE4C5J6EJT2AF7ZXAVCNFSM6AAAAABJOM63GOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCMRSGEZDEMQ>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***
> .com>
>
|
Beta Was this translation helpful? Give feedback.
-
|
Please create a new ticket for the WEBP-related request. I will include it in the next plan. |
Beta Was this translation helpful? Give feedback.
-
On our website, we have noticed that enabling the thumbnail publisher (see config) increases OME's cpu usage a lot, when compared to a script that periodically runs ffmpeg to generate thumbnails by pulling the hls stream from OME. Any reasons this might happen and how can we reduce it?
Server.xml https://gist.github.com/hernanrz/0defd82474647685bc02bfe7e50c513a
thumbnail script for comparison https://gist.github.com/hernanrz/f9db82df1796b95209b425d51cab372b
Beta Was this translation helpful? Give feedback.
All reactions