switch to ffmpeg-sidecar

[louis030195]

https://github.com/CapSoftware/ffmpeg-sidecar

https://github.com/CapSoftware/Cap/blob/63719073805f1fd040343ae27366272a29918078/apps/desktop/src-tauri/src/main.rs#L75

switch to use this lib to install ffmpeg in screenpipe lib

Update: little concern with cap fork is that it's using their s3 ffmpeg build so if they close it, product is down, but can work around this later i guess

[louis030195]

/bounty 100

[algora-pbc]

💎 $100 bounty • Screenpi.pe

Steps to solve:

1. Start working: Comment /attempt #194 with your implementation plan
2. Submit work: Create a pull request including /claim #194 in the PR body to claim the bounty
3. Receive payment: 100% of the bounty is received 2-5 days post-reward. Make sure you are eligible for payouts

Thank you for contributing to mediar-ai/screenpipe!

Add a bounty • Share on socials

Attempt	Started (GMT+0)	Solution
🟢 @Mubashirshariq	Aug 27, 2024, 3:06:14 PM	WIP
🟢 @Neptune650	Oct 25, 2024, 1:19:19 AM	#580

[Mubashirshariq]

/attempt #194

Algora profile	Completed bounties	Tech	Active attempts	Options
@Mubashirshariq	2 bounties from 2 projects	JavaScript, TypeScript, HTML		Cancel attempt

[Neptune650]

/attempt #194

Algora profile	Completed bounties	Tech	Active attempts	Options
@Neptune650	3 mediar-ai bounties + 3 bounties from 2 projects	C++, C, Rust & more		Cancel attempt

[algora-pbc]

💡 @Neptune650 submitted a pull request that claims the bounty. You can visit your bounty board to reward.

[louis030195]

hey adding more context here

it's complex task with things involved:

• need to know if it's unlocking h265 encoding and if this encoding is worthwhile (ask AI) if it reduces storage usage, how much, and how much it affects resource usage and quality
• need to make sure it does not break anything across all OS, including video rendering in the app, encoding, etc.
• need to make sure users in china can still use screenpipe (e.g. not blocked, they don't need a VPN ideally)
• need to cleanup old ffmpeg
• need to make sure the download links of ffmpeg won't be broken like suddenly maintainer decide to kill the thing
• all this both for audio and screen encoding

[Neptune650]

hey adding more context here

it's complex task with things involved:

• need to know if it's unlocking h265 encoding and if this encoding is worthwhile (ask AI) if it reduces storage usage, how much, and how much it affects resource usage and quality
• need to make sure it does not break anything across all OS, including video rendering in the app, encoding, etc.
• need to make sure users in china can still use screenpipe (e.g. not blocked, they don't need a VPN ideally)
• need to cleanup old ffmpeg
• need to make sure the download links of ffmpeg won't be broken like suddenly maintainer decide to kill the thing
• all this both for audio and screen encoding

@louis030195 Hey there, this shouldn't cause any issues at all because what this does is install ffmpeg if it's not found, it's exactly the same as if the user already installed it or was shipped.

[Neptune650]

Also, after installing it, we set the ffmpeg path it's installed to, we could use the native bindings, but again this shouldn't cause any issues because it's really just vanilla ffmpeg.

[louis030195]

no, there are many different versions of ffmpeg that supports some encodings or don't (from my own experience) which results in crashes if not supported

that' why we have special if for windows vs macos encoding which don't support the same things

[Neptune650]

hey adding more context here

it's complex task with things involved:

* need to know if it's unlocking h265 encoding and if this encoding is worthwhile (ask AI) if it reduces storage usage, how much, and how much it affects resource usage and quality

All of the ffmpeg builds provided support h265 encoding. (look here) But acceleration of course depends on the OS and hardware.

* need to make sure it does not break anything across all OS, including video rendering in the app, encoding, etc.

Within my tests in CI and in VMs, it all worked fine.

* need to make sure users in china can still use screenpipe (e.g. not blocked, they don't need a VPN ideally)

https://github.com/nathanbabcock/ffmpeg-sidecar/blob/3fd9cafff733ebb2ee55a86010acd226cefdf2aa/src/download.rs#L39-L45

I checked all the URLs myself, none of them are blocked in China. I tested using this website: https://www.dotcom-tools.com/china-firewall-test

* need to cleanup old ffmpeg

I think it would be good to leave it there as a fallback.

* need to make sure the download links of ffmpeg won't be broken like suddenly maintainer decide to kill the thing

As I linked above, it's all different links and not by the maintainer but other people, so shouldn't break suddenly.

@louis030195 Tell me what you think, and merge #580 if appropriate.

[louis030195]

hmm @Neptune650

any chance you can do some measurement of difference between using h265 and current? i need more conviction it's worthwhile to switch

like maybe simple bash script that runs screenpipe on --data-dir /tmp/test123 and measure how much storage it consume after 10-60 mins and how much resource (cpu, gpu, ram) and run this with and without h265 (or some more efficient test)

[twilwa]

once merged, does this swap audio capture over to ffmpeg? if so, not sure if i closed/my ticket got closed about audio on linux always recording output streams, but it may resolve that issue. If so, as long as the performance is identical or slightly worse, it'd be worth it (for me/any other linux users, presumably) imo

[algora-pbc]

🎉🎈 @Neptune650 has been awarded $100! 🎈🎊

• Share on socials
• Give feedback