Added support to extract language from video

[plon-Susk7]

Solved issue #319

1. Made changes to detect_lang_of_audio and test_detect_lang_of_audio and their corresponding requirements file.
2. Used ffmpeg library to extract audio from video.
3. Renamed file name from detect_lang_of_audio to detect_lang_of_media. Similar renaming done for test file and requirements file.

[aatmanvaidya]

hi @plon-Susk7 - I am facing an error when I run the test from inside the docker container.

ResourceWarning: Enable tracemalloc to get the object allocation traceback
.Error extracting audio: ffmpeg error (see stderr output for detail)
E
======================================================================
ERROR: test_english_detection_video (core.operators.test_detect_lang_of_media.Test.test_english_detection_video)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/app/core/operators/test_detect_lang_of_media.py", line 28, in test_english_detection_video
    lang = detect_lang_of_media.run(audio_file_path,'video')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/app/core/operators/detect_lang_of_media.py", line 236, in run
    extract_audio_from_video(audio_file["path"])
  File "/usr/app/core/operators/detect_lang_of_media.py", line 136, in extract_audio_from_video
    .run(quiet=True, overwrite_output=True)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/app/venv/lib/python3.11/site-packages/ffmpeg/_run.py", line 325, in run
    raise Error('ffmpeg', out, err)
ffmpeg._run.Error: ffmpeg error (see stderr output for detail)

am I doing something wrong?

---

also there are other some small things I feel we should do

1. Can you profile this operator on video's of different lengths for RAM usage and time. Here is the wiki that might help you - https://github.com/tattle-made/feluda/wiki/Optimization. You can just time.time() to calcuate the duration and use the memary code from the wiki to profile for RAM. You can choose videos of different lengths 5min, 10min, 10min, 20min and 30min. Here are the results of profiling the same operator on audio files.
2. Majorly when we use the operator's, we first download the media item from a hosted URL or our AWS cloud service. So once the processing is done, can you make sure the file is safely processed and deleted.
   There is already code present in the operator to delete the file - code line. Can you also just make sure that the video file gets deleted. Just in case if you want, here is a JSON file with a lot of CDN links for audio and video files you can use for testing.
3. Just like other functions in the operator, can you add 2 line documentation comment writing the input and output of the run() function. This is just to make sure someone else using the operator know's what the run() function expects.

Let me know if I can be of help in any of the above!

[plon-Susk7]

hi @plon-Susk7 - I am facing an error when I run the test from inside the docker container.

ResourceWarning: Enable tracemalloc to get the object allocation traceback
.Error extracting audio: ffmpeg error (see stderr output for detail)
E
======================================================================
ERROR: test_english_detection_video (core.operators.test_detect_lang_of_media.Test.test_english_detection_video)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/app/core/operators/test_detect_lang_of_media.py", line 28, in test_english_detection_video
    lang = detect_lang_of_media.run(audio_file_path,'video')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/app/core/operators/detect_lang_of_media.py", line 236, in run
    extract_audio_from_video(audio_file["path"])
  File "/usr/app/core/operators/detect_lang_of_media.py", line 136, in extract_audio_from_video
    .run(quiet=True, overwrite_output=True)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/app/venv/lib/python3.11/site-packages/ffmpeg/_run.py", line 325, in run
    raise Error('ffmpeg', out, err)
ffmpeg._run.Error: ffmpeg error (see stderr output for detail)

am I doing something wrong?

also there are other some small things I feel we should do

1. Can you profile this operator on video's of different lengths for RAM usage and time. Here is the wiki that might help you - https://github.com/tattle-made/feluda/wiki/Optimization. You can just time.time() to calcuate the duration and use the memary code from the wiki to profile for RAM. You can choose videos of different lengths 5min, 10min, 10min, 20min and 30min. Here are the results of profiling the same operator on audio files.
2. Majorly when we use the operator's, we first download the media item from a hosted URL or our AWS cloud service. So once the processing is done, can you make sure the file is safely processed and deleted.
   There is already code present in the operator to delete the file - code line. Can you also just make sure that the video file gets deleted. Just in case if you want, here is a JSON file with a lot of CDN links for audio and video files you can use for testing.
3. Just like other functions in the operator, can you add 2 line documentation comment writing the input and output of the run() function. This is just to make sure someone else using the operator know's what the run() function expects.

Let me know if I can be of help in any of the above!

I'll make the necessary changes soon.

[aatmanvaidya]

hi, did you accidentally close the PR? or any specific reason to do so?

[plon-Susk7]

Sorry Aatman this was accidental. I'll open it again once changes are made.

[plon-Susk7]

2. Majorly when we use the operator's, we first download the media item from a hosted URL or our AWS cloud service. So once the processing is done, can you make sure the file is safely processed and deleted.
   There is already code present in the operator to delete the file - code line. Can you also just make sure that the video file gets deleted. Just in case if you want, here is a JSON file with a lot of CDN links for audio and video files you can use for testing.

We are converting the mp4 file to speech file. So isn't removing videos same as removing speech here? Are we supposed to physically remove the file from disk?

[aatmanvaidya]

2. Majorly when we use the operator's, we first download the media item from a hosted URL or our AWS cloud service. So once the processing is done, can you make sure the file is safely processed and deleted.
   There is already code present in the operator to delete the file - code line. Can you also just make sure that the video file gets deleted. Just in case if you want, here is a JSON file with a lot of CDN links for audio and video files you can use for testing.

We are converting the mp4 file to speech file. So isn't removing videos same as removing speech here? Are we supposed to physically remove the file from disk?

yes we have to physically remove the file from disk

[plon-Susk7]

Video Length	CPU Time (s)	RAM Usage
5 mins	3.75	619.9 MiB
10 mins	6.14	628.8 MiB
15 mins	8.46	638.1 MiB
20 mins	10.99	647.1 MiB
30 mins	15.29	665.6 MiB

[plon-Susk7]

Video Length CPU Time (s) RAM Usage
5 mins 3.75 619.9 MiB
10 mins 6.14 628.8 MiB
15 mins 8.46 638.1 MiB
20 mins 10.99 647.1 MiB
30 mins 15.29 665.6 MiB

stats after profiling

[aatmanvaidya]

@plon-Susk7 thank you so much for your work and effort, things look good, merging the PR!