Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use WebCodec API AudioDecoder in spectrogram rendering #270

Open
hudson-newey opened this issue Jan 16, 2025 · 1 comment
Open

Use WebCodec API AudioDecoder in spectrogram rendering #270

hudson-newey opened this issue Jan 16, 2025 · 1 comment

Comments

@hudson-newey
Copy link
Member

hudson-newey commented Jan 16, 2025

At the moment we are decoding audio files using a worker, audio worklet, audio buffer processor, and a couple SharedArrayBuffer's.

This has multiple drawbacks

  1. We are decoding audio files in JavaScript (extremely slow)
  2. It means that our bundle is not one file, and it has assets/ directory for the audio worklets, and buffer builder processor
  3. Because we use SharedArrayBuffer, we need the COEP, CORP headers, and all assets need to be served with a CORS header. This means that it can sometimes be difficult to implement these web components in your website because you need to update your serving headers (sometimes not possible with third parties such as GitHub pages).

We originally used this architecture because Firefox and Safari didn't support the AudioDecoder web codec API needed to decode audio files using browsers C++/C/ASM/ffmpeg code.

However, Firefox recently implemented support for the WebCodec API (see: https://bugzilla.mozilla.org/show_bug.cgi?id=1746557#c20), and Safari is not too far behind.

We should migrate to the AudioDecoder API for faster performance, better bundling, and support for looser header security.

@hudson-newey
Copy link
Member Author

If this issue is completed, the following issues are no longer relevant:

Fixes: #228

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant