Use wasm-vips instead of sharp

[JonasKruckenberg]

Maybe we can switch the underlying library from sharp to wasm-vips witch would increase memory safety, expose more functionality and enable live in-browser transformations.

This issue will track further investigations.

[yamiteru]

I've been thinking about using libvips for my serverless image compression server. I'm a huge fan of Rust so I was planning on implementing in in Rust. I'm wondering if wasm version is significantly faster than the js sharp one. I think the package is written in C/C++ and uses js just as a frontend api. So wouldn't this be basically doing the same? Only in wasm.

[JonasKruckenberg]

Well kinda yeah. Wasm-vips exposes a more low level api, that's basically the same as the libvips C api so it should in theory enable more flexibility, but first and foremost this would make it work without a native node add-on (like sharp uses) those can be tricky to install on some machines. But yeah in theory the Wasm version should not be faster than sharp no

I have played around with Wasm-vips a bit and it seems getting typescript to like it will not be easy 😢

[yamiteru]

What's wrong with integrating it with TS?

[JonasKruckenberg]

The typescript declarations generated by emscripten are made for in browser usage it seems so typescript will complain when you try to import it via es-modules

[yamiteru]

I don't know if it solves anything, but I think this is really cool https://wasmer.io and it might help with the issue

[JonasKruckenberg]

Thanks for the pointer, this issue is not super important anyway so I'll look around a bit more as well 👍

[polarathene]

would increase memory safety, expose more functionality and enable live in-browser transformations.

What sort of memory safety issues is it addressing? Is there any issues at present that come to mind with libvips regarding this that the wasm build resolves/avoids?

What additional functionality does it expose or enable this package to expose?

Live in-browser transformations may be a nice perk, but for many other use cases I don't think that is relevant when you consider the tradeoffs of the switch? Perhaps it's better to support as an alternative rather than a replacement if you see value in using wasm-vips?

---

I'm wondering if wasm version is significantly faster than the js sharp one. I think the package is written in C/C++ and uses js just as a frontend api. So wouldn't this be basically doing the same? Only in wasm.

No, wasm is not doing the same. The sharp lib bridges the native libvips package, while wasm-vips is a wasm build/port. You can likewise use JS to interact with the wasm build, but there isn't any speed advantage, you should expect less.

It is useful to perform transforms on image data via client-side instead, saving any latency and bandwidth of a server-side service, but the kicker is the wasm build is slower, plus has a payload weight over 4MB uncompressed, but 1.4MB via gzip 9 compression. With a CDN and caching on client-side that is perhaps not so bad depending on context of deployment.

---

this would make it work without a native node add-on (like sharp uses) those can be tricky to install on some machines.

This is a fair point and can make a good case as a fallback. It has been problematic to install for example in Alpine linux in the past, but that was some time ago and I think is a non-issue now. IIRC a binary build is distributed for a detected OS by sharp and if incompatible it will try to compile a build locally for that system.

For some users, that can be frustrating to deal with and get working (although it's usually well documented AFAIK), a better solution for that is probably using something like Docker with a little API packaged to perform operations like SaaS services offer.

wasm-vips may be an alternative solution, but for some users may not be accessible in environments that lack proper support (eg enterprise with outdated browsers or niche browsers clients that are popular in certain regions of the world that lack wasm support).

---

The wasm build is still a while off from a stable release, which you should probably wait on anyway?

In Feb this year, Next.JS reached out to sponsor development on the project for their own image optimization feature where they'd like to use the wasm build for similar reasons cited here.

For serverless usage, there's apparently some concerns (at least with Cloudflare limits).

The original announcement blogpost details some of the drawbacks. Modern browsers have been improving on WASM support since with SIMD for example, although I'm not sure if that has the same SIMD compatibility to resolve related issues about how dependencies were handling SIMD.

Since that blogpost, the linked benchmarks have been updated. WebP is 2x slower, PNG 3x slower, and JPEG over 6x slower. Non-issue for interactive in browser usage, but that can have notable impact on build times for what is likely the majority of other cases.

[JonasKruckenberg]

What sort of memory safety issues is it addressing? Is there any issues at present that come to mind with libvips regarding this that the wasm build resolves/avoids?

Kinda right yeah,libvips is a well made and well maintained project so it may not be as important, but given that the core package is free to be used (and abused?) in a bunch of different use cases, wasm would provide and extra level of security and stability.
But I agree it's a bit niche.

It is useful to perform transforms on image data via client-side instead, saving any latency and bandwidth of a server-side service, but the kicker is the wasm build is slower, plus has a payload weight over 4MB uncompressed, but 1.4MB via gzip 9 compression.

Yeah no question, my thought wasn't really that people would deploy that in their own websites to save latency, but rather to get a cool demo going where one could try out different directives and see the results instantly.

It has been problematic to install for example in Alpine linux in the past, but that was some time ago

I still vividly remember the time when I tried getting native addons to work on windows was a whole days work for me, but granted that was quite a while ago and things have gotten a lot better since.
So you might be right

The wasm build is still a while off from a stable release, which you should probably wait on anyway?

The original announcement blogpost details some of the drawbacks. Modern browsers have been improving on WASM support since with SIMD for example, although I'm not sure if that has the same SIMD compatibility to resolve related issues about how dependencies were handling SIMD.

Yeah I agree, wasm-vips is not where we'd want it as a dependency yet, but since browser support is also still lacking a bit the whole wasm idea will be on the backburner for a couple months minimum right?

Since that blogpost, the linked benchmarks have been updated. WebP is 2x slower, PNG 3x slower, and JPEG over 6x slower.

I wouldn't have expected anything else but nice to see data on!

[polarathene]

All good, just please don't drop native support via sharp for the wasm variant alone. It would absolutely be awesome for a client-side demo and as a fallback, but I don't see that as worth the tradeoff for bulk processing speed.

Windows shouldn't be as bad these days now that WSL is a thing, or leveraging docker containers.

[JonasKruckenberg]

No worries this won't be happening any time soon and when it does I'll make sure it's not worse than before 😉