Do you use Wuffs? Tell us!

[nigeltao]

This is not an issue so much as a lightweight way of gathering information on who is using Wuffs. This is mostly to satisfy our curiosity, but might also help us decide how to evolve the project.

So, if you use Wuffs for something, please chime in here and tell us more!

[nigeltao]

Thanks to metallb/metallb#5 for the inspiration.

[mvdan]

Shouldn't come as a surprise, but I use it :) currently playing with it for https://github.com/mvdan/zstd. A bit rough and unstable, but now that I can have method calls, I should be able to implement more and more bits of zstd.

By the way - I know you created a mailing list, and I subscribed - but I created a #wuffs channel on gophers.slack.com too. Not much happening there yet, but just in case it wasn't on your radar.

[nigeltao]

The Wuffs GIF decoder is being trialled by Skia, the 2-D graphics library used by both the Android operating system and the Chromium web browser.

[ArniDagur]

When using Wuffs, do you guys have any syntax highlighting? If so, which editor/extension do you use? I want to try Wuffs, but programming without syntax highlighting seems intimidating. Thanks!

[mvdan]

I personally program without any editor support, which is acceptable given there is wuffsfmt.

[nigeltao]

I use vim, but my use of syntax highlighting probably isn't what you're expecting. I wrote more about this at https://nigeltao.github.io/blog/2018/colorful-text.html

$ cat ~/.vim/syntax/wuffs.vim 
" wuffs.vim: Nigel Tao's vim syntax file for Wuffs.

if exists("b:current_syntax")
  finish
endif
let b:current_syntax = "wuffs"

syn case match

syn match       wuffsSpaceError        display excludenl "\s\+$"
syn match       wuffsAssertion         /^	*assert .*/
syn match       wuffsAssertion         /^	*inv .*/
syn match       wuffsAssertion         /^	*pre .*/
syn match       wuffsAssertion         /^	*post .*/
syn match       wuffsComment           /\/\/.*/
syn match       wuffsDeclaration       /^packageid .*/
syn match       wuffsDeclaration       /^pri .*/
syn match       wuffsDeclaration       /^pub .*/
syn match       wuffsDeclaration       /^use .*/

hi wuffsAssertion   ctermfg=darkcyan
hi wuffsComment     ctermfg=gray
hi wuffsDeclaration ctermfg=yellow

hi def link     wuffsSpaceError        Error

[bitbank2]

I don't use Wuffs, but I've written a drop-in replacement for GIFLIB that I believe is even faster than Wuffs at LZW decoding. I use the output image as the dictionary and for images that compress well, it can decode > 20x faster than the original GIFLIB. I have only implemented the new API for most features, but if you need speed, it may be worth a look --> https://github.com/bitbank2/giflib-turbo

[davidmankin]

I wanted to try wuffs to build a simple CLI tool to decode the very simple TFRecord format but I ran into a few challenges trying to get started. I'll list them here and then file independent issues.

• The documentation about how to write your own .wuffs code is hard to grok. It has an incredibly trivial hello example, but no more documentation showing how to write a decoder.
• The coroutines document didn't contain a single coroutine example
• The i/o document was low level about iobuffer but didn't offer any examples about how to actually use io_reader or io_writer.
• I looked for both an IntelliJ or VSCode language plugin to help me learn to write wuffs, but there's nothing. I'd want, at least, syntax highlighting to help me know if I've made a typo. Better would be code completion so when I type arg.src. it would tell me what methods I could call on the io_reader.
• I want to use wuffs in a Bazel based project, so I'd like to see a starlark library I could include that would paper over any toolchain stuff I don't want to learn to just get started.
• I'd love to be able to just write wuffs and compile it into a program to decode my file, rather than have to write a bunch of C code. Maybe what I need is just to start with the zcat c code and replace the gzip code with my decoder?
• The example/ directory doesn't contain any examples of wuffs code?! I don't want to have to read C code.
• No CRC32C support, as far as I can tell, even though the README is careful to point out that there are two common formats of CRC32.

It would be great if there was a tutorial example of a whole project step by step, much like what I'm trying to do. E.g. "create decoder.wuffs. paste in this tiny bit. create decoder.c. paste in this tiny bit. create Makefile. paste in this tiny bit. run make. try your program. see you've built cat. modify wuffs code adding this bit to verify x. run make. see your cat now requires uppercase starting letter. replace the last tiny bit with this tiny bit. see your cat now requires an md5 checksum to match the content before it will print it out" or some other not-quite trivial example that uses some features of the language progressively.

As it is the documentation seems mostly aimed to people who already know what they're doing here.

[nigeltao]

@davidmankin thanks for the list. They're valid issues, but fixing all of them is also a sizable amount of work. As the trivial /hello-wuffs-c example that you've already seen states, we've "prioritized users of Wuffs the Library over users of Wuffs the Language" for now. To set your expectations, that prioritization isn't likely to change in the short term without additional people to work on it.

[davidmankin]

Thanks for the reply. I hoped that maybe since the caveat was written in 2019 that we'd be into the supporting users of the language. I'm not sure I understand what I'd use the library for. Maybe if I get far enough to understand the wuffs the language I could at least help contribute docs. As it is, I have no idea if what i'm doing is right or will work…

[pjanx]

My image viewer has reached general usefulness, and I'm inching towards something that could be called a stable version.

I need PNG metadata readout: sRGB/iCCP/gAMA (for display, definitely want to take care of applying these independently) and complete support of text chunks (for thumbnails). I've recently seen commits tackling the former, but until text is finished, I must use another library.

I probably won't use write support for that format, but it would be next on my list.

My main issue with the codebase is that although the documentation is well above average, it's also quite lacking in places, in particular around the C API, and it's very hard to locate the pieces that are present.

[nigeltao]

I need PNG metadata readout: sRGB/iCCP/gAMA... and complete support of text chunks.

Let's move the sRGB/iCCP/gAMA discussion to #39 and the text chunk discussion to the freshly created #55.

[nigeltao]

My image viewer has reached general usefulness,

@pjanx I don't have an account to file an issue on the project page, so I'll mention it here. Where fastiv-io.c has this line:

#define WUFFS_IMPLEMENTATION

You might want to also copy/paste this recent addtion to the Wuffs example programs:

// Defining the WUFFS_CONFIG__STATIC_FUNCTIONS macro is optional, but when
// combined with WUFFS_IMPLEMENTATION, it demonstrates making all of Wuffs'
// functions have static storage.
//
// This can help the compiler ignore or discard unused code, which can produce
// faster compiles and smaller binaries. Other motivations are discussed in the
// "ALLOW STATIC IMPLEMENTATION" section of
// https://raw.githubusercontent.com/nothings/stb/master/docs/stb_howto.txt
#define WUFFS_CONFIG__STATIC_FUNCTIONS

This can noticably impact build times and binary sizes.

---

Also, there's a comment later:

// We might want to employ a more generic way of magic identification,
// and with some luck, it could even be integrated into Wuffs.

which might be obsolete, given that a few lines later, you say:

switch (wuffs_base__magic_number_guess_fourcc(prefix))

[pjanx]

It's fine, Gitea is sorely lacking the cross-account ability of GitLab, and I recently got persistently annoying spammers over direct registration. Mail would also be fine for contact.

I have actually tried the static functions some time ago and decided against them, as it didn't improve much for me, partly due to already using so much of the library. To the contrary, it produced -Wunused-function warnings that I didn't want to disable. If memory serves me right.

Wuffs is currently limited in what it can identify. At some later point, I realised Wuffs' ‘registry’ is more or less arbitrary, and can be extended, so that comment could use some adjustments indeed. I actually might need to merge the common guesses of shared-mime-info (file-based), Wuffs (hardcoded), and gdk-pixbuf (plugin-based).

[pjanx]

It turns out I haven't had a good look at Gitea authentication settings, I should have GitHub OAuth sign-in now.

Anyway, I'm here to say that Abydos also has a Wuffs plugin for image loading: http://snisurset.net/code/abydos/

[dev0x13]

I've got acquainted with Wuffs the Library when I was looking for a faster way to decode PNG images for the commercial computer vision product I develop at work. Previously it was using OpenCV (hence libpng with some wrapping code), and replacing it with Wuffs improved the decoding performance tremendously. I also look forward to the upcoming JPEG decoding support in Wuffs which seems very promising because as of now it shows performance comparable to libjpeg-turbo without even using SIMD.
What I found surprising then is that Wuffs is barely known by computer vision community while they care a lot about image decoding performance typically being a bottleneck in a neural network training pipeline or even in a production integration. For example, the NVIDIA's DALI library (a collection of highly optimized building blocks for loading and processing image) which is used heavily by CV engineers, still uses OpenCV (hence libpng with the default zlib under the hood) for PNG decoding.
From my perspective, one of the reasons for this unfair lack of attention and adoption of Wuffs the Library is its steep learning curve due to a complex low level API (btw, I've discovered the much simpler Auxiliary C++ API long time after I spent quite some time integrating with the C API and implementing robust metadata parsing procedure, so the docs might need some adjustment in that regard) and limited availability, programming language wise. So I decided to write a Python interface for Wuffs the Library starting with image decoding. The package is already available here, and I hope that it'll be useful for some folks and will help people to discover Wuffs - the hidden gem of image decoding.

[nigeltao]

@pjanx

My main issue with the codebase is that although the documentation is well above average, it's also quite lacking in places, in particular around the C API, and it's very hard to locate the pieces that are present.

and

@dev0x13

I've discovered the much simpler Auxiliary C++ API long time after I spent quite some time integrating with the C API and implementing robust metadata parsing procedure, so the docs might need some adjustment in that regard

Thanks for the feedback. The C API could still do with some more docs per se, but I've recently pushed 129d800 that should contain a few more signposts for anyone in the future in a similar situation.

[dev0x13]

@nigeltao
Thank you for adjusting the docs and for adding pywuffs mention!

[Ono-Sendai]

Just enabled Wuffs for reading PNGs in glare-core: glaretechnologies/glare-core@bd7fc5e
Which is used in Substrata, a Metaverse project: https://substrata.info/

[Ono-Sendai]

Some benchmarks:
doDecodeFromBufferLibPNG took 14.877 ms
doDecodeFromBufferWithWuffs took 8.5301 ms

Not checking CRCs with either LibPNG or Wuffs.
Windows 10, Visual Studio 2022, x64, no enhanced instruction set (e.g. no AVX). AMD Ryzen 9 5950X.
Optimisations enabled (of course).

[sgraham]

When using Wuffs, do you guys have any syntax highlighting? If so, which editor/extension do you use? I want to try Wuffs, but programming without syntax highlighting seems intimidating. Thanks!

Sorry, not a useful note about users (yet!) but this issue is the first hit for [vim syntax wuffs] so I wanted to drop a note about a quick hack vim syntax repo here https://github.com/sgraham/vim-wuffs .

It's "Christmas-tree style" though per my eye-preferences, and despite Nigel's protestations above. Patches welcome!

[yota-code]

Hello ! I would be interested in using wuffs to decode jpeg xl !
However I remember reading, I don't remember where, that wuffs can only work on size limited image (16k pixels or so), is that true ?

And before diving in, I would like to know if wuffs could handle progressiveness ?

[nigeltao]

Wuffs' standard library has a JPEG decoder, which supports progressive JPEGs. Its GIF and PNG decoders also support interlacing and progressive output (producing an intermediate image when not all of the source bytes have been received).

Wuffs has no JPEG-XL decoder yet.

Wuffs does not have a 16k pixel limit. You may be thinking of WebP instead (which does have that limit).

[yota-code]

my bad, I found where I read about this limit, and it was indeed for webp https://github.com/nigeltao/qoir

As a matter of fact, I was thinking of using the wuffs language, not the library...
But I tried to gather documentation on the format and the topic is complex ! A jpeg XL decoder might be available in the library before I could successfully implement one myself 😄

[solidpixel]

We integrated Wuffs into the Arm ASTC texture compressor about 18 months ago, primarily for faster PNG loading, but the improved robustness for bad images is a huge benefit. Thanks!

https://github.com/ARM-software/astc-encoder