Willow has been added in commit 7ba9d32

I use this with a few other libraries so my Dockerfile looks like:

#...

COPY ./wagtail /code/wagtail/
COPY ./libs /code/libs/

RUN cd /code/wagtail/ \
     && pip install -e .[testing,docs]

RUN cd /code/libs/ \
    && pip install -e wagtail-headless-preview wagtail-localize wagtail-markdown wagtailmedia Willow

# ...

I wonder if there's scope to install everything in /code/libs/ so then anything in there can be picked up automatically without having to edit the Dockerfile

Yes, I think we should definitely figure out a way to pip install everything in /code/libs.
Probably just a matter of doing some clever shell magic to list the folder names after pip install -e

I've create a PR which should theoretically allow you to put local versions of dependencies into the libs folder and then the Dockerfile will pick it up and local pip install it. Would be nice if you could try it out some time @zerolab .