CHANGELOG.md

1.7.0 (2023-09-04)

Bug Fixes

• addons: async TI download status, LoRA improvements (de8cfdc)
• circleci: pytest --ignore=Real-ESRGAN (d7038b5)
• circleci: remove conda from pytest call (2f29af2)
• diffusers: bump to aae2726 (jul30) post v0.19.2 + fixes (6c0a10a)
• Dockerfile: -yqq for apt-get, apt-utils, extra deps (bf470da)
• Dockerfile: TZ tzdata fix (9c5d911)
• misc: fix failing tests, pipeline init in rare circumstances (9338648)
• prime/update: commit these useful utility scripts (7b167c0)
• upsample: return $meta for kiri (b9dd6b7)
• x_attn_kwargs: only pass to pipeline if set (3f1f980)

Features

• checkpoints: use correct pipeline for "inpaint" in path (16dd383)
• loras: ability to specify #?scale=0.1 -> cross_attn_kwargs (747fc0d)
• pytorch2: bump deps, drop conda/xformers (a3d8078)
• sdxl,compel: Support. AutoPipeline default, safety_check fix (993be12)
• sdxl: fix sd_xl, loras; ability to init load specific pipeline (7e3af77)
• textualInversion: very early support (2babd53)
• upsample: initial RealESRGAN support for runtime downloads (8929508)

1.6.0 (2023-07-12)

Bug Fixes

• BaseStorage: mv misplaced .query from BaseArchive to BaseStorage (0c7a757)
• conversion: recognize "safetensor" anywhere in filename (1ceab7d)
• deps: bump diffusers to b9feed8, lock bitsandbytes==0.39.1 (be1c322)
• deps: diffusers to 0.17.0 + latest commits, other packages (a6e9db0)
• pipelines: pass revision/precision for community pipelines too (20311cf)
• safety_checker: drop DummySafetyChecker and just use None (e4fbf22)

Features

• checkpoints: support #fname query in HTTPStorage (0cb839d)
• dreambooth: update / merge in all upstream changes to date (a40129a)
• loras: use load_lora_weights (works with A1111 files too) (7a64846)
• storage: allow #a=1&b=2 params; HTTP can use #fname=XXX (4fe13ef)

1.5.0 (2023-05-24)

Bug Fixes

• app: async fixes for download, train_dreambooth (0dcbd16)
• app: diffusers callback cannot be async; use asyncio.run() (7854649)
• app: up sanic RESPONSE_TIMEOUT from 1m to 1hr (8e2003a)
• attn_procs: apply workaround only for storage not hf repos (b98710f)
• attn_procs: load non-safetensors attn_procs ourself (072e7a3), closes /github.com/huggingface/diffusers/pull/2448#issuecomment-1453938119
• deps: pin websockets<11.0 for sanic (33ae2f4)
• inference: return $error NO_MODEL_ID vs later crash on None (46ea977)
• storage: actually, always set self.status (default None) (c309ca9)
• storage: don't set self.status to None (9b88b80)
• storage: extract with dir= must not mutate dir (download, logs) (b1f8f87)
• tests: pin urlllib3 to < 2, avoids break in docker package (ccf8231)

Features

• app: run pipeline via asyncio.to_thread (e87f7e7)
• attn_procs: from_safetensors override, save .savetensors fname (5fb6487)
• cors: add sanic-ext and set default cors-origin to "*" (eb2a385)
• diffusers: bump to 0.15.0 + 2 weeks with lpw fix (9965cb5) (77e9078)
• diffusers: bump to latest diffusers, 0.14 + patches (see note) (48a99a5)
• download: async, status; download.py: use download_and_extract (bb7434a)
• HTTPStorage: store filename from content-disposition (2066c44)
• loadModel: send loadModel status (db75740)
• status: initial status work (d1cd39e)
• storage: support misc tar compression; progress (a8c8337)
• stream_events: stream send()'s to client too (08daf4f)

1.4.0 (2023-02-28)

Bug Fixes

• checkpoints: new conversion pipeline + convert w/o MODEL_URL (cd7f54d)
• diffusers: bump to latest commit (includes v0.13.1) (400e3d7)
• diffusers: bump to recent commit, includes misc LoRA fixes (7249c30)
• loadModel: pass revision arg too (cd5f995)

Features

• attn_procs: initial URL work (see notes) (6348836)
• callback: if modelInput.callback_steps, send() current step (2279de1)
• gpu: auto-detect GPU (CUDA/MPS/cpu), remove hard-coded (#20) (682a342)
• lora: callInput attn_procs to load LoRA's for inference (cb54291)
• send: set / override SEND_URL, SIGN_KEY via callInputs (74b4c53)

1.3.0 (2023-01-26)

Bug Fixes

• diffusers: bump to v0.12.0 (635d9d9)
• diffusers: update to latest commit (87632aa)
• dreambooth: bump diffusers, fixes fp16 mixed precision training (0f5d5ff)
• dreambooth: merge commits to v0.12.0 (NB: mixed-precision issue) (88f04f8)
• pipelines: fix clearPipelines() backport from cloud-cache (9577f93)
• requirements: bump transformers,accelerate,safetensors & others (aebcf65)
• re: use raw strings r"" for regexps (41310c2)
• tests/lambda: export HF_AUTH_TOKEN (9f11e7b)
• test: shallow copy to avoid mutating base test inputs (8c41167)

Features

• downloads: RUNTIME_DOWNLOAD from HF when no MODEL_URL given (73784a1)

1.2.2 (2023-01-09)

Bug Fixes

• dreambooth: runtime_dls path fix; integration tests (ce3827f)
• loadModel: revision = None if revision == "" else revision (1773631)

1.2.1 (2023-01-05)

Bug Fixes

• build-download: support regular HF download not just cloud cache (52edf6b)

1.2.0 (2023-01-04)

Features

• build: separate MODEL_REVISION, MODEL_PRECISION, HF_MODEL_ID (fa9dd16)

1.1.0 (2023-01-04)

Features

• downloads: allow HF_MODEL_ID call-arg (defauls to MODEL_ID) (adaa7f6)
• downloads: allow separate MODEL_REVISION and MODEL_PRECISION (6edc821)

1.0.2 (2023-01-01)

Bug Fixes

• diffusers: bump to 2022-12-30 commit 62608a9 (2f29165)

1.0.1 (2022-12-31)

Bug Fixes

• ci: different token, https auth (ecd0b5d)

1.0.0 (2022-12-31)

Bug Fixes

• app: clearPipelines() before loadModel() to free RAM (ec45acf)
• app: init: don't process MODEL_ID if not RUNTIME_DOWNLOADS (683677f)
• dockerfile: bump diffusers to eb1abee693104dd45376dbddd614320f2a0beb24 (1769330)
• downloads: model_url, model_id should be optional (9a19e7e)
• dreambooth: don't crash on cleanup when no class_data_dir created (36e64b1)
• dreambooth: enable mixed_precision training, default to fp16 (0430d23)
• gitScheduler: fix deprecation warning s/from_config/from_pretrained/ (92b2b43)
• pipelines: community pipelines, set torch_dtype too (0cc1b63)
• pipelines: fix clearPipelines(), load model w/ correct precision (3085412)
• requirements: bumps transformers from 4.22.2 to 4.25.1 (b13b58c)
• s3: incorrect value for tqdm causing crash (9527ece)
• send: container_id detection, use /containers/ to grep (5c0606a)
• tests: default to DPMSolverMultistepScheduler and 20 steps (a9c7bb0)

Features

• ability for custom config.yaml in CHECKPOINT_CONFIG_URL (d2b507c)
• add PyPatchMatch for outpainting support (3675bd3)
• app: don't track downloads in mem, check on disk (51729e2)
• app: runtime downloads with MODEL_URL (7abc4ac)
• app: runtime downloads, re-use loaded model if requested again (b84e822)
• callInputs: MODEL_ID, PIPELINE, SCHEDULER now optional (ef420a1)
• cloud_cache: normalize model_id and include precision (ad1b2ef)
• diffusers: bump to v0.10.12 and one commit after (6b68afd) (ec9117b)
• diffusers: bump to v0.9.0 (0504d97)
• docker: diffusers-api-base image, build, run.sh (1cbfc4f)
• dockerfile: FROM_IMAGE build-arg to pick base image (a0c37a6)
• Dockerfile: make SDv2 the default (+ some formatting cleanup) (c1e73ef)
• dockerfile: runtime downloads (b40ae86)
• Dockerfile: SAFETENSORS_FAST_GPU (62209be)
• download: default_path as normalized_model_id.tar.zst (5ad0d88)
• download: delete .zst file after uncompress (ab25280)
• download: record download timings (7457e50)
• downloads: runtime checkpoint conversion (2414cd9)
• dreambooth: save in safetensors format, tar up with -v (5c3e86a)
• errors: try...catch everything, return as JSON (901679c)
• getScheduler: make DPMSolverMultistepScheduler the default (085d06f)
• k-diffusion: add pip package for use in k-diffusion shedulers (3e901ad)
• models: store in ~/.cache/diffusers-api (volume support) (8032ec1)
• pipelines: allow calling of ALL PIPELINES (official+community) (1ccbaad)
• pipelines: initial community pipeline support (7af45cf)
• s3: s3client(), file_exists() methods (0308af9)
• s3: upload/download progress indicators (76dd303)
• send: prefer env var CONTAINER_ID if set to full docker uuid (eec5112)
• send: SEND_URL and SIGN_KEY now settable with build-vars (01cf354)
• test: allow TEST_URL to override https://localhost:8000/ (9b46387)
• tests: allow override BANANA_API_URL (aca6aca)

CHANGELOG

• NEXT MAIN

  • Callinputs MODEL_ID, PIPELINE and SCHEDULER are now optional. If not specified, the default will be used, and returned in a $meta key in the result.

  • Tests: 1) Don't specify above defaults where possible, 2) Log exact inputs sent to container, 3) Log the full result sent back, substituting base64 image strings with their info, 4) format stack traces on caught errors from container.

• NEXT MAIN (and already posted to forum)

  • Latest diffusers, SDv2.1. All the latest goodness, and upgraded some dependencies too. Models are:

    • stabilityai/stable-diffusion-2-1-base (512x512)
    • stabilityai/stable-diffusion-2-1 (768x768)

  • ALL THE PIPELINES. We no longer load a list of hard-coded pipelines in init(). Instead, we init and cache each on first use (for faster first calls on cold boots), and, all pipelines, both official diffusers and community pipelines, are available. Full details

  • Dreambooth: Enable mixed_precision training, default to fp16.

  • [Experimental] Runtime downloads (Dreambooth only for now, more on the way)

  • S3: Add upload/download progress indicators.

  • Stable Diffusion has standardized image instead of init_image for all pipelines. Using init_image now shows a deprecation warning and will be removed in future.

  • Changed sd-base to diffusers-api as the default tag / name used in the README examples and optional [./build][build script].

  • Much better error handling. We now try...except both the pipeline run and entire inference() call, which will save you a trip to banana's logs which don't always even show these errors and sometimes just leave you with an unexplained stuck instance. These kinds of errors are almost always a result of problematic callInputs and modelInputs used for the pipeline call, so finding them will be a lot easier now.

• 2022-11-29

  • Diffusers v0.9.0, Stable Diffusion v2.0. Models:

    • "stabilityai/stable-diffusion-2" - trained on 768x768
    • "stabilityai/stable-diffusion-2-base" - trained on 512x512
    • "stabilityai/stable-diffusion-2-inpainting" - untested
    • ""stabilityai/stable-diffusion-x4-upscaler" - untested

    https://github.com/huggingface/diffusers/releases

    NB: SDv2 does not include a safety_checker. The model itself is "safe" (it's much harder to create NSFW content). Trying to "turn off" the (non-existent) safety checker will throw an error, we'll handle this more gracefully in a future release. This also means you can safely ignore this warning on loading:

    You have disabled the safety checker for
    <class diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'>
    by passing safety_checker=None. Ensure that...
    

  • DPMSolverMultistepScheduler. Docker-diffusers-api is simply a wrapper around diffusers. We support all the included schedulers out of the box, as long as they can init themselves with default arguments. So, the above scheduler was already working, but we didn't mention it before. I'll just quote diffusers:

    DPMSolverMultistepScheduler is the firecracker diffusers implementation of DPM-Solver++, a state-of-the-art scheduler that was contributed by one of the authors of the paper. This scheduler is able to achieve great quality in as few as 20 steps. It's a drop-in replacement for the default Stable Diffusion scheduler, so you can use it to essentially half generation times.

  • Storage Class / S3 support. We now have a generic storage class, which allows for special URLs anywhere anywhere you can usually specify a URL, e.g. CHECKPOINT_URL, dest_url (after dreambooth training), and the new MODEL_URL (see below). URLs like "s3:///bucket/filename" will work how you expect, but definitely read docs/storage.md to understand the format better. Note in particular the triple forwardslash ("///") in the beginning to use the default S3 endpoint.

  • Dreambooth training, working but still in development. See this forum post for more info.

  • PRECISION build var, defaults to "fp16", set to "" to use the model defaults (generally fp32).

  • CHECKPOINT_URL conversion:

    • Crash / stop build if conversion fails (rather than unclear errors later on)
    • Force cpu loading even for models that would otherwise default to GPU. This fixes certain models that previously crashed in build stage (where GPU is not available).
    • --extract-ema on conversion since these are the more important weights for inference.
    • CHECKPOINT_CONFIG_URL now let's to specify a specific config file for conversion, to use instead of SD's default v1-inference.yaml.

  • MODEL_URL. If your model is already in diffusers format, but you don't host it on HuggingFace, you can now have it downloaded at build time. At this stage, it should be a .tar.zst file. This is an alternative to CHECKPOINT_URL which downloads a .ckpt file and converts to diffusers.

  • test.py:

    • New --banana arg to run the test on banana. Set environment variables BANANA_API_KEY and BANANA_MODEL_KEY first.
    • You can now add to and override a test's default json payload with:
      • --model-arg prompt="hello"
      • --call-arg MODEL_ID="my-model"
    • Support for extra timing data (e.g. dreambooth sends train and upload timings).
    • Quit after inference errors, don't keep looping.

  • Dev: better caching solution. No more unruly root-cache directory. See CONTRIBUTING.md for more info.

• 2022-11-08

  • Much faster init() times! For runwayml/stable-diffusion-v1-5:

    • Previously: 4.0s, now: 2.4s (40% speed gain)

  • Much faster inference() times! Particularly from the 2nd inference onwards. Here's a brief comparison of inference average times (for 512x512 x50 steps):

    • [Cold] Previously: 3.8s, now: 3.3s (13% speed gain)
    • [Warm] Previously: 3.2s, now: 2.1s (34% speed gain)

  • Improved test.py, see Testing

• 2022-11-05

  • Upgrade to Diffusers v0.7.0. There is a lot of fun stuff in this release, but notably for docker-diffusers-api TODAY (more fun stuff coming next week!), we have much faster init times (via fast_load ) and the greatly anticipated support for the Euler schedulers ( a1ea8c0 ).

  • We now use the full scheduler name for callInputs.SCHEDULER. "LMS", "DDIM", "PNDM" all still work fine for now but give a deprecation warning and will stop working in a future update. The full list of supported schedulers is: LMSDiscreteScheduler, DDIMScheduler, PNDMScheduler, EulerAncestralDiscreteScheduler, EulerDiscreteScheduler. These cover the most commonly used / requested schedulers, but we already have code in place to support every scheduler provided by diffusers, which will work in a later diffusers release when they have better defaults.

• 2022-10-24

  • Fixed img2img and inpainting pipelines. To my great shame, in my rush to get the new models out before the weekend, I inadvertently broke the above two models. Please accept my sincere apology for any confusion this may have caused and especially any of your wasted time in debugging this 🙇

  • Event logs now shown without SEND_URL. We optionally log useful info at the start and end of init() and inference(). Previously this was only logged if SEND_URL was set, to send to an external REST API for logging. But now, even if we don't send it anywhere, we'll still log this useful info. It now also logs the diffusers version too.

• 2022-10-21

  • Stable Diffusion 1.5 released!!!

    Accept the license at: "runwayml/stable-diffusion-v1-5"

    It's the new default model.

  • Official Stable Diffusion inpainting model

    Accept the license at: "runwayml/stable-diffusion-inpainting",

    A few big caveats!

    1. Different model - so back to a separate container for inpainting, also because:
    2. New pipeline that can't share model struct with other pipelines (see diffusers#920).
    3. Old pipeline is now called StableDiffusionInpaintPipelineLegacy (for sd-1.4)
    4. model_input takes image now, and not init_image like the legacy model.
    5. There is no strength parameter in the new model (see diffusers#920).

  • Upgrade to Diffusers v0.7.0.dev0

  • Flash attention now disabled by default. 1) Because it's built on an older version of diffusers, but also because 2) I didn't succeed in getting much improvement out of it. Maybe someone else will have better luck. I think you need big batch sizes to really see the benefit, which doesn't suit my use case. But please anyone who figures anything out, let us know.