Skip to content

Latest commit

 

History

History
94 lines (69 loc) · 4.88 KB

README.md

File metadata and controls

94 lines (69 loc) · 4.88 KB
Raw

Latent Consistency Models

Official Repository of the paper: Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference.

Project Page: https://latent-consistency-models.github.io

Try our 🤗 Hugging Face Demos: Hugging Face Spaces 🔥🔥🔥

Join our LCM discord channels for discussions. Coders are welcome to contribute.

News

  • (🔥New) 2023/10/24 We support SD-Webui and ComfyUI now!! Thanks for @0xbitches. See the link: SD-Webui and ComfyUI.
  • (🔥New) 2023/10/23 Running on Windows/Linux CPU is also supported! Thanks for @rupeshs See the link.
  • (🔥New) 2023/10/22 Google Colab is supported now. Thanks for @camenduru See the link: Colab
  • (🔥New) 2023/10/21 We support local gradio demo now. LCM can run locally!! Please refer to the "Local gradio Demos".
  • (🔥New) 2023/10/19 We provide a demo of LCM in 🤗 Hugging Face Space. Try it here.
  • (🔥New) 2023/10/19 We provide the LCM model (Dreamshaper_v7) in 🤗 Hugging Face. Download here.
  • (🔥New) 2023/10/19 LCM is integrated in 🧨 Diffusers library. Please refer to the "Usage".

🔥 Local gradio Demos:

To run the model locally, you can download the "local_gradio" folder:

  1. Install Pytorch (CUDA). MacOS system can download the "MPS" version of Pytorch. Please refer to: https://pytorch.org
  2. Install the main library:
pip install diffusers transformers accelerate gradio==3.48.0
  1. Launch the gradio: (For MacOS users, need to set the device="mps" in app.py)
python app.py

Demos & Models Released

Ours Hugging Face Demo and Model are released ! Latent Consistency Models are supported in 🧨 diffusers.

Hugging Face Demo: Hugging Face Spaces

Replicate Demo: Replicate

Tungsten Demo: Tungsten

LCM Model Download: LCM_Dreamshaper_v7

By distilling classifier-free guidance into the model's input, LCM can generate high-quality images in very short inference time. We compare the inference time at the setting of 768 x 768 resolution, CFG scale w=8, batchsize=4, using a A800 GPU.

Usage

You can try out Latency Consistency Models directly on: Hugging Face Spaces

To run the model yourself, you can leverage the 🧨 Diffusers library:

  1. Install the library:
pip install diffusers transformers accelerate
  1. Run the model:
from diffusers import DiffusionPipeline
import torch

pipe = DiffusionPipeline.from_pretrained("SimianLuo/LCM_Dreamshaper_v7", custom_pipeline="latent_consistency_txt2img", custom_revision="main")

# To save GPU memory, torch.float16 can be used, but it may compromise image quality.
pipe.to(torch_device="cuda", torch_dtype=torch.float32)

prompt = "Self-portrait oil painting, a beautiful cyborg with golden hair, 8k"

# Can be set to 1~50 steps. LCM support fast inference even <= 4 steps. Recommend: 1~8 steps.
num_inference_steps = 4 

images = pipe(prompt=prompt, num_inference_steps=num_inference_steps, guidance_scale=8.0, lcm_origin_steps=50, output_type="pil").images

BibTeX

@misc{luo2023latent,
      title={Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference}, 
      author={Simian Luo and Yiqin Tan and Longbo Huang and Jian Li and Hang Zhao},
      year={2023},
      eprint={2310.04378},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}