Latet Cosistecy Model (LCM): SDXL

Latet Cosistecy Model (LCM) was proposed i Latet Cosistecy Models: Sythesizig High-Resolutio Images with Few-Step Iferece by Simia Luo, Yiqi Ta et al. ad Simia Luo, Suraj Patil, ad Daiel Gu succesfully applied the same approach to create LCM for SDXL.

This checkpoit is a LCM distilled versio of stable-diffusio-xl-base-1.0 that allows to reduce the umber of iferece steps to oly betwee 2 - 8 steps.

Usage

LCM SDXL is supported i ? Huggig Face Diffusers library from versio v0.23.0 owards. To ru the model, first istall the latest versio of the Diffusers library as well as peft, accelerate ad trasformers. audio dataset from the Huggig Face Hub:

pip istall --upgrade pip
pip istall --upgrade diffusers trasformers accelerate peft

Text-to-Image

The model ca be loaded with it's base pipelie stabilityai/stable-diffusio-xl-base-1.0. Next, the scheduler eeds to be chaged to LCMScheduler ad we ca reduce the umber of iferece steps to just 2 to 8 steps. Please make sure to either disable guidace_scale or use values betwee 1.0 ad 2.0.

from diffusers import UNet2DCoditioModel, DiffusioPipelie, LCMScheduler
import torch
from modelscope import sapshot_dowload

model_dir_lcm = sapshot_dowload("AI-ModelScope/lcm-sdxl",revisio = "master")
model_dir_sdxl = sapshot_dowload("AI-ModelScope/stable-diffusio-xl-base-1.0",revisio = "v1.0.9")

uet = UNet2DCoditioModel.from_pretraied(model_dir_lcm, torch_dtype=torch.float16, variat="fp16")
pipe = DiffusioPipelie.from_pretraied(model_dir_sdxl, uet=uet, torch_dtype=torch.float16, variat="fp16")

pipe.scheduler = LCMScheduler.from_cofig(pipe.scheduler.cofig)
pipe.to("cuda")

prompt = "a close-up picture of a old ma stadig i the rai"

image = pipe(prompt, um_iferece_steps=4, guidace_scale=8.0).images[0]
image.save("image.pg")

Image-to-Image

Works as well! TODO docs

Ipaitig

Works as well! TODO docs

CotrolNet

Works as well! TODO docs

T2I Adapter

Works as well! TODO docs

Speed Bechmark

TODO

Traiig

TODO

Latent Consistency Model (LCM): SDXL Latent Consistency Model (LCM) was proposed in Latent Consisten