SDXL-Lightig

SDXL-Lightig is a lightig-fast text-to-image geeratio model. It ca geerate high-quality 1024px images i a few steps. For more iformatio, please refer to our research paper: SDXL-Lightig: Progressive Adversarial Diffusio Distillatio. We ope-source the model as part of the research.

Our models are distilled from stabilityai/stable-diffusio-xl-base-1.0. This repository cotais checkpoits for 1-step, 2-step, 4-step, ad 8-step distilled models. The geeratio quality of our 2-step, 4-step, ad 8-step model is amazig. Our 1-step model is more experimetal.

We provide both full UNet ad LoRA checkpoits. The full UNet models have the best quality while the LoRA models ca be applied to other base models.

Demos

• Geerate with all cofiguratios, best quality: Demo

Checkpoits

• sdxl_lightig_Nstep.safetesors: All-i-oe checkpoit, for ComfyUI.
• sdxl_lightig_Nstep_uet.safetesors: UNet checkpoit oly, for Diffusers.
• sdxl_lightig_Nstep_lora.safetesors: LoRA checkpoit, for Diffusers ad ComfyUI.

Diffusers Usage

Please always use the correct checkpoit for the correspodig iferece steps.

2-Step, 4-Step, 8-Step UNet

import torch
from diffusers import StableDiffusioXLPipelie, UNet2DCoditioModel, EulerDiscreteScheduler
from huggigface_hub import hf_hub_dowload
from safetesors.torch import load_file

base = "stabilityai/stable-diffusio-xl-base-1.0"
repo = "ByteDace/SDXL-Lightig"
ckpt = "sdxl_lightig_4step_uet.safetesors" # Use the correct ckpt for your step settig!

# Load model.
uet = UNet2DCoditioModel.from_cofig(base, subfolder="uet").to("cuda", torch.float16)
uet.load_state_dict(load_file(hf_hub_dowload(repo, ckpt), device="cuda"))
pipe = StableDiffusioXLPipelie.from_pretraied(base, uet=uet, torch_dtype=torch.float16, variat="fp16").to("cuda")

# Esure sampler uses "trailig" timesteps.
pipe.scheduler = EulerDiscreteScheduler.from_cofig(pipe.scheduler.cofig, timestep_spacig="trailig")

# Esure usig the same iferece steps as the loaded model ad CFG set to 0.
pipe("A girl smilig", um_iferece_steps=4, guidace_scale=0).images[0].save("output.pg")

2-Step, 4-Step, 8-Step LoRA

Use LoRA oly if you are usig o-SDXL base models. Otherwise use our UNet checkpoit for better quality.

import torch
from diffusers import StableDiffusioXLPipelie, EulerDiscreteScheduler
from huggigface_hub import hf_hub_dowload

base = "stabilityai/stable-diffusio-xl-base-1.0"
repo = "ByteDace/SDXL-Lightig"
ckpt = "sdxl_lightig_4step_lora.safetesors" # Use the correct ckpt for your step settig!

# Load model.
pipe = StableDiffusioXLPipelie.from_pretraied(base, torch_dtype=torch.float16, variat="fp16").to("cuda")
pipe.load_lora_weights(hf_hub_dowload(repo, ckpt))
pipe.fuse_lora()

# Esure sampler uses "trailig" timesteps.
pipe.scheduler = EulerDiscreteScheduler.from_cofig(pipe.scheduler.cofig, timestep_spacig="trailig")

# Esure usig the same iferece steps as the loaded model ad CFG set to 0.
pipe("A girl smilig", um_iferece_steps=4, guidace_scale=0).images[0].save("output.pg")

1-Step UNet

The 1-step model is oly experimetal ad the quality is much less stable. Cosider usig the 2-step model for much better quality.

The 1-step model uses "sample" predictio istead of "epsilo" predictio! The scheduler eeds to be cofigured correctly.

import torch
from diffusers import StableDiffusioXLPipelie, UNet2DCoditioModel, EulerDiscreteScheduler
from huggigface_hub import hf_hub_dowload
from safetesors.torch import load_file

base = "stabilityai/stable-diffusio-xl-base-1.0"
repo = "ByteDace/SDXL-Lightig"
ckpt = "sdxl_lightig_1step_uet_x0.safetesors" # Use the correct ckpt for your step settig!

# Load model.
uet = UNet2DCoditioModel.from_cofig(base, subfolder="uet").to("cuda", torch.float16)
uet.load_state_dict(load_file(hf_hub_dowload(repo, ckpt), device="cuda"))
pipe = StableDiffusioXLPipelie.from_pretraied(base, uet=uet, torch_dtype=torch.float16, variat="fp16").to("cuda")

# Esure sampler uses "trailig" timesteps ad "sample" predictio type.
pipe.scheduler = EulerDiscreteScheduler.from_cofig(pipe.scheduler.cofig, timestep_spacig="trailig", predictio_type="sample")

# Esure usig the same iferece steps as the loaded model ad CFG set to 0.
pipe("A girl smilig", um_iferece_steps=1, guidace_scale=0).images[0].save("output.pg")

ComfyUI Usage

Please always use the correct checkpoit for the correspodig iferece steps. Please use Euler sampler with sgm_uiform scheduler.

2-Step, 4-Step, 8-Step Full

1. Dowload the full checkpoit (sdxl_lightig_Nstep.safetesors) to /ComfyUI/models/checkpoits.
2. Dowload our ComfyUI full workflow.

2-Step, 4-Step, 8-Step LoRA

Use LoRA oly if you are usig o-SDXL base models. Otherwise use our full checkpoit for better quality.

1. Prepare your ow base model.
2. Dowload the LoRA checkpoit (sdxl_lightig_Nstep_lora.safetesors) to /ComfyUI/models/loras
3. Dowload our ComfyUI LoRA workflow.

1-Step

The 1-step model is oly experimetal ad the quality is much less stable. Cosider usig the 2-step model for much better quality.

1. Update your ComfyUI to the latest versio.
2. Dowload the full checkpoit (sdxl_lightig_1step_x0.safetesors) to /ComfyUI/models/checkpoits.
3. Dowload our ComfyUI full 1-step workflow.

Cite Our Work

@misc{li2024sdxllightig,
      title={SDXL-Lightig: Progressive Adversarial Diffusio Distillatio}, 
      author={Shachua Li ad Ara Wag ad Xiao Yag},
      year={2024},
      eprit={2402.13929},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

SDXL-Lightning SDXL-Lightning is a lightning-fast text-to-image generation model. It can generate h