FreeU文本生成图像模型

FreeU: Free Lunch in Diffusion U-Net

模型描述

FreeU，一种免费大幅提高扩散模型样本质量的方法：无需训练，无需引入额外参数，也无需增加内存或采样时间。

FreeU, a method that substantially improves diffusion model sample quality at no costs: no training, no additional parameter introduced, and no increase in memory or sampling time.

期望模型使用方式以及使用范围

如何使用

代码范例

import cv2
from modelscope.pipelines import pipeline
from modelscope.utils.constant import Tasks

prompt = "a photo of a running corgi"  # prompt
output_image_path = './result.png'
input = {'prompt': prompt}

base_model = 'AI-ModelScope/stable-diffusion-v1-5' # SD1.5
freeu_params = {'b1': 1.5, 'b2': 1.6, 's1': 0.9, 's2': 0.2}

# base_model = 'AI-ModelScope/stable-diffusion-v2-1'  # SD2.1
# freeu_params = {'b1': 1.4, 'b2': 1.6, 's1': 0.9, 's2': 0.2}

# base_model = 'AI-ModelScope/stable-diffusion-xl-base-1.0'  # SDXL
# freeu_params = {'b1': 1.3, 'b2': 1.4, 's1': 0.9, 's2': 0.2}


pipe = pipeline(
    Tasks.text_to_image_synthesis,
    model='damo/multi-modal_freeu_stable_diffusion',
    base_model=base_model,
    freeu_params=freeu_params)

output = pipe(input)['output_img']
cv2.imwrite(output_image_path, output)
print('pipeline: the output image path is {}'.format(output_image_path))

相关论文以及引用信息

@article{si2023freeu,
  title={FreeU: Free Lunch in Diffusion U-Net},
  author={Si, Chenyang and Huang, Ziqi and Jiang, Yuming and Liu, Ziwei},
  journal={arXiv preprint arXiv:2309.11497},
  year={2023}
}

Clone with HTTP

 git clone https://www.modelscope.cn/damo/multi-modal_freeu_stable_diffusion.git