Japanese Stable Diffusion XL

Model Details

Japanese Stable Diffusion XL (JSDXL) is a Japanese-specific SDXL model that is capable of inputting prompts in Japanese and generating Japanese-style images.

示例代码

from modelscope.utils.constant import Tasks
from modelscope.pipelines import pipeline
import cv2

pipe = pipeline(task=Tasks.text_to_image_synthesis, 
                model='AI-ModelScope/japanese-stable-diffusion-xl',
                use_safetensors=True,
                model_revision='master')

prompt = '柴犬、カラフルアート'
output = pipe({'text': prompt})
cv2.imwrite('result.png', output['output_imgs'][0])

Model Details

• Developed by: Stability AI
• Model type: Diffusion-based text-to-image generative model
• Model Description: This model is a fine-tuned model based on SDXL 1.0. In order to maximize the understanding of the Japanese language and Japanese culture/expressions while preserving the versatility of the pre-trained model, we performed a PEFT training using one Japanese-specific compatible text encoder. As a PEFT method, we applied Orthogonal Fine-tuning (OFT) for better results and training stability.
• License: STABILITY AI JAPANESE STABLE DIFFUSION XL COMMUNITY LICENSE

Uses

Direct Use

The model is intended for research purposes only. Possible research areas and tasks include

• Generation of artworks and use in design and other artistic processes.
• Applications in educational or creative tools.
• Research on generative models.
• Safe deployment of models which have the potential to generate harmful content.
• Probing and understanding the limitations and biases of generative models.

Excluded uses are described below.

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Limitations and Bias

Limitations

• The model does not achieve perfect photorealism
• The model cannot render legible text
• The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
• Faces and people in general may not be generated properly.
• The autoencoding part of the model is lossy.

Bias

While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.

How to cite

@misc{JSDXL, 
    url    = {[https://huggingface.co/stabilityai/japanese-stable-diffusion-xl](https://huggingface.co/stabilityai/japanese-stable-diffusion-xl)}, 
    title  = {Japanese Stable Diffusion XL}, 
    author = {Shing, Makoto and Akiba, Takuya}
}

Contact

• For questions and comments about the model, please join Stable Community Japan.
• For future announcements / information about Stability AI models, research, and events, please follow https://twitter.com/StabilityAI_JP.
• For business and partnership inquiries, please contact partners-jp@stability.ai. ビジネスや協業に関するお問い合わせはpartners-jp@stability.aiにご連絡ください。