? Requirements

This repo consists of DialogGen (a prompt enhancement model) and Hunyuan-DiT (a text-to-image model).

The following table shows the requirements for running the models (The TensorRT version will be updated soon):

• An NVIDIA GPU with CUDA support is required.
• We have tested V100 and A100 GPUs.
• Minimum: The minimum GPU memory required is 11GB.
• Recommended: We recommend using a GPU with 32GB of memory for better generation quality.
• Tested operating system: Linux

?️ Dependencies and Installation

Begin by cloning the repository:

git clone https://github.com/tencent/HunyuanDiT
cd HunyuanDiT

We provide an environment.yml file for setting up a Conda environment. Conda's installation instructions are available here.

# 1. Prepare conda environment
conda env create -f environment.yml

# 2. Activate the environment
conda activate HunyuanDiT

# 3. Install pip dependencies
python -m pip install -r requirements.txt

# 4. (Optional) Install flash attention v2 for acceleration (requires CUDA 11.6 or above)
python -m pip install git+https://github.com/Dao-AILab/flash-attention.git@v2.1.2.post3

? Download Pretrained Models

To download the model, support git clone and modelscope SDK

Then download the model using the following commands:

# Create a directory named 'ckpts' where the model will be saved, fulfilling the prerequisites for running the demo.
mkdir ckpts
# Use the huggingface-cli tool to download the model.
# The download time may vary from 10 minutes to 1 hour depending on network conditions.
git clone https://www.modelscope.cn/modelscope/HunyuanDiT.git
mv HunyuanDiT/* ckpts/

Note：If an No such file or directory: 'ckpts/.huggingface/.gitignore.lock' like error occurs during the download process, you can ignore the error and retry the command by executing huggingface-cli download Tencent-Hunyuan/HunyuanDiT --local-dir ./ckpts

All models will be automatically downloaded. For more information about the model, visit the Hugging Face repository here.

? Inference

Using Gradio

Make sure you have activated the conda environment before running the following command.

# By default, we start a Chinese UI.
python app/hydit_app.py

# Using Flash Attention for acceleration.
python app/hydit_app.py --infer-mode fa

# You can disable the enhancement model if the GPU memory is insufficient.
# The enhancement will be unavailable until you restart the app without the `--no-enhance` flag. 
python app/hydit_app.py --no-enhance

# Start with English UI
python app/hydit_app.py --lang en

Using Command Line

We provide 3 modes to quick start:

# Prompt Enhancement + Text-to-Image. Torch mode
python sample_t2i.py --prompt "渔舟唱晚"

# Only Text-to-Image. Torch mode
python sample_t2i.py --prompt "渔舟唱晚" --no-enhance

# Only Text-to-Image. Flash Attention mode
python sample_t2i.py --infer-mode fa --prompt "渔舟唱晚"

# Generate an image with other image sizes.
python sample_t2i.py --prompt "渔舟唱晚" --image-size 1280 768

More example prompts can be found in example_prompts.txt

More Configurations

We list some more useful configurations for easy usage:

? BibTeX

If you find Hunyuan-DiT useful for your research and applications, please cite using this BibTeX:

@misc{hunyuandit,
      title={Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding},
      author={Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu, Zheng Fang, Weiyan Wang, Jinbao Xue, Yangyu Tao, JianChen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, Qinglin Lu},
      year={2024},
}