MotionCtrl Model Card
[**Project Page**](https://wzhouxiff.github.io/projects/MotionCtrl/) **|** [**Paper (ArXiv)**](https://arxiv.org/pdf/2312.03641.pdf) **|** [**Code**](https://github.com/TencentARC/MotionCtrl)
[? **Gradio demo (MotionCtrl+VideoCrafter)**](https://huggingface.co/spaces/TencentARC/MotionCtrl) **|** [? **Gradio demo (MotionCtrl+SVD)**](https://huggingface.co/spaces/TencentARC/MotionCtrl_SVD)
Introduction
MotionCtrl, when provided with specific camera movements indicated by camera poses or several trajectories, can independently control both the camera and object motion in a generated video. It can be deployed on both Text-to-Video generation models, such as LVDM/VideoCrafter and AnimateDiff, as well as Image-to-Video generation models like SVD.
MotionCtrl + SVD
- Model: motionctrl.pth
- Results:
MotionCtrl + LVDM/VideoCrafter
- Model: motionctrl_svd.ckpt
- Results:
MotionCtrl + AnimateDiff
- Model: [Coming Soon]()
- Results:
Usage
- Download directly in this repository OR Download with python script
from huggingface_hub import hf_hub_download
motionctrl_lvdm_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl.ckpt", repo_type="model")
motionctrl_svd_ckpt = hf_hub_download(repo_id="TencentARC/MotionCtrl", filename="motionctrl_svd.ckpt", repo_type="model")
- Then generate the controllable video following the instructions in our GitHub repository.
Citation
@inproceedings{wang2023motionctrl,
title={MotionCtrl: A Unified and Flexible Motion Controller for Video Generation},
author={Wang, Zhouxia and Yuan, Ziyang and Wang, Xintao and Chen, Tianshui and Xia, Menghan and Luo, Ping and Shan, Yin},
booktitle={arXiv preprint arXiv:2312.03641},
year={2023}
}
评论