Stable Video 3D

Stable Video 3D (SV3D) is a generative model based on Stable Video Diffusion that takes in a still image of an object as a conditioning frame, and generates an orbital video of that object.

Model Details

This model was trained to generate 21 frames at resolution 576x576 given a context frame of the same size, finetuned from SVD Image-to-Video. Please check our tech report and video summary for details.

We release two variants of the model:

1. SV3D_u: This variant generates orbital videos based on single image inputs without camera conditioning.
2. SV3Dp: Extending the capability of SVD3u, this variant accommodates both single images and orbital views allowing for the creation of 3D video along specified camera paths.

Model Description

• Developed by: Stability AI
• Model type: Generative image-to-video model
• License: StabilityAI Non-Commercial Research Community License. If you want to use this model for your commercial products or purposes, please contact us here to learn more.

Model Sources

• Repository: https://github.com/Stability-AI/generative-models
• Tech report: https://stability.ai/s/SV3D_report.pdf
• Video summary: https://youtu.be/Zqw4-1LcfWg
• Project page: https://sv3d.github.io
• arXiv page: https://arxiv.org/abs/2403.12008

Training Dataset

We use renders from the Objaverse dataset, utilizing our enhanced rendering method that more closely replicate the distribution of images found in the real world, significantly improving our model’s ability to generalize. We selected a carefully curated subset of the Objaverse dataset for the training data, which is available under the CC-BY license.

Usage

For usage instructions, please refer to our generative models GitHub repository

Out-of-Scope Use

The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model. The model should not be used in any way that violates Stability AI's Acceptable Use Policy.