Stability AI’s new AI model can generate a 3D scene based on a single image.
Stability AI introduces a new AI model, Stable Virtual Camera, which the company claims can convert 2D images into “immersive videos with realistic depth and perspective”. The model can create a 3D scene based on just one image. Stable Virtual Camera is currently only available for research use under a non-commercial license.
From 2D to 3D
Last summer, Stability AI launched a model that converts one video into new videos from eight different perspectives. The newly announced model doesn’t go from video to 3D model, but from 2D image to 3D video. This multi-view diffusion model generates new views of a scene based on one or more input images from different camera angles, resulting in a 3D video.
Stable Virtual Camera can generate videos in various formats: square (1:1), portrait (9:16), and landscape (16:9) with aspect ratios up to 1,000 frames. Furthermore, the model can generate 3D videos based on one or up to 32 input images. The company notes that in certain scenarios, such as images of people, animals, or dynamic structures (e.g., water), the results may be of lower quality.
Moreover, users can determine the different camera angles for the 3D video themselves. The model is capable of generating videos that travel along “dynamic” camera paths, such as ‘Spiral’, ‘Dolly Zoom’, ‘Move’, or ‘Pan’.
Stable Virtual Camera is currently only available for research use under a non-commercial license and can be downloaded from Hugging Face.