Stability AI Just Announced “Video 4D” with with Dynamic 3D Object Viewing

Stability AI has introduced Stable Video 4D, a new generative AI model that adds a new dimension to video generation by accepting video input and generating multiple novel-view videos from 8 different perspectives.

A next-gen AI video model: Stable Video 4D builds upon Stability AI’s existing Stable Video Diffusion model, taking the concept further by enabling users to view dynamically moving 3D objects from arbitrary camera angles:

• The model combines the strengths of Stability AI’s previously-released Stable Video Diffusion and Stable Video 3D models, fine-tuned with a carefully curated dynamic 3D object dataset.
• Stable Video 4D is a first-of-its-kind network that performs both novel view synthesis and video generation in a single network, unlike existing works that leverage separate networks for these tasks.

Understanding the 4D approach: Stable Video 4D goes beyond the typical 3D concept of width, height, and depth by incorporating time as the fourth dimension:

• The four dimensions include width (x), height (y), depth (z), and time (t), allowing users to view a moving 3D object from various camera angles and at different timestamps.
• The model synthesizes 8 novel view videos from scratch by using the original input video as guidance, without explicit transfer of pixel information from input to output.

Potential applications and future development: Stability AI sees Stable Video 4D being used in various industries and use cases:

• Potential applications include movie production, gaming, AR/VR, and other scenarios requiring the viewing of dynamically moving 3D objects from arbitrary camera angles.
• Currently, Stable Video 4D can process single-object videos of several seconds with a plain background, and the company plans to generalize it to longer videos and more complex scenes in the future.

Analyzing the impact: Stable Video 4D represents a significant advancement in generative AI for video, offering new possibilities for content creation and visualization across various industries:

• By enabling the generation of multiple novel-view videos from different perspectives and timestamps, Stable Video 4D opens up new avenues for creative exploration and immersive experiences.
• The model’s ability to synthesize videos from scratch using the input video as guidance, without explicit pixel information transfer, showcases the growing sophistication of AI-driven video generation techniques.
• As Stability AI continues to refine and expand the capabilities of Stable Video 4D, it has the potential to revolutionize how video content is created and consumed, particularly in areas such as entertainment, gaming, and virtual reality.