Scenedreamer

SceneDreamer is an AI tool that can generate 3D scenes from 2D image collections. It uses an unconditional generative model to synthesize large-scale 3D landscapes with 3D consistency, well-defined depth, and free camera movement. The tool does not require any 3D annotations and is trained solely on in-the-wild 2D images. At its core, SceneDreamer utilizes an efficient 3D scene representation, a generative scene parameterization, and an effective renderer. The 3D scene is represented using a bird"s-eye-view (BEV) representation comprised of a height field and a semantic field, enabling 3D scenes to be represented with quadratic complexity and disentangled geometry and semantics. The generative neural hash grid is used to parameterize the latent space, encoding generalizable features across scenes. A neural volumetric renderer, trained through adversarial training, is employed to produce photorealistic images. SceneDreamer is capable of generating diverse and vivid unbounded 3D worlds.