This AI Image Generator Transforms 2D Images Into 3D Images
A new generative AI tool claims that it can coherently produce 3D images out of 2D ones.
The makers of DreamCraft3D call it a “hierarchical 3D content generation method that produces high-fidelity 3D objects.”
The examples given by DreamCraft3D, which was developed at Tsinghua University, Beijing, are mightily impressive but in all of the examples, an AI image is used as the input.
However, as is now common knowledge, images generated from artificial intelligence (AI) are nearing comparable levels to real photos so perhaps this technology could be used to make 3D images out of photographs taken by the hands of a human.
To create coherent 3D images, researchers perform “score distillation sampling via a view-dependent diffusion model.” This prioritizes geometry consistency and then “bootstrapped score distillation” boosts the texture.
“We train a personalized diffusion model, Dreambooth, on the augmented renderings of the scene, imbuing it with 3D knowledge of the scene being optimized,” write the researchers.
“The score distillation from this 3D-aware diffusion prior provides view-consistent guidance for the scene.”
80.lv notes that the authors of the paper are attempting to solve the problem of inconsistency of generated 3D objects which is commonplace in other models.
DreamCraft3D might be the best attempt yet to fix this issue and is yet another example of the growing prowess of AI image generators.
As generative AI continues to transform the imaging landscape, tools like this could offer photographers and visual artists the opportunity to create different and compelling imagery.
While AI is feared and loathed by many artists, tools such as Photoshop’s Generative Fill offer ways of leveraging the cutting-edge technology.
More about DreamCraft3D’s generative AI model can be viewed on GitHub. Check out PetaPixel’s story on the origins of 3D images.
Image credits: Courtesy of Jingxiang Sun et al.
