AI could be about to make 3D modeling in a whole lot quicker and easier, if Nvidia has its way.

Fresh from the company’s research lab, Neuralangelo is an AI model that transforms 2D video clips into “extremely high fidelity” 3D structures.

“Creative professionals can then import these 3D objects into design applications, editing them further for use in art, video game development, robotics, and industrial digital twins,” the company explained in its latest paper.

Real-world high-fidelity

Using videos as reference points is hardly anything new - even when you do throw relatively new and experimental AI into the mix. But, as Nvidia notes in its latest paper, “current methods struggle to recover detailed structures of real-world scenes.” In other words, they can replicate objects and scenes, but it won't feel lifelike.

And it’s that high fidelity Neuralangelo is promising to deliver. According to Nvidia Research, the minds behind the model, it's able to vividly recreate the textures of different materials, like glass, marble, and even roof shingling. The model has also proved capable of reconstructing building interiors and exteriors.

“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” said Ming-Yu Liu, senior director of research and one of the paper's co-authors.

Neuralangelo’s applications in art and games development are already clear: think of the vast cityscapes of Grand Theft Auto, the real-world historical setting of Assassin’s Creed. Whole virtual towns could soon be realized based on videos recorded on a smartphone. But it’s only a matter of time before AI models like these also form a core part of all the best architecture software and best landscape design software, if it really can streamline real-life reconstructions.

In a demo released by the company, results were admittedly impressive: highly detailed AI renderings of Michelangelo's David based on video footage that lived up to boasts of “detailed large-scale scene reconstruction from RGB video.” But we're still a way off from dropping AI 3D models straight into projects without requiring some finishing touches.