NVIDIA demos Instant NeRF: composite 2D photos into 3D scenes in seconds – nVIDIA – cnBeta.COM

NVIDIA’s latest AI demo was impressive: The tool can render dozens of 2D snapshots into a 3D scene. In the video, it is written that 75 years ago, Polaroid realized the imaging of photos in minutes, and now NVIDIA can compose 3D scenes from 2D photos in seconds.

As you can see in the animation below, a model dressed like Andy Warhol is holding a vintage Polaroid camera. The tool is called Instant NeRF, referring to “Neural Radiation Fields” — a technique developed in 2020 by researchers at UC Berkeley, Google Research and UC San Diego.

If you want a detailed explanation of neural radiation fields, you can read it here. But in a nutshell, the method maps the color and light intensity of different 2D shots, and then generates data that connects these images from different vantage points to render a finished 3D scene. In addition to images, the system also needs data about the camera’s position.

For several years, researchers have been improving this model from 2D to 3D, adding more detail to the finished render and increasing rendering speed. NVIDIA says its new Instant NeRF model is one of the fastest ever developed, reducing rendering time from a few minutes to a process that is “almost instantaneous.”

NVIDIA’s Isha Salian writes: “Instant NeRF can be used to create avatars or scenes for virtual worlds, capture video conference participants and their environments in 3D, or reconstruct scenes for 3D digital maps.” The technology can be used to train robots and automated Drive a car and learn about the size and shape of real-world objects by capturing 2D images or video clips of them. It can also be used in the architecture and entertainment industries to rapidly generate digital representations of real environments that creators can modify and build upon.”