A global team of computer scientists have developed a novel method to replicate physical objects in Virtual Reality using the point-and-shoot camera with a flash on a smartphone.
The team have demonstrated the use of a novel algorithm to recreate a variety of physical objects in Virtual Reality using basic flash photography capability, such as that of a smartphone.
The difficulty of replicating physical objects
Min H. Kim, associate professor of computer science at KAIST in South Korea and lead author of the research, said: “To faithfully reproduce a real-world object in the VR/AR environment, we need to replicate the 3D geometry and appearance of the object. Traditionally, this has been either done manually by 3D artists, which is a labour-intensive task, or by using specialized, expensive hardware. Our method is straightforward, cheaper and efficient, and reproduces realistic 3D objects by just taking photos from a single camera with a built-in flash.”
The novel algorithm
The algorithm uses basic flash photography to replicate the geometry and appearance of 3D objects. The results were consistently reproduced using examples of diverse objects such as metal, wood, plastic, ceramic, resin, and a detailed miniature Nefertiti statue.
The novel algorithm, which does not require any input geometry of the target object, successfully captured the geometry and appearance of 3D objects with basic, flash photography and reproduced consistent results. Examples that were showcased in the work included diverse set of objects that spanned a wide range of geometries and materials, including metal, wood, plastic, ceramic, resin and paper, and comprised of complex shapes like a finely detailed mini-statute of Nefertiti.
Kim added: “Many traditional methods using a single camera can capture only the 3D geometry of objects, but not the complex reflectance of real-world objects, given by the SVBRDF. Using only 3D geometry cannot reproduce the realistic appearance of the object in the AR/VR environment. Our technique can capture high-quality 3D geometry as well as its material appearance so that the objects can be realistically rendered in any virtual environment.”