Three years later, the Mugsy rig can be replaced with a recent generation smartphone’s front-facing selfie camera. Instead of spending an hour in a chair surrounded by hundreds of cameras, users simply have to pan their smartphone across their face, from side-to-side, and then recreate a series of 65 specific facial expressions. The researchers say the process now takes about three-and-a-half minutes, and using a neural network that was previously trained on the 3D facial data captured from 255 diverse subjects inside a camera rig similar to Mugsy, the new approach can generate surprisingly lifelike 3D avatar models.
Advertisement
The process still isn’t instant. Once the face scans and various expressions have been performed and captured, a computer with serious number-crunching capabilities still needs about six hours to render the results. But this is once again where the cloud shows its usefulness, as individual users won’t require a high-end PC at home—all the rendering can be done somewhere else. The process also won’t work if someone is wearing glasses, and it does a bad job at recreating long hair. It also only works on heads, so while the 3D avatars created might look realistic enough to finally escape the uncanny valley, only interacting with your friends and family’s disembodied heads is still going to bring with it some level of creepyness.