Researchers have developed an AI that can create a 3D model of your face just by looking at a single photo.
Typically, 3D face reconstruction poses ‘extraordinary difficulty,’ as it requires multiple images and must work around the varying poses and expressions, along with differences in lightning, according to the team.
By training a neural network on a dataset of both 2D images and 3D facial models or scans, however, their AI can reconstruct the entire face – even adding in parts that might not have been visible in the photo.
and click here to try it on your own snaps
Researchers have developed an AI that can create a 3D model of your face just by looking at a single photo. In one example on their website, the researchers tested it out on a photo of former president Barack Obama
The system developed by researchers at the University of Nottingham and Kingston University relies on a convolutional neural network (CNN) to overcome some of the challenges of 3D face reconstruction.
The network learned how to map a face from pixels to 3D coordinates, and essentially works with any picture of a face, the team explains in the paper published to arXiv.
It can even reconstruct faces in arbitrary poses or expressions.
To prove its capabilities, the researchers have also released a version you can try yourself, by uploading a photo of your choice.
For the best results, they recommend using a frontal photo, or as close to this as possible.
The network learned how to map a face from pixels to 3D coordinates, and essentially works with any picture of a face, the team explains in the paper published to arXiv. Apple CEO Tim Cook is pictured
To prove its capabilities, the researchers have also released a version you can try yourself , by uploading a photo of your choice. A 3D image of Rihanna is shown above
While the AI might make the process seem easy, researchers continue to struggle with 3D face reconstruction, the team explains in the paper.
‘Current systems often assume the availability of multiple facial images (sometimes from the same subject) as input, and must address a number of methodological challenges such as establishing dense correspondences across large facial poses, expressions, and non-uniform illumination,’ the authors write.
‘In general these methods require complex and inefficient pipelines for model building and fitting.’
The new AI, on the other hand, does not adhere to these limitations.
By training a neural network on a dataset of both 2D images and 3D facial models or scans, however, their AI can reconstruct the entire face – even adding in parts that might not have been visible in the photo. A 3D render of Rihanna’s face is pictured
The team trained the convolutional neural network on a dataset of more than 60,000 2D facial images and 3D meshes.
It doesn’t require accurate alignment, and can overcome arbitrary differences in pose and expression for its reconstruction.
According to the researchers, their system achieves a ‘direct regression of volumetric representation of the 3D facial geometry,’ from the image.
The system developed by researchers at the University of Nottingham and Kingston University relies on a convolutional neural network (CNN) to overcome some of the challenges of 3D face reconstruction. Its scan of actor Elijah Wood is pictured
For the best results, they recommend using a frontal photo, or as close to this as possible. This image of Selena Gomez, for example, did not turn out quite as well as the others
This means it can predict the coordinates of the 3D vertices based on the given 2D image.
And, these reconstructions can be further improved by incorporating 3D facial landmark localizations.
As of now, the team says this is the first time this particular method has been used, and offers a ‘very simple approach’ to the challenge.
Typically, 3D face reconstruction poses ‘extraordinary difficulty,’ as it requires multiple images and must work around the varying poses and expressions, along with differences in lightning, according to the team. But, the new system can overcome these challenges