How do you make Queen Victoria smile like Miley Cyrus?
That’s not the beginning of a joke. Technology has helped animate those regal features, and may also improve everything from the films you see at the cinema to the gurning expressions in your selfies.
Weiyao Lin at Shanghai Jiao Tong University in China and colleagues are behind the project. Their aim was to design a programme that could clone someone’s facial expression, and seamlessly paste it onto a face in another picture. It sounds simple, until you consider the way even a simple smile can stretch and pull skin across the whole face. Replicating those contortions on a second face is particularly tricky, given the vast differences in people’s facial shape: previous attempts sometimes end up with a stitched-together image that resembles Frankenstein’s monster.
Lin’s algorithm, however, analyses two still photos - one neutral, and one of the desired expression - to measure minute differences in the musculature of the face and the relative size of the facial features. From this, the programme can estimate the way your flesh moves to create the expression, and then applies it to any other face of your choosing. His research was recently published in the Journal of Visual Communication and Image Representation.
We recently asked Lin and his colleagues to apply his algorithm to some famous historical figures. The results are a little eerie.
We gave the famously mysterious Mona Lisa a disdainful sneer:
Lin also transferred Miley Cyrus's "twerking face" to Her Royal Highness Queen Victoria:
Lin hopes that the same technology could allow actors to seamlessly animate figures in films like Frozen – without the complex motion-sensing technology that’s currently used. “This not only facilitates the process of professional movie production, but also allows millions of amateurs to create their DIY special-effect videos and share on the internet,” Lin says.
An automatic algorithm could also be used to improve video calls; rather than transferring a continuous video, the algorithm could track your facial expression to animate a static photo or avatar. That should save the cost of transmitting all that extra data involved in a live video, while still communicating the essential information – your emotions.
Less seriously, you might just use the programme to correct a gormless face in a selfie. “Imagine one took a picture in a scenic spot and want to share it to friends. However, the expression may not look good in the picture. In this case, this technique can be applied to transfer a smile to the face, and make the expression more satisfactory,” Lin says.
We’re not sure whether the famously stern Queen Victoria would agree it’s always an improvement, however.
David Robson is BBC Future’s feature writer. He is @d_a_robson on twitter.
Follow us on Facebook, Twitter, Google+ and LinkedIn