This post originally appeared on MIT Technology Review
Facebook fears that AI-generated “deepfake” videos could be the next big source of viral misinformation—spreading among its users with potentially catastrophic consequences for the next presidential election.
Its solution? Making lots of deepfakes of its own, to help researchers build and refine detection tools.
Facebook has directed its team of AI researchers to produce a number of highly realistic fake videos featuring actors doing and saying routine things. These clips will serve as a dataset for testing and benchmarking deepfake detection tools. The Facebook deepfakes will be released at a major AI conference at the end of the year.
The rise of deepfakes has been driven by recent advances in machine learning. It has long been possible for movie studios to manipulate images and video with software and computers, and algorithms capable of capturing and recreating a person’s likeness have already been used to make point-and-click tools for pasting a person’s face onto someone else. The term “deepfake” is taken from a Reddit user who released such a tool in 2017. It refers to deep learning, the AI technique employed.
Making a deepfake normally requires two video clips. Algorithms learn the appearance of each face in order paste one onto the other while maintaining each smile, blink, and nod. Different AI techniques can also be used to recreate a specific person’s voice. Methods for spotting forged media already exist but they often involve painstaking expert analysis.
The company’s CTO, Mike Schroepfer, says the growing sophistication of deepfake clips, which show real people saying and doing things that never happened, is far ahead of any existing techniques for identifying video manipulation. So devising ways to flag or block potential fakes may be vital.
“We have not seen this as huge problem on our platforms yet, but my assumption is if you increase access—make it cheaper, easier, faster to build these things—it clearly increases the risk that people will use this in some malicious fashion,” said Schroepfer, who is spearheading the initiative, last night. “I don’t want to be in a situation where this is a massive problem and we haven’t been investing massive amounts in R&D.”
The social network will dedicate $10 million for funding the detection technology through grants and challenge prizes. Together with Microsoft, Partnership on AI, and academics from institutions including MIT, UC Berkeley and Oxford University, the company is launching a deepfake challenge, which will offer cash rewards for the best detection methods.
One of the big worries is that mass-produced video fakes could be used to spread highly contagious misinformation during the next year’s US election. Several US Senators have sounded the alarm about the threat and Ben Sasse (R–Nebraska) introduced a bill to make it illegal to create or distribute fakes with malicious intent.
Manipulated videos seem ideal for distribution on social platform. Earlier this year, a clip that appeared to show Nancy Pelosi slurring (made simply by slowing the footage down) rapidly spread across Facebook. The company refused to remove the post, as well as a deepfake of Mark Zuckerberg, instead choosing to flag the clips as fake with fact-checking organizations.
It makes sense for Facebook to try to get out ahead of the issue, especially after the fallout from the last presidential election. As details of political misinformation campaigns emerged, Facebook faced intense criticism for allowing such propaganda to spread.
Promoting the deepfake challenge might have untended consequences though, says Henry Ajder, an analyst at Deeptrace, a UK company that’s working on tools for spotting forged clips. He notes that the idea of deepfakes can let politicians dodge accountability by claiming that real information has been forged (see “Fake America great again”). “The mere idea of deepfakes is already creating a lot of problems,” Ajder says. “It’s a virus in the political sphere that’s infected the minds of politicians and citizens.”
Moreover, despite the alarm, Ajder, who tracks deepfakes in the wild, doubts that the technology will be weaponized for political ends for some time. He believes they will more immediately become a potent tool of cyber-stalking and bullying.
A few methods for detecting deep fakes already exist. Simple methods involve analyzing the data in a video file or looking for telltale mouth movement and blinking, which is more difficult for an algorithm to capture and recreate. A method developed recently by a group of leading experts involves training a deep learning algorithm to recognize the specific way a person’s head moves since this is not something that algorithms typically learn.
Comparing the effort to the fight against spam email, Schroepfer said Facebook may not be able to catch the most sophisticated fakes. “We’ll catch the obvious ones,” he said. But he said Facebook isn’t employing these methods yet because the forgeries are evolving so rapidly.
Many experts have been surprised and alarmed by the speed with which AI forgeries are progressing. Just this week, a Chinese app called Zao sparked debate by posting videos showing deepfake videos supposedly created from a still image. Hao Li, a visual effects artist and an associate professor at the University of Southern California, has warned that it may be possible to mass produce undetectable deepfakes before long (see “The world’s top deepfake artist is wrestling with the monster he created”).