As someone who studies the radical right and their voters for a living, I have a longstanding interest in the spread of disinformation on social media and the technologies that foster it. One new-ish development in this area is the disruption caused by generative AI and its potential for creating deepfakes.
Of course, deepfakes are nothing new: the autocracies of the 20th century were already quite good at creating them in the 1930s and 1940s.
Source: Wikimedia. Alexander Malchenko, originally at the centre of the photograph, has been edited out.
Modifying negatives with razor, brush, and gouache, however, required a lot of time and skill. With the advent of Photoshop and similar programs, things became easier, but only relatively speaking. Creating fakes was still nothing anyone could do.
But when some random guy on shrooms used Midjourney to make the pope don a Balenciaga jacket by simply describing this scene in a few words, everyone and their grandparents realized that something had changed.
And not necessarily for the better: at about the same time, it emerged that Germany’s radical right AfD party is using AI to generate fake images of scary immigrants. But creating fake video and audio – Obama or Merkel or Biden apparently saying outrageous things and such – still required money, expertise, and very expensive computing equipment, right? Right.
And that’s why I was intrigued when someone mentioned heygen, a company that offers the services of “avatars”: AI-generated talking heads that can be used in videos to narrate a script. The ready-made avatars cover a good range of ethnicities and accents, but if that is not enough, you can create an “instant avatar” from sound and video that you submit on their website. While I had some misgivings, I wanted to test this, so I registered for a free trial. The result is this video.
Democratising deepfake creation
Heygen claims that even 30 seconds of video and audio from your phone are sufficient to get started. But thanks to the pandemic, I still have a reasonably good webcam at my desk, and a green screen to hide the mess behind me. I talked to the camera for just over two minutes. As per the instructions, I smiled a lot and took care to close my mouth occasionally. Uploading the footage, typing my script, and rendering the video took another 10 minutes or so. It also cost me one of my two credits that came with the free trial.
The result is not perfect. The voice is just a tiny bit robotic (shouldn’t there be a bit of pause after the rhetorical question?) and the reflection on my forehead is unfortunate. At one point, my avatar seems to jump a bit towards the camera. Also, people who know me in real life know that I do not smile that much. Even if I did, the slightly maniacal facial expressions do not mesh well with the somewhat bleak narration. And why this odd hand gesture towards the end? But although I did not try very hard (e.g. I did not remove my glasses, ignored the option to insert more pauses, and did not read the useful hints for improving the avatars’ enunciation), the result that I got in under 15 minutes is at least plausible. While there is something uncanny about the whole performance, if you watch it on a small screen and squint a bit, it looks and talks like me.
Should we be worried?
Heygen’s offer is not exactly cheap, but it doesn’t break the bank either. They clearly aim at people who produce explainer videos or similar educational stuff at scale and don’t want to hire professional speakers/actors. For about $1,200 per year, you can produce 12 hours of video. If I was still in the business of recording endless lectures from home, I might be tempted to have my clone read out the script (no more re-takes!).Of course, 12 hours would not even cover a single semester. But if I was a bad actor, 12 hours of deepfaked TikToks and tweets might be plenty to cover an electoral campaign or two.
Heygen seems to have at least some safeguards in place. I had to remove a (pretty mild) expletive and a reference to identity theft from my script before the video would render. But would they recognize a public figure in the footage that’s uploaded to their platform? Could I use a screengrab from Olaf Scholz’s latest press conference (heygen is multilingual) and have him say that AfD voters are morons? Could I upload a few minutes of PMQ and have Sunak say that anyone from India is now welcome to settle in the UK, while Starmer promises to re-apply for EU membership?
Perhaps I could. And at the end of the day, this would not have to be perfect: just good enough to sow doubt and division.
I have only one credit left and other, more urgent things to do, so I will not test the limits of what this platform can do. But on the face of it, it looks like reasonably well-funded far-right actors the world over no longer need Russia’s troll factories to wreak havoc. And like, as always, we shouldn’t trust anything that’s on the internet.
Of course you didn’t.
In times the radical rights “Flood the zone with shit” Kai offers intersting insights and kickstarts thinking.