At MediaNama’s Deepfakes and Democracy event held on January 17, Gautham Koorma, machine learning engineer and researcher from UC Berkeley, explained the various deepfake generation techniques available to users, the techniques available to detect deepfakes, the accuracy of these techniques, reasons why watermarking and methods like hashing, used for detecting child sexual abuse material (CSAM) don’t work for deepfakes, and more.
What are various methods to generate deepfakes?
Gautham Koorma outlined various methods available to generate deepfake images, videos, and audio:
Deepfake images
- Using Generative adversarial networks (GANs): “Essentially what a GAN does is it takes what’s random noise, generates an image through a generator and then sends it to another network called a detector. This process goes on in an adversarial fashion until the generator can create an image that fools the detectors.” Websites like thispersondoesnotexist.com use this technique to generate images of people that look real but aren’t.
- Using diffusion-based models: Stability AI and Midjourney utilize diffusion-based models to generate images based on the model’s interpretation of text.
- Face swapping: “You’ve two people, a donor and a target. And what you can do is remove essentially from the face part of the target, their features, and then take the features of the donor and put it in.” While these were technical a few years ago, right now there are easy tutorials that anybody can follow.
Deepfake videos
- Face swapping: This works similarly to image face swap since videos are nothing but a sequence of frames or images. So a video face swap tool creates a deepfake version for each frame and stitches them together to create a video. The controversial video involving actress Rashmika Mandanna is an example of a face swap deepfake.
- Lip sync: This is similar to a face swap. “Essentially what it does is it keeps most of the face the same, but it takes an audio and just generates the lip movement. So, you can take, for example, a politician, and if they don’t speak a regional language like Kannada or Malayalam, you take an audio in Kannada or Malayalam, and you can have their lips move accordingly and put that audio.”
- Puppet master: “This is a little bit more sophisticated, but essentially what it does is if you have an image of a person, you can have someone like me sitting behind, and that image is the puppet and I’m the puppet master, and it’ll take my facial expressions and control that image.”
Deepfake audio
- Text-to-speech: “Essentially the way text-to-speech works is it takes text input, converts that into what’s called an embedding or an intermediate representation, and then sends it through a vocoder. And the vocoder is trained on the voice of a person. So, for example, if I had hours in Nikhil’s voice, I could train the vocoder on that and then take text, create an intermediate representation and start generating speech in Nikhil’s voice.”
- Voice cloning: “There’s another technique called voice cloning. You might’ve seen it on social media. There’s a lot of memes of the prime minister singing different songs. You can go to YouTube, take a song, put it into an open-source tool (like this), go to the net, find a model of the president, or for that matter, you can find models of Salman Khan and many other celebrities (over here, for example), put it into the tool and start playing with it to generate audio in their voice.”
What are some techniques to detect deepfakes?
1. Detection using machine learning: This is when you identify evidence of manipulation in digital media using machine learning (ML). ML has better performance than statistical methods of detection used earlier, but the drawback is that the internal workings of this approach are less transparent and harder to interpret. You will get a “yes” or “no” answer, but not any explanation as to why the media is real or fake. The role of human analysts is important in this process because these models tend to get things wrong quite often, Koorma explained.
- Accuracy: Koorma and fellow researchers were able to detect deepfakes involving cloned voices with 90 percent accuracy using various techniques but only in a controlled, lab setting. This number falls significantly when this audio content is in the wild i.e. shared on social media platforms. “For example, when you upload an audio clip to Facebook or when you send it on WhatsApp, each of these platforms do something called transcoding, essentially changing the bit rate, changing some properties of the media. And once that happens, we see that the accuracy of detection drops a lot, sometimes higher than 10 per cent. So, it’s really hard to detect these things once they’ve gone on social media,” Koorma elaborated. This problem extends beyond audio to images and video as well.
- Consequences of wrong detection: AI detectors can be wrong. For example, one of the popular AI detector tools got images from the war in Gaza wrongly labelled as a deepfake “and that has grave consequences” because multiple prominent news outlets carried this false flag in their reporting, Koorma pointed out. Even if the algorithm is 90 percent accurate, it is not enough because you are getting a false detection in 10 out of 100 cases. “That is huge at the scale of social media, where you have billions of posts, versus if you’re doing it for something in a court case, it’s still a reasonable tolerance. So, when you’re using these algorithms on social media for automated detection of deepfakes you want to get 99.999% accuracy,” Koorma added.
- Easier to detect deepfake audio than video: Detecting deepfake videos is the hardest because of the computational processing required. Audio detection is showing great results in lab settings. It’s also easier to develop algorithms to detect audio deepfakes of famous people because it is easier to train such detection algorithms. “So, protecting world leaders, protecting really important celebrities with hours of media online is a relatively easier problem than trying to create a detector that casts a wide net and wants to catch every fake out there,” Koorma said.
- Platforms don’t engage in detection because of the complexity involved: Koorma said that he isn’t aware of any platform using detection at the source because of the variety of challenges involved including the computation complexity. “When you’re uploading a video, every time you have to analyze it using many models, plus even if you do that, the accuracy is relatively not at the level that they would want to productionize.” So platforms are rather taking down deepfakes after it has been flagged to them.
- It’s an arms race between detectors and adversaries: Social media platforms and researchers are publishing papers on detection but it remains a hard problem “because the adversary, once they know of these detection algorithms that we have published in our research, they just make their generation even better. It’s an arms race and it’s always been,” Koorma remarked.
2. Authentication based on provenance info: This technique involves identifying a deepfake based on the provenance of the content such as when the media was created, when it was edited, what software was used, etc. All this is embedded as metadata or imperceptible watermarks by the software used to make them.
- All watermarks can be broken: Researchers at the University of Maryland have looked at every watermarking technique that exists out there, and they’ve broken every one of them. “I think we should be using watermarking, but keeping in mind its downside, that it can be easily broken by a sophisticated adversary. […] For a non-sophisticated actor, it might be a little bit more difficult, but this is why the push for detection, because once you have removed that watermark, then what do you do?” Koorma asked.
- Can fingerprinting or adding provenance be mandated? Tarunima Prabhakar, co-founder of Tattle Civic Technologies, suggested that fingerprinting (adding provenance info to) a deepfake at the source is going to be necessary and to prevent users from removing the fingerprint, platforms should penalise content that doesn’t have a fingerprint. She warned that detecting deepfakes using algorithms isn’t going to work in the future when AI gets to a sophistication where it completely bypasses technical detection. But even if the image was to carry a fingerprint, these can be faked, Medianama Editor Nikhil Pahwa pointed out.
- Providing signatures or verification marks for celebrities and prominent entities: Alternatively, an audience member asked if it is possible to have a specific signature of an official media of the government or from celebrities so that content can be checked for the signature variable, almost like a verification symbol on official media. Tarunima Prabhakar responded that media organisations like the New York Times are experimenting with this, but Koorma warned that it is easy to fake a signature as well.
3. Hashing: Hashing is a commonly used technique in detecting child sexual abuse material (CSAM). Essentially, law enforcement and other agencies that receive reports of CSAM material, create a hash value for the reported material and store it in a database that is shared with social media platforms. Whenever a user uploads content on a social media platform, the platform compares a hash of the upload to the hashes in the CSAM database and doesn’t allow the upload if there is a similarity. This technique is resistant to minor modifications such as resizing and colour changes. Because of this, it is good at detecting CSAM or copyrighted content.
- Why hashing won’t work for deepfakes: Using this technique for deepfakes is more complicated. “The biggest problem with deepfakes is that [something new is] generated every time, and the content can change quite a bit, as opposed to the classical photographs that we have. These techniques are based on the idea that there’s a database of hashes that we can cross-check with really quickly. And so it’s really hard for a company to maintain a database of hashes of every possible content that can be generated,” Koorma elaborated. Another issue with hashing is the problem of hash colliders where two different images have the same neural hash, Saikat Dutta, CEO and co-founder of DeepStrat, pointed out.
- Solving the CSAM problem is also going to be much harder now: “Before the advent of generative AI, CSAM was usually photographs, disturbing photographs of explicit material involving children that could be collected by law enforcement agencies. The scale at which they were being generated was limited. […] This is not true for deepfakes because now what we’re seeing is the likeness of a child being taken and used to generate explicit material, which can be generated at scale and volumes that are unprecedented,” Koorma explained.
- Using hashing is also problematic for privacy and freedom of speech: One of the biggest pushback against using hashing, even for CSAM, is a privacy concern because the same hashing can then be used to identify individuals and videos to some extent, Koorma pointed out. MediaNama has also previously covered how CSAM-detecting technology can be used to engage in censorship and curb free speech. For instance, an authoritarian government can ask a platform to automatically take down images criticising the government by maintaining a hash bank of images flagged by the government.
STAY ON TOP OF TECH NEWS: Our daily newsletter with the top story of the day from MediaNama, delivered to your inbox before 9 AM. Click here to sign up today!
Also Read
- 11 Talking Points From MediaNama’s ‘Deepfakes And Democracy’ Discussion #NAMA
- How Deepfakes Could Lead To Greater Misinformation In The Upcoming Elections #NAMA
- Can Deepfakes Be Used To Exploit User Verification Mechanisms? #NAMA
- How Can The Government And Companies Regulate Spread Of Deepfakes? #NAMA
