November comes with a great Amped Authenticate update! We are adding deepfake detection capabilities, which increases the already broad coverage of Authenticate for localizing forgeries. Moreover, we’ve updated the Social Media Identification‘s signature database with hundreds of new signatures and added one more social media, or rather a messaging app: Facebook Messenger. In this blog post, we’ll first quickly review what deepfakes are, focusing on synthesized faces, and then we’ll go practical and discuss each improvement!
See the New Features in Action!
Deepfakes are the latest evolution of image forgery, and they are getting more and more popular and dangerous. A deepfake is a fake image or video generated with a deep ANN, that is an artificial neural network (ANN) with many layers of neurons. ANNs are not something new, they’ve been an Artificial Intelligence (AI) tool for decades, but they have become effective for working with images only recently, due to the availability of powerful processors and GPUs.
Deepfakes may be used to create a completely synthetic face (“synthetization”), to change a person’s face with someone else’s face (so-called “face-swaps”), to change what a subject is saying (“lip-sync”), or even to change the words and movements of someone’s head so that they are like a puppet, or a guided actor (“re-enactment”). Each application requires a different AI architecture and workflow.
With this release of Amped Authenticate, we’re delivering a new filter called Face GAN Deepfake, which lets you analyze all faces in an image to determine whether they are real or synthetic faces obtained with a Generative Adversarial Network (GAN).
A GAN is a deep-learning architecture that employs two competing neural networks: a Generator network which tries to synthesize realistic content (usually faces), and a Discriminator network which is trained to distinguish real faces from synthesized ones. An iterative feedback process lets the Generator improve to the point that the Discriminator can no longer distinguish between real and forged faces. This process is visually depicted below.
Once the process is finished, the Discriminator is discarded, and the Generator can be used to create fake faces. Synthetized faces generated with modern GANs are very hard to recognize, to the point that researchers found most viewers are completely fooled by them. If you want an idea of how realistic GAN-generated faces are, look at https://thispersondoesnotexist.com (refresh the page to see more results).
All of this was about what face GAN deepfakes are; now let’s see how you can detect such deepfakes with Amped Authenticate!
The Face GAN Deepfake filter
The current state-of-the-art technologies for deepfake detection are also based on AI, specifically on deep learning. Indeed, GAN-generated images often contain most of the properties and statistics of authentic images, so detecting such forgeries based on some hand-crafted artifact analysis is very hard.
The Amped research team has thus developed and trained a deep neural network to detect GAN-generated images. To validate our detection system, we participated in an international challenge proposed by the University of Catania; our system was independently tested by the challenge organizers on a large and diverse dataset (5 different face synthesis GANs were used) and obtained an overall accuracy score of 90.05% (whereas the median score of participants was 72.62%). You can find details about the challenge and how our detection system works in the related scientific paper, which has been published by the prestigious Journal of Imaging.
And now, this powerful detection tool is available in Amped Authenticate, under the Local Analysis category!
The Face GAN Deepfake filter first localizes faces within the image and then uses AI to classify each face and provide you with a label and a confidence score between 0 and 1. When the confidence score is above 0.8, the label can either be Real (green color) or Fake (red color).
When the score is lower, Authenticate will play it safer and write Uncertain Real or Uncertain Fake. For example, if we strongly downscale, filter and compress the original image shown above, here’s what we get:
There are two ways of viewing the results, available under the Output Mode parameter: “Overlay” on the original image, or “Collage” of faces. While the “Overlay” method is visually effective for images with large faces, it may generate too small boxes and labels when there are many faces, and some of which are small.
That is where the “Collage” method is preferable.
The collage can fit up to 30 faces in a single image; if your image contains a larger number of faces, you may consider creating two cropped versions of it, save them to a lossless format, and analyze them separately.
Hint: if the resulting image looks too tiny when generating the report, raise the “Report Max Image Width” parameter either in the Program Options or in the Project Properties from the default value (600) to a higher one.
The filter has one more parameter called Face Detection, which tunes the sensitivity of the, you guessed it, face detection system. There are three options: “Medium” (default), “Sensitive”, and “Robust”. If the filter didn’t detect the face you hoped to analyze, try setting the Face Detection parameter to “Sensitive”. On the other hand, if the filter happened to analyze tiny faces in the background or it even picked up something which is not a face, then try using the “Robust” value. For example, changing the Face Detection to “Sensitive” in the previous example will pick up and analyze one more face.
Most importantly, the filter also works when a synthesized image is pasted into an original picture to replace someone’s face. For example, I started from the original image presented at the beginning of this post and used GIMP to change my colleague’s face. The process required me to scale and rotate the deepfake, match the skin color to the original subject, and work a bit on the edges.
Nevertheless, Authenticate’s Face GAN Deepfake does its job!
Before we move to the next topic, a remark must be done: as you know, Amped policy is to be very cautious with using AI for forensics. As you can read in our popular article “Can AI Be Used for Forensics and Investigations?“, we believe that currently, deep learning-based AI techniques should not be used for image enhancement for evidentiary purposes; however, they can be considered for image analysis, which is what Authenticate is mostly about. Our AI-based deepfake detector does not create new data; it classifies existing data. Moreover, we want to remark that our technique is fully disclosed in a scientific paper, and the training dataset has been made available to the scientific community.
As to the produced results, it’s important to keep in mind that the confidence score provided by the system is just a way to measure how much the network feels confident about the result; it does not guarantee anything: if a face is marked as “Real 1.00”, that doesn’t mean it is undoubtedly authentic, it simply means that the tool is unable to detect any sign of a synthesized generation.
Annotate Gets Alignment and Snapping Features
In its previous update, we introduced powerful annotation tools to Authenticate. With this release, they’re getting even better: when you have multiple annotation objects and drag one of them over the image, magenta-dashed lines will appear to help you line up the dragged object with existing ones. Lines will also allow you to align objects to the image center and borders.
There is also a snapping feature that will help you position objects without any pixel-hunting effort. You can temporarily disable snapping by holding the ALT key while dragging the object: magenta lines will still appear, but they won’t capture the object.
Major update to the Social Media Identification filter database
Nowadays, it is increasingly common for an investigation to involve imagery from Social Media Platforms (SMPs), such as Facebook, Instagram, Twitter, etc., or from messaging apps, such as Messenger and WhatsApp.
Most SMPs will strip the original metadata, downscale the image, and recompress it as part of the uploading process. Therefore, uploading to SMPs acts as a “counter-forensic” or “laundering” process when detecting manipulations. Indeed, many image forgery detection algorithms that are very effective on full-sized manipulated images become way less reliable when applied to social media images, because of the downscale-and-recompress process.
For the above reasons, a decent image authenticity analysis cannot be oblivious to whether the investigated image comes from an SMP. That’s why Amped Authenticate has featured the Social Media Identification filter for several years. The filter is based on a nearest-neighbor classifier that extracts some of the features from the analyzed image and compares them against a dataset of thousands of pictures downloaded from SMPs; when a match is found, the corresponding SMP is marked as compatible with the questioned image.
Since SMPs continuously update their processing rules, it is crucial to update the reference database. For this reason, we include a significant update in this release: we have gathered original images from iOS and Android devices and uploaded them to SMPs to include the most recent signatures in the database. Moreover, we’ve added the “Facebook Messenger” platform to the database, which was not present before.
Overall, this means that the number of entries in the database increased by more than 900 elements, and we now cover the following SMPs: Facebook, Flickr, Google+, Instagram, Messenger, Telegram, Tinypic, Tumblr, Twitter, WhatsApp.
Other Improvements and Bugfixes
Besides the main novelties above, this release contains improvements to other features.
PRNU Identification: better support for vertical images
When you use the PRNU Identification filter to create a Camera Reference Pattern, Authenticate automatically looks for the Exif Orientation metadata and applies it to the image. This metadata is stored at capture time and tells whether the image should be rotated, which is, of course, an essential element when creating a CRP, since all images must be aligned in the same way.
For this reason, until today, we were preventing creating a CRP when only vertically-oriented images were available, and no Exif metadata was found. From this release, we are allowing the creation of a CRP file even in such a case, but Authenticate will warn the user before proceeding.
VPF – Double Encoding Detection: improved performance
We improved the implementation of the Double Encoding Detection tool for videos, which will now use less memory and give the possibility to abort processing.
- Fixed a bug that caused the wrong scientific reference to be presented in the report for some filters; references in the program manual were instead correct;
- Added some missing translations.
Don’t Delay – Update Today
If you have an active support plan, you can update straight away by going into the menu Help>Check for Updates Online within Amped Authenticate. If you need to renew your SMS plan, please contact us or one of our authorized distributors. And remember that you can always manage your license and requests from the Amped Support portal.