Deepfake Forensics Workflow for Image Analysis

Reading time: 8 min

Analyzing suspected deepfakes requires more than detection alone. This post explains a forensically sound workflow for image and video analysis that helps investigators produce findings that are explainable, reproducible, and defensible.

The research community developed several tools for detecting and analyzing deepfakes under various circumstances. But where should a practitioner begin when an image is suspected of being a deepfake?

In this post, we outline a scientific workflow and the underlying reasoning required to analyze an image or video. This workflow is based on a forensically sound methodology designed not just to detect manipulation, but to produce evidence through an approach that is explainable, reproducible, and defensible. Before starting: note that this post is written for forensic practitioners who are already familiar with the fundamentals of digital imaging and core forensic approaches.

The analysis generally starts from the “naive” question: “Is it a deepfake?”
However, to a forensic expert, the technical interpretation is much broader.
For this specific talk, we will consider the detection of generated images and natural images modified through AI technologies, i.e., the image is “contaminated” in some way with AI-based editing techniques.

Key Takeaways

• Deepfake analysis should follow a forensically sound workflow, not a single-tool detection approach.
• The goal is to identify possible manipulation and to produce findings that are explainable, reproducible, and defensible.
• A robust workflow combines AI-based triage, metadata inspection, format analysis, geometric checks, and pixel-level analysis.
• AI detection alone is not evidential and should be treated as a starting point for further forensic examination.
• Scientific image and video analysis depends on methodology, validation, and consistency, especially when results may need to support an investigation or legal process.
• A structured forensic workflow helps analysts turn an initial AI flag into documented, corroborated, and defensible findings.

Watch the Podcast Episode: Beyond the Deepfake Detection Button

Want to go deeper into the reasoning behind this workflow?

In this episode of the Amped Podcast, “Deepfake Forensics: Beyond the Deepfake Detection Button”, we discuss why suspected AI-generated or AI-manipulated media cannot be assessed with a single detector alone.

The conversation explores how forensic practitioners can combine AI-based triage with metadata, format, compression, geometric, and pixel-level analysis to build explainable and defensible findings.

How to Analyze a Suspected Deepfake: A Forensically Sound Workflow

A forensically sound deepfake analysis should follow a structured process. In practice, the workflow can include these stages:

1. AI-based triage (non-evidential)
   Use AI tools as an initial screening step to flag possible signs of manipulation. These results can help prioritize analysis, but they should not be treated as evidence on their own.
2. Metadata and container inspection
   Examine file metadata, codec information, timestamps, software markers, and container structure to identify inconsistencies, processing history, or signs of re-encoding.
3. Compression analysis
   Assess whether the image has been compressed twice or exposes local compression inconsistencies.
4. Geometric consistency checks
   Evaluate whether spatial relationships in the image or video are physically plausible, including perspective, proportions, facial alignment, lighting direction, and scene geometry.
5. Image-domain forensic checks
   Analyze pixel-level and signal-level characteristics such as noise patterns, compression traces, local artifacts, resampling indicators, and other statistical anomalies.
6. Cross-validation of findings
   Compare results across methods rather than relying on a single indicator. A defensible conclusion should be supported by converging evidence from multiple forensic techniques.
7. Documentation and reproducibility
   Record each analytical step, tool, assumption, and result so that the examination can be explained, reproduced, and defended if challenged.

Why AI-based Detection Should Be Used for Triage, Not Evidence

The first single-shot weapon in our arsenal is “using AI to fight AI”. AI-based tools are excellent for triage but are not forensically reliable as standalone evidence due to limited explainability (see our related post for why).

They are particularly useful when processing large datasets: In front of a massive amount of images, the manual inspection of each piece of media can be unfeasible. The batch analysis of the data, instead, can make the selection of relevant content easier and faster. In this case, a tool that automatically flags synthetic-generation clues can be a game-changer for the analyst in terms of processing time.  

In practice, they act as an initial filter, helping analysts prioritize which content requires deeper forensic inspection.

In the example above, the Authenticate Diffusion Model Detection filter triggers a red flag! This suggests the image was likely generated by an AI tool. However, while this filter has a low false-alarm rate on photorealistic images, AI-based results alone cannot stand up in court.

Metadata Analysis in Deepfake Forensics

Another check that can lead to a “quick win” is the analysis of metadata and file format information. In certain instances, the traces of a generative system are stored directly within the textual metadata. See, for instance, the following examples:

In a few cases, when the image is generated through a text2image method, the prompt can be found in the metadata, like the following example.

“Evidence Image Value:  A bustling street market in Marrakech, Morocco, filled with colorful spices and textiles..Steps: 20, Sampler: DPM++ 2M, Schedule type: Karras, CFG scale: 5, Seed: 739955706, Size: 512×512, Model hash: 6ce0161689, Model: v1-5-pruned-emaonly, Version: v1.9.4“.

While this would be an open-and-shut case, metadata is rarely this useful. In most investigations, relevant metadata is stripped or “scrambled” by social media processing (e.g., WhatsApp, X, or Instagram).

Format Analysis: JPEG vs Non-JPEG

Then, the practitioner is required to go deeper with the analysis. The first major crossroad in the workflow is given by the image format. JPEG or Non-JPEG, that is the question.

Indeed, if the image is a JPEG, the state of the art provides an extensive arsenal of tools designed to characterize compression traces. By analyzing JPEG inconsistencies, we can determine if an image has been globally recompressed or locally manipulated. For instance, in the following example, a small AI-based “inpainting” edit is exposed through the analysis of local inconsistency in the compression level, detectable via Authenticate ADJPEG filter.

Although the technique cannot exactly determine how the red portion has been modified, it can definitely prove that something is wrong with the image. Furthermore, an inconsistency at the JPEG level is strongly explainable and has a solid scientific foundation. You can definitely go into the courtroom with these results.

Note, furthermore, that in this specific case, the image is hardly recognized as AI-generated because a very small part is modified! The JPEG analysis made the difference.

In the case we saw at the beginning, unfortunately, the image we are analyzing is in PNG format, and no relevant metadata or discriminative format characteristics are available.
Is there something else we can do?

Using Geometric Analysis to Detect Inconsistencies

Luckily for us, the image includes reflections! Mathematical models are clearly defined and can be used to assess if the reflections are consistent within a real image¹.

Authenticate’s Reflections Filter can do the job and the results are clear: reflections are inconsistent (the details on how the reflection analysis works can be found here).

This geometric inconsistency verification cannot characterize the generation/manipulation process. However, it is a forensically sound way to prove that the image is unreliable. The method is also clearly explainable, making it valuable for expert testimony.

Depending on the image content, various geometric-based features can be checked in the image, like shadows and perspective. These features are very robust to compression and should be strongly considered when the image has a limited resolution or is strongly compressed.

Note that even when geometric features cannot be used, the analysis of the inconsistencies at a local level can be performed through a series of algorithms developed by Image Forensics researchers. Authenticate implements the most relevant to expose unexpected correlation patterns left by an AI-based local manipulation.

For instance, in the following example, the error analysis of DCT coefficients highlights traces of inpainting: in the central part of the image, Authenticate’s Correlation Map filter highlights a noise pattern that is completely uncorrelated with the content.

In summary, a proper workflow for deepfake detection should include the combined analysis of different image domains (metadata, container, codec, pixel statistics, geometric properties). This is because generation and manipulation processes can leave a number of forensic traces in various image domains.

In the considered example of the “reflected girl”, for instance, the combination of deepfake detection and reflection filters proves that the image is unreliable (with an explainable methodology) and it is likely to be generated with an AI-based method.

Conclusion

The above analysis highlights that answering the simple question “is it fake?” requires much more than a single click on a deepfake detector. Investigation skills and deep knowledge of image encoding and compression are needed to fully examine a piece of media under investigation.

---

FAQ: Deepfake Forensics Workflow

---

1. Bianchi, Stefano, et al. “Assessing shadows and reflections consistency in AI-generated images.” 2024 IEEE International Workshop on Information Forensics and Security (WIFS). IEEE, 2024. ↩︎