In the past years, policymakers dedicated a lot of attention to cybersecurity, artificial intelligence, and privacy, especially in relation to video surveillance and public safety. While these are very important topics, I think we need to also dedicate some attention to the more fundamental issues that often go unnoticed and relate to the proper understanding of image and video evidence, both during investigations and in court.
Introduction
In the past months, I’ve dedicated a good portion of my time to raising public awareness about video evidence’s challenges. As we said quite a few times “you don’t know what you don’t know”. The very first step in addressing a problem is realizing that you have one.
It’s widely recognized, and corroborated by scientific studies, that images and videos are the most effective sources of evidence. Yet, unfortunately, they are not always treated with the same rigor and scientific approach that is applied to other forensic fields.
This dichotomy is sadly present even in the pop culture. In every investigative fiction à la CSI, DNA, toxicology, and fingerprints forensic experts come to the crime scene or in the lab, with extreme professionalism and a white coat. On the other end, when there’s a video to analyze, a random investigator screams at the screen “zoom on that!”. This is very indicative of the public perception of video evidence and how its complexities and challenges are underrated.
“With great power comes great responsibility”. Given the power and amount of video evidence, the risk of letting a criminal free or destroying the life of an innocent person being accused on the basis of a misleading analysis is higher than with any other kind of evidence. For this reason, it’s of utmost importance to treat it very seriously.
What are the issues of working with video evidence?
1 – Video evidence seems easy, but it’s actually easy to get it wrong
Anybody can hit “Play” or download a photo on their PC, and try to interpret it. These very low entry barriers imply that those who have to deal with video evidence may not have the right technical preparation or the right tools. Why? Everybody can “watch” a photo.
We dedicated a full series of articles to the various pitfalls that can happen if you are not aware of the technical limitations of video evidence and the bias of the human visual system in their interpretation. I will repeat just two examples here because they are probably the most common and dangerous.
The first is related to infrared imaging.
Infrared Imaging
Most video surveillance cameras, in low light conditions, use an infrared illuminator. In this case, the values of the pixels that represent an object in a video file don’t have any relation with its real color. For example, the fact that a shirt seems “bright” in an infrared image, doesn’t imply that it’s actually of a bright color (it could even be black). It just indicates how its fabric reflects the infrared. If this fact is not known, this can easily cause identification errors and mislead an entire investigation. Infrared and visible light are two completely different signals that can’t be compared, at least for what regards their value. If we work with the awareness that infrared imaging has this limit, our judgment won’t be biased by a misleading representation.
In the example below, you can see the same shirt taken by the same camera, in the same position, with both visible light and infrared. It can be noticed that they seem to be two completely different clothes. Just imagine how not knowing this can lead the investigation in the wrong direction and cause a wrong identification!
Digital Compression
Another issue is related to the impact of digital compression.
Compression is applied to most images and essentially all videos in digital format. Without compression, the storage space on our phones would allow filming a few minutes at most. After that, the memory would be full. Furthermore, sharing videos on the internet would be practically impossible. What does compression do? It reduces the amount of data, exploiting redundancies and similarities inside a video frame and consecutive frames. It also removes less perceptually relevant details from the video. If we speak about photos of our vacations, this is useful and sensible. However, when speaking about the forensic context, this process may remove small but very important details and even create new fake details (called “artifacts”).
With the proper technical preparation, it’s possible to understand which parts of the image to rely upon and which ones we can’t trust. This will minimize the bias that an investigator could have, for example, when trying to interpret the characters of a license plate in a low-quality image.
Furthermore, we should always remember that the fact that we don’t see something in a video, doesn’t mean that it didn’t happen. Frame rate, resolution, compression, and other factors heavily affect its interpretation. During a trial, it’s not uncommon to discuss the reliability of a block composed of a few pixels.
In the image below, it’s very difficult to understand what is that detail on the side of the car. It seems like it is a scratch, but it is actually an artifact of the compression.
Another very effective example is shown by a video of Putin that went viral on the internet. The public thought that during his speech he used a green screen and faked his appearance on TV as it seemed that his hand passed through the microphone. Many thought the video was the result of bad editing. However, from a higher quality video, it was clear that it was a compression artifact.
2 – Proprietary video formats complicate investigations and introduce cybersecurity risks
The vast majority of files coming from video surveillance systems are in a proprietary format. This means that usually, they are not normal MP4 or AVI files that can be played with standard software, such as Windows Media Player or VLC. However, they need the player created by the system producer. Very often, these players are of low quality, have significant issues, and are completely inadequate for a forensic analysis of the video. We have already written about these issues many times.
In order to perform video analysis, proprietary files need to be converted. This is another critical phase. A conversion done without the right care can very easily introduce these issues:
- The loss of file originality, with a clear impact on the chain of custody that we must ensure for every kind of evidence
- The loss of quality: if the video is transcoded, this may cause a noticeable quality loss
- The loss of metadata and information about the original encoding, such as timestamp and the kind of compression applied on different parts of the image; this information could have been used to evaluate the reliability of the visual information
Furthermore, the search and use of proprietary players can cause many issues from the IT and cybersecurity points of view. To cope with these issues we have implemented support for proprietary video formats in our tools Amped DVRConv, Amped Replay, and Amped FIVE.
3 – Image quality is often low: it must be processed in the right way
Video evidence often doesn’t have sufficient quality to obtain the needed information, for example, to identify a person or a vehicle. However, using the right tools and techniques, if there are the right technical conditions, it is possible to get excellent results.
The processing must be done in a rigorous way, using methodologies applicable for the forensic settings and scientifically validated, otherwise, the evidence could be disqualified during the trial. According to research carried out with our users, video enhancement can produce useful results in about 50% of the cases.
As an example, you can see below a video of a vehicle forensically enhanced with Amped FIVE.
4 – We can’t always trust video evidence: deepfakes, manipulation, and misattribution
Authenticating images and videos is becoming of utmost importance. Lately, deepfakes (image manipulation with artificial intelligence techniques) are everywhere in the news. But, in actual cases, it’s actually much more common to see very simple fakes.
It’s not even needed to modify an image to use it in a malicious way. Let’s just think about the reuse of a photo done at another time or the staging of an event used as evidence. All this is much easier to do than a deepfake or even a manipulation with Photoshop. We need to think more proactively and consider image and video manipulation as a real possibility, and thus to take it into account when we start to analyze video evidence. Our tool Amped Authenticate can help with this.
In the example below, an app has been used to replace the face of Leonardo DiCaprio with another subject.
Unfortunately, we have recently seen much more serious uses of this technology, like the deepfake video of Zelensky, which has been very quickly debunked.
If you want to learn more about deepfakes, check this video out.
5 – Forensic video analysis: interpreting in the right way without bias
A critical part of video evidence analysis is determined by human factors. One of the most pressing issues is to limit human bias during the interpretation. Again, even just being aware of the issue is a good start. It is very important to give investigators or analysts only the strictly necessary technical information to perform their job and to apply bias mitigation techniques.
It is very easy to get “too involved” in a case and instinctively look for confirmation for our beliefs and opinions. This can, and often does happen, because we are humans. Even without any malicious or explicit intent, our mind is used to exploit biases as a shortcut to difficult situations.
If you want to learn more on this topic, you can read this article from our blog, and a manual from the UK government: “Cognitive bias effects relevant to forensic science examinations”.
6 – Judges and juries are easy to mislead on technical topics
It’s often difficult for judges, juries, and the public to evaluate the technical ability and reliability of the various sides’ expert witnesses. Let’s say that one side does a correct and very cautious analysis. The other side is less careful with respect to the various forensic procedures. But, gives a much stronger speech, maybe with an opposite outcome of the analysis. Often this is enough to cast doubts on the previous analysis, even if that was the correct one. Without a minimum of scientific and technical education and awareness, this is a very difficult issue to overcome.
Often, the ability to communicate is much more important than the actual correctness of the analysis.
7 – Impact of Artificial Intelligence, Privacy, and Cybersecurity
The title of this paragraph is a bit buzzwordy (it seems that just the terms “blockchain” and “metaverse” are missing 😉). I started this article from these points, telling that they are very much under the radar but we need to go one step beyond them.
They are not specific to video evidence and forensics. Nonetheless, that should be taken into account in this context as well.
I put all these aspects in a single paragraph because it’s important to mention them. However, going into further detail is out of the scope of this post.
Using artificial intelligence to work on video evidence should be handled with care. I already mentioned my position on various aspects in this previous blog post. While expressed in different terms, it has many points in common with the Artificial Intelligence Act from the European Union.
For what regards privacy, we must remember that there’s a natural tension between the need to acquire as much data as possible during investigations, their forensic use, and privacy considerations. Nevertheless, working within the boundaries of privacy regulations is a very important aspect that should be taken into account even by law enforcement. Worldwide, there’s growing adoption of stricter privacy frameworks that typically follow the steps of the European GDPR. Working in, or for, law enforcement doesn’t exempt you from such regulations. Quite the opposite, actually.
Finally, the cybersecurity aspect is very important. Forensics and investigation are by nature sensitive industries, where data and security must be taken very seriously. Software and systems should be safe, as much as possible. As an example, many of our users work on video evidence on air-gapped computers, completely disconnected from the internet. I already mentioned the issue with proprietary players, but that is just the tip of the iceberg.
How to work correctly on image and video evidence?
The above-mentioned issues could be solved. The overarching idea is that we need to create awareness and culture on this topic at 360 degrees. Starting from police officers to forensic analysts, judges, prosecutors, attorneys, up to politicians and the general public. These issues should be solved both down and up the chain of command.
Once digested that video evidence should not be taken for granted, but should be handled scientifically and with care. We would need to create awareness programs for all the parts of the judicial system. It’s not needed, nor recommended, that all police officers, attorneys, and judges became forensics experts. However, it’s essential to create a cultural basis where everybody understands the importance and sensitivity of video.
What are the main points?
1 – Awareness and education
Educating the users and the public on the pitfalls and complexities of video evidence. Expose people involved in the field to some form of short education program (a few hours) to open their eyes to the concepts that are currently unknown to most people.
2 – Originality and acquisition
Maintaining the chain of custody to guarantee access to the original evidence that has been acquired. All the operators should understand the importance of a correct acquisition. Too often we see surveillance video footage wrongly captured filming a screen with a smartphone. Without going to these extremes, even the simple exporting from a surveillance system or a DVR should be done in a way that preserves the chain of custody. This would grant the best possible quality and reduce the risk of the evidence being rejected in court.
3 – Authentication and verification
Verifying the integrity of the file (is it original and unaltered?) and how much we can trust its content authenticity. Furthermore, this process should be able to verify that the acquisition has been done properly.
4 – Correct and transparent processing
Adoption of clear guidelines for processing, using algorithms validated by the scientific community, and adopting a workflow that is correct, repeatable, and reproducible. There are several guidelines of excellent quality available, such as those from ENFSI, SWGDE, OSAC, and the UK FSR. They should be more widely followed and adopted.
What can we obtain by working properly with images and videos?
1 – Better security
Working correctly would allow us to streamline the processing workflow instead of reinventing the wheel every time. While it’s true that every case is different, in the end, the underlying issues and objectives of an investigation are often very similar. Most of the time, the purpose of the analysis is identifying a person, a vehicle, or understanding the dynamics of an event.
Working correctly would allow us to solve more crimes, and thus improve the security of our cities and countries.
2 – Better justice
Using original data, assessing their authenticity and integrity, and processing them in the right way, we can grant a more fair trial and reduce judicial errors.
3 – More speed, fewer costs
Doing things the right way doesn’t take more time since it actually involves a more efficient workflow. This means less time spent by human resources and fewer issues caused by mismanaged evidence.
Conclusion
As you can see, there are only advantages to working on video evidence in the right way. Only one thing is needed, a change of mindset. Video evidence is not just “a video”, it’s “evidence”, and should be treated as such.
