Category Archives: How to

Using Enhanced Images in Court

I recently testified in court as a forensic image and video expert and, as is sometimes the case, the use of some filters to enhance images was questioned. As I have written before, there is some processing that should be entirely avoided, since it lacks accuracy and repeatability. For example, we should avoid techniques which add new information relying on data obtained by a training set, or techniques which have a random component.

Some years ago, there was a school of thought that said, only classical image processing techniques available for the analog photography can be applied to digital photography in the forensic context. What are the risks of applying the wrong processing? We are not interested in having a “pleasant” image, we are concerned about extracting information from it. The risks of wrong processing are:

  • Removing existing information: for example, removing the grain in a dark image can remove also important details.
  • Adding new information: for example, creating or amplifying image artifacts which may be misinterpreted as a real detail.

In this reasoning, we are not referring to details at the pixel level, but at the image semantic content. In general, if I resize an image, I add a lot of new pixels but if the processing is correct I am not adding any new relevant information.

It’s important to understand that most of the image processing techniques present a compromise: I enhance something at the expense of damaging something else. For example, if I lighten an image to show better a dark part, it’s very likely to lose details in the parts of the image that are already bright enough.

For this reason, it’s very difficult, in general, to say which techniques are good and which techniques are bad. Their applicability must be related to the specific case and the parameters used. Filters are just tools, and as such, they can be used in the right way, obtaining better images, or in the wrong way, damaging the image quality or presenting wrong information.

Because of this, it’s important not to blindly apply different enhancement and restoration filters, but to apply them in order to correct a specific defect. Similarly, the tuning of their parameters must be consistent with the amount of defect I want to correct. Abusing the filters can create images which are much worse than the original.

It is therefore important, as I’ve said many times, to work with experts who have specific experience in the forensic image and video analysis field. Who know what to do, and how to identify what has been done incorrectly.

A lot of pressure may be put on the processing done by the experts, but most people ignore that there are many other processing and possible issues happening during the image acquisition and visualization phases.

A lot of processing happens in the camera itself, from CCTV to smartphones. Unless raw image pictures are used, and this is very rare, the value of the pixels in an image are hugely dependent on the processing and encoding which automatically happens inside the device to obtain the ratio between image quality and technical limitations that the producer wished to obtain.

And then, even to simply visualize the image, there’s a lot going on under the hood. Different software can decode the image in a slightly different way which can enormously impact the final result, and a lot of image processing happens on the graphics card of the PC, on the screen, or on a projector. Just play with the brightness of the projector to realize how much the visible information in an image can be impacted by such simple tuning.

There is then the most critical part of the processing: our eyes and our brain. Different people see and want to see different things in the same image. Analyzing things in an objective and unbiased way is often very difficult unless you can measure things numerically. And in fact, avoiding and limiting the various types of biases are one of the most important aspects of forensic science currently studied.

This article, written by Martino Jerian, was originally published in Lawyer Monthly magazine. Click here for the published article. 

Amped Authenticate Update 11362: JPEG Dimples, Improved JPEG HT, Social Media Identification, and much more!

Not long has passed since the release of Amped Authenticate 10641 but… yes, the next one is already out! Amped Authenticate 11362 is now released with a lot of improvements, including two new filters based on JPEG Dimples, one of the last discoveries of the image forensics scientific community!

JPEG Dimples

Despite many attempts to send JPEG into retirement, today the vast majority of digital images still use it. Amped Authenticate users know that traces left by JPEG compression are a superb asset when it comes to investigating the digital history of an image, as witnessed by the vast JPEG-based toolkit that Authenticate provides: quantization table analysis, JPEG ghosts, inconsistencies in blocking artifacts, double quantization traces in the DCT coefficients, and more.

But JPEG is still full of new surprises nowadays! A few months ago, while Amped was attending (and sponsoring!) the IEEE 2017 International Workshop on Information Forensics and Security (WIFS 2017), a new footprint was presented to the scientific community: JPEG Dimples (click here to see the original work Photo forensics from JPEG dimples by Shruti Agarwal and Prof. Hany Farid).

JPEG Dimples manifest themselves as a grid of slightly brighter/darker pixels, spaced by 8 pixels in each dimension. Like most image forensic fingerprints, even JPEG Dimples are hardly visible by the human eye, but they can be easily detected with a proper algorithm.

But why does this grid appear? And why is it important for our analysis? We’ll answer these questions in detail in a future blog post, however the reason behind JPEG Dimples is rather simple: during the DCT coefficients quantization phase, different operators exist to approximate decimal values to integer values: the round operator (which approximates the decimal number to the nearest integer) the floor operator (approximation to the nearest smaller integer) or the ceil operator (approximation to the nearest bigger integer). The table below shows the difference in approximating a Value (first column) to an integer using round, floor and ceil.

Value Round Floor Ceil
9.8 10 9 10
6.3 6 6 7
4.5 5 4 5
-7.3 -7 -8 -7

Obviously, using floor tends to produce smaller values in the 8-by-8 DCT block than using round, and the opposite with ceil. And when we go back to the pixel domain, this leads to a slightly darker or brighter pixel on the top-left corner of the pixel block (see example below)! Measuring the presence of this grid will tell us to which degree an image contains the JPEG Dimples footprint.

Image showing Dimples

Example of an image showing strong JPEG Dimples

Now you may be wondering “well, how many cameras will ever be using floor or ceil in place of the more classical round?” Not so few, actually. According to the work presented at WIFS 2017, more than 60% of tested cameras do introduce Dimples. We also carried out an internal evaluation on Amped datasets and numbers were less upsetting, still, we found Dimples in roughly 30% of tested cameras. A footprint with such a spread could not be missing in Amped Authenticate, and so here we are. Continue reading

Extracting Channels

If you’ve attended one of my classes or lectures, you’ve likely heard me say the following phrase many times, “There’s what you know, and there’s what you can prove.” The essence of this statement forms the basis of the Criminal Justice system as well as science.

What I “know” is subject to bias. What I “know” is found in the realm of truth. As a Kansas City Chiefs supporter, I “know” that the Oakland Raiders are a horrible team. I “know” that their fans are the worst in the world. After all, the Chiefs are the best and their fans are as pure as the wind-driven snow. This is “true” to me. Whilst funny and used to illustrate a point (I’m sure there are some really great people among the Raiders fan base), truths are things we “know.” Truths are rooted deep in feelings/emotions and unlikely to be changed by facts. There is a segment of the US population that believes it true that Elvis is still alive and that he’s likely hanging out on some Caribbean island with Tupac and Biggy Smalls.

Facts are measurable; they form the basis of tests of reliability. I can measure the temperature in a specific location and you, standing in the same location, can perform the same test and come to the same measurement. Supported by facts, our tests in this discipline become reliable, repeatable, and reproducible. Our conclusions can thus be trusted.

What on earth does this all have to do with Amped FIVE and Forensic Multimedia Analysis? I’m glad you asked.

By now, you’re well familiar with the fact that Amped Software operationalizes tools out of image science, math, statistics, etc. We also operationalize tools and training out of the world of psychology. By this I mean if we’re going to work in the visual world, we must know how that visual world operates not only from a mechanical standpoint but also from how the brain processes the inputs from its collection devices.

Continue reading

Amped FIVE Update 11284: Multiplexed Stream Support, Proprietary Timestamp, Remove Frames Filter, and a Whole Lot More

Whilst it’s been a busy time for us here at Amped with the demand for training higher than ever, we have made sure our development is continuous and we’re here again with another huge update for Amped FIVE.

A Completely Revamped Conversion Engine

As you will know, one of the biggest struggles within the world of CCTV and video analysis is the ever-increasing number of proprietary formats. Our support and development team are constantly receiving requests for new format support and in our latest update, we have enabled conversion support for BVR, DVS, H64, PSF and SHV formats, along with some variations of other formats already supported in previous versions.

All these formats are multiplexed streams. This is when a manufacturer has placed all camera footage into a single time-based video stream.

The latest FIVE not only converts the files straight away, but demultiplexes each video stream, splitting them into their own individual chains within the software. Under the Convert DVR Advanced tab you will find the options to enable this time-saving function.

Files to Convert > All, one chain per file.

No more mixed streams, no more time wasted writing carving scripts. A few clicks will now save you hours!

Multiplexed single stream decoding is huge, so expect a dedicated blog post in the next few weeks looking more deeply into decoding files of this type.

But the new conversion engine does not stop there! There are a lot of benefits even on single stream video files. Standard conversion done with vanilla FFmpeg is often not enough – there may be the risk of losing video frames because of wrongly interpreted proprietary metadata. Our new engine not only cleans almost every proprietary video format, being in MPEG4, H263, H264 and H265, but for many of them also recovers the proprietary timestamp. We found more than 50 different variations of timestamp formats!

Continue reading

Video Redaction with Amped FIVE

First of all, let me introduce myself. I’m Lucy Carey-Shields, the newest member of the Amped team! Originally from the UK I studied Computer Forensics at degree level and was a volunteer police officer with a UK police force for six years. I later went on to work for another UK police force for almost four years as a digital forensics technician, mostly working with CCTV and video whilst also providing forensic acquisition of mobile devices. Whilst working at Amped I’ll be providing support as well as putting the software through its paces, so I look forward to hearing from you all! Now let’s dive into my first Amped blog post! 

When dealing with video, we often have to hide sensitive information or protect a person’s identity, particularly if the video is to be shared with a wider audience and we need to control the display of certain information. Amped FIVE has a filter for that!

Having used two or more different tools to load, process and then redact sensitive footage in the past, I know how time-saving having all these features in one piece of software can be (and how critical time can be in a law enforcement environment).

The Hide Selection filter allows you to pixelate, blur or blacken anything you want masked in a video quickly. In this instance, we’ll explore both dynamic tracking and manual tracking during the use of Hide Selection. Hide Selection can be found under the Presentation group of filters, typically used at the end of a workflow.

Redaction, whilst usually done towards the end of processing a video, is arguably one of the more critical steps in a workflow as revealing sensitive data or someone’s identity could have serious and potentially dangerous consequences. With this in mind, it’s important we ensure frame by frame accuracy so that the subjects we want to censor are completely disguised. FIVE allows you to apply the filter by selecting the necessary points – maintaining that important frame by frame accuracy. Continue reading

Using Snapshots in your Project

The ability to save a frame as a “Snapshot” has been a feature in Amped FIVE for quite some time. A simplified explanation of the use of Snapshots in interacting with third-party programs can be found here.

Today, I want to expand a bit on the use of Snapshots in your processing of video files.

There are often times that users have been asked to produce a BOLO flyer of multiple subjects and problems with the video file complicate the fulfillment of the request.

  • The subjects aren’t looking towards the camera at the same time / within the same frame.
  • There’s only one good frame of video to work with and you need to crop out multiple subjects.

Enter the Snapshot tool.

The Snapshot tool, on the Player Panel, saves the snapshot of the currently displayed image (frame) and its relative project.

When you Right Click on the button, a menu pops up.

The post linked above talks about working with the listed third-party tools. In this case, we’ll save the frame out, selecting a file type and manually enter an appropriate file name.

We can choose from a variety of file types. In most cases, analysts will choose a lossless format like TIFF.

The results, saved to the working folder, are the frame of video as a TIFF and its relative project file (.afp).

Working in this way, analysts can quickly and easily work with frames of interest separate from the video file. The same frame can be added to the project several times, repeated as necessary (in the case of cropping multiple subjects and objects from the same frame).

Amped FIVE is an amazingly flexible tool. The Snapshot tool, found in the Player Panel, provides yet another way to move frames of interest out of your project as files, or out to a third-party tool.

If you’d like more information about our tools and training options, contact us today.

Working Scientifically?

On Tuesday, May 22, I will be in Providence (RI, USA) at the Annual IACP Technology Conference to present a lecture. The topic, “Proprietary Video Files— The Science of Processing the Digital Crime Scene” is rather timely. Many years ago,  the US Federal Government responded to the NAS Report with the creation of the Organization of Scientific Area Committees for Forensic Science (OSAC). I happen to be a founding member of that group and currently serve as the Video Task Group chairperson within the Video / Imaging Technology and Analysis Subcommittee (VITAL). If one was to attempt to distill the reason for the creation of the OSAC and its on-going mission, it would be this: we were horrible at science, let’s fix that.

Since the founding of the OSAC, each Subcommittee has been busy collecting guidelines and best practices documents, refining them, and moving them to a “standards publishing body.” For Forensic Multimedia Analysis, that standards publishing body is the ASTM. The difference between a guideline / best practice and a standard is that the former tend towards generic helpful hints whilst the latter are specific and enforceable must do’s. In an accredited laboratory, if there is a standard practice for your discipline you must follow it. In your testimonial experience, you may be asked about the existence of standards and if your work conforms to them. As an example, in section 4 of ASTM 2825-12, it notes the requirement that your reporting of your work should act as a sort of recipe such that another analyst can reproduce your work. Whether used as bench notes, or included within your formal report, the reporting in Amped FIVE fully complies with this guidance. There is a standard out there, and we follow it.

Continue reading

What’s the Difference?

It was a slow week on one of the most active mailing lists in our field. Then, Friday came along and a list member asked the following question:

If I exported two copies of the same frame from some digital video as stills. Then slightly changed one. Something as small as changing one pixel by a single RBG value….so it is technically different…

… Does anyone know any software that could look at both images and then produce a third image that is designed to highlight the differences? In this case it would be one pixel …

To which, my colleague in the UK (Spready) quickly replied – Amped FIVE’s Video Mixer set to Absolute Difference. Ding! Ding! Ding! We have the winning answer! Let’s take a look at how to set up the examination, as well as what the results look like.

I’ve loaded an image into Amped FIVE twice. In the second instance of the file within the project, I’ve made a small local adjustment with the Levels filter. You can see the results of the adjustment in the above image.

With the images loaded and one of them adjusted, the Video Mixer, found in the Link filter group, is used to facilitate the difference examination.

On the Inputs tab of the Video Mixer’s Filter Settings, the First Input is set to the original image. The Second Input is set to the modified image, pointing to the Levels adjustment.

On the Blend tab of the Video Mixer’s Filter Settings, set the Mode to Absolute Difference.

Continue reading

Identify Social Media Files with Amped Authenticate

Amped Authenticate Update 10641 introduced the new Social Media Identification filter. It can be found in the File Analysis filter group.

The filters in the File Analysis group are generally looking at the file’s container to return relevant information about the file. The Social Media Identification filter examines the file for traces of information that may indicate the file’s social media source. The key word here is “may.”

The workflow that I will explain here is typical in the US and Canada. Take from it what you need in order to apply it to your country’s legal system.

Let’s begin.

Continue reading

Amped Authenticate & Griffeye Analyze DI Pro: a synergy that empowers forensic analysts!

The partnership between Amped Software and Griffeye keeps growing and so does the integration between Griffeye Analyze DI Pro and Amped Authenticate. Analyze DI Pro is a media investigation software for handling large volumes of images and videos, filter irrelevant digital files, prioritize, correlate and identify the most pertinent material in investigations. It will let you scan and import data from a device or from a folder on your workstation. Once the import is complete, you can easily browse and intelligently sort/filter media.

In this post, we’ll take a look at what Griffeye Analyze DI Pro enables you to do when linked with the Amped Authenticate plugins. Let’s create a case and import a folder containing a few JPEG files.

Analyze DI Pro lets you look at image metadata, and Amped Authenticate users know how interesting they are, but, we also know that a single image may contain hundreds of Exif metadata, and reading all of them is quite a boring job. Luckily, from the very same panel above we can call in Amped Authenticate File Format Analysis to automatically spot suspicious metadata. Once you installed Authenticate and the corresponding plugin in Analyze DI Pro, this is just as simple as right-clicking on one or all the images and then hit the “Plugin” voice and select “Amped Authenticate – File Format Analysis” from the pop-up list as shown below.

Continue reading