Hands-off the keyboard!

I’ve had a few questions about our tool’s reporting feature, so I thought a blog post would help explain and illustrate the philosophy behind our report creation process. Here it goes.

To understand why we format our reports in the way that we do, you must first understand the legal and regulatory environment in which forensic analysis exists in much of the world. We don’t just create tools in a vacuum. We didn’t make a tool for another industry and repurpose it for forensic science. Amped FIVE is purpose built for the forensic analysis of video and images.

Thus, we’ll start our tour at the ASTM. ASTM’s E2825-12 is at the heart of why our reports are formatted as they are.

More specifically, in Section 4 of E2825-12 it notes the following:

4.2.1 – Processing steps are documented in a manner sufficient to permit a comparably trained person to understand the steps taken, the techniques used, and …

Amped FIVE’s reports are created to satisfy this guidance – every time, automatically.

If you are using a tool that does not have a reporting feature, how do you satisfy this guidance in your reports? Do you type it out manually? Do you modify a history log? How much time and attention to detail does this take? From my own work, I realized a 5x increase in productivity by not having to collect all this information and transfer it to the report. This productivity increase means more cases processed – and processed accurately. If you’re in private practice, how would you like a 5x increase in billable hours?

Recalling my work in assisting the LA DA’s Office in People v Abdullah ((BA353334) Los Angeles County Superior Court, December 2009), the opposing counsel’s expert did not produce what could be considered an accurate or complete report. Thus, it was necessary to have him testify to his work. He was not able to articulate what was done or how his tools worked (in this case, Premiere Pro). He incorrectly described the resizing functionality of the tool. When asked by the DDA for clarification, he declared, “while, I’m no expert.” At this point, the court was astonished. He was immediately excused by the Judge and the case was on hold whilst the attorney secured another expert.

That is a worst case scenario about what can happen when you don’t produce a report that meets the E2825 guidance.

This is the why – why Amped Software reports are formatted in the way that they are. Let’s look within the report to break down why we include the elements that we do for the individual tools / filters / tests.

  • The first line lists the filter / tool / test used. In this case, Levels.
  • The next line gives a “plain English” explanation of what the filter / tool / test does.
  • The Details section is given to provide a more detailed explanation of what is being done by the filter / tool / test.

These are given to satisfy Marx / Frye / Daubert / FRE – generally, that the “expert” should be able to explain the technique employed with sufficient simplicity and clarity without resorting to technical jargon. We don’t leave it up to you to figure out how to explain it – we provide this “plain English” for you.

The next portion deals with the Parameters – these are the numbers that are provided so that you can repeat your work (test / re-test accuracy) or that opposing counsel’s expert can reproduce what you’ve done. Simply take the parameters and plug them in – you should get the same result (repeatability). Another similarly trained / equipped person plugs in your numbers and gets the same result (reproducibility). If a report does not contain a tool’s parameters / values, how can the test or method be repeated / reproduced? It can’t. Thus, it doesn’t satisfy the guidance in E2825.

Finally, we provide the source of the algorithm. We do this for a few very important reasons.

Daubert asks of a technique: “(1) whether the theory or technique in question can be and has been tested; (2) whether it has been subjected to peer review and publication; (3) its known or potential error rate; (4) the existence and maintenance of standards controlling its operation; and (5) whether it has attracted widespread acceptance within a relevant scientific community.” All of this is information in the paper / book cited in the References. The source may also note limitations – things that are not an appropriate use of the technique. As an example, is it appropriate to use simple planar geometry or the Rule of Thirds to measure a 3D object in the 2D space of video / images?

We only operationalize tools / filters / tests that have been peer-reviewed and have published sources – rock solid science. In the case of our use of Levels, we’re operationalizing the work of Dr. Jain – which can be found on pg. 234-241 of his book. If you were inclined to test our implementation of Levels – you can get the algorithm from this source and input the values reported in the Parameters section. For contrast / brightness type work, we’ve operationalized algorithms like Retinex and Contrast Limited Adaptive Histogram Equalization.

A similar algorithm drives Photoshop’s Shadows / Highlights. The difference is that Adobe does not tell you how their algorithm works, it’s source. I can operationalize Retinex in MATLAB if necessary. No such guidance exists for operationalization of Adobe’s Shadows / Highlights. It’s proprietary and patented. It’s also important to note that Photoshop is made for photographers – not forensic image / video analysts. It’s not purpose built for science – it’s a wonderful art tool.

Some worry that they are responsible for knowing about the papers / books listed in the References section and that they may be questioned on them in detail. That may happen. It also may happen that you are questioned about the operation of the transfer case or automatic transmission of the vehicle that you drove to the crime scene. You don’t need to know how these items work in order to drive your vehicle. Your attorney will know why the References are listed and should object accordingly – noting that the References are there to satisfy the relevant standards. I will say that I’ve been asked some quite irrelevant questions during testimony. They can ask anything they want – so I don’t worry about it much or take it personally.

The other big reason that our report is formatted the way that it is has to do with Human Factors. There is a human tendency to engage in a sort of cognitive bias, the logical fallacy of appeal to authority. This fallacy entails the citing of degrees and/or certificates as the source of the authority for testimony. It’s like saying, “I hold a math degree, therefore you must believe me.” WRONG. The math either works or it doesn’t. Your results can either be reproduced or they can’t. It’s not about you, it’s about the experiment / tests and the evidence.

Your CV is there to let the court know of your qualifications and nothing more. It’s not a golden ticket that magically makes you believable. Once you make the case about you, you become the focus – not the evidence. Once you open up that line of questioning, it’s not a fun experience.

To sum it up, I realize that there are a lot of tools out there. Regardless of the tool you use, there are rules to this work. These rules are found at the ASTM, the state / federal rules of evidence, and the court’s precedents (Frye / Daubert). If you do this work, you’re responsible for them – just as you’re responsible for the entire vehicle code when you drive your car.

If you’re not using Amped FIVE, how do you formulate your report to satisfy these demands? How long does it take? Do you know how the tools work, their accuracy, and reliability? Have they been the subject of actual peer-review? If you’re creating your reports manually, what happens if you make a mistake?

Why worry about all this? We’ve got you covered with a well thought out and implemented toolset and reporting feature.

If you’d like more information about our products or training sessions, contact us today.