Working Scientifically?

On Tuesday, May 22, I will be in Providence (RI, USA) at the Annual IACP Technology Conference to present a lecture. The topic, “Proprietary Video Files— The Science of Processing the Digital Crime Scene” is rather timely. Many years ago,  the US Federal Government responded to the NAS Report with the creation of the Organization of Scientific Area Committees for Forensic Science (OSAC). I happen to be a founding member of that group and currently serve as the Video Task Group chairperson within the Video / Imaging Technology and Analysis Subcommittee (VITAL). If one was to attempt to distill the reason for the creation of the OSAC and its on-going mission, it would be this: we were horrible at science, let’s fix that.

Since the founding of the OSAC, each Subcommittee has been busy collecting guidelines and best practices documents, refining them, and moving them to a “standards publishing body.” For Forensic Multimedia Analysis, that standards publishing body is the ASTM. The difference between a guideline / best practice and a standard is that the former tend towards generic helpful hints whilst the latter are specific and enforceable must do’s. In an accredited laboratory, if there is a standard practice for your discipline you must follow it. In your testimonial experience, you may be asked about the existence of standards and if your work conforms to them. As an example, in section 4 of ASTM 2825-12, it notes the requirement that your reporting of your work should act as a sort of recipe such that another analyst can reproduce your work. Whether used as bench notes, or included within your formal report, the reporting in Amped FIVE fully complies with this guidance. There is a standard out there, and we follow it.

Looking forward to my lecture, I will present a few scenarios where the common practices from the world of science are often omitted from the reports of the processing of the digital crime scene and the evidence collected / processed therefrom. Once such scenario is the noting of any limitations in the processing of the evidence.

Every scientific enquiry is limited in some way. A forensic science enquiry is no different. This is why it does not seem reasonable to use the words “prove” and “disprove” with respect to one’s findings. It is always possible that future research / development may cast doubt on the
validity of any hypothesis or the conclusions from an analysis.

As an example, I can recall a time when mobile device acquisition and analysis tools were all made abroad and geared to a technology that wasn’t utilized in my area. Companies based their technology on the GSM standard, whilst California was using CDMA. The results of our acquisition and analysis of mobile devices were severely limited by the available technology as GSM and CDMA phone vendors had different ways of storing information inside their devices. It was important to note this limitation as an indication that there might be relevant information within the captured data, but finding it proved beyond the abilities of our technology. Eventually, technology was developed to process the multiple technologies.

This also comes into play when examining the information within a multimedia container. Do you have enough original data to form / support a conclusion? If not, how do you present your findings? If the question is “what is the license plate?” and the evidence container includes only 3 frames of video with original (not predicted) data of the area of the license plate, do you note this as a limitation in your report? If the location of the license plate area (X,Y) within the frame, on these 3 frames in the container, is not in the same place or position from frame to frame, do you note this limitation? Do you have the tools to register the perspective of the area of interest in order to rectify the area for further processing? If not, do you note this limitation? If you don’t have the ability to register the perspective (you can quickly do this in FIVE), what then?

Some may think that noting limiting factors will enhance the opposition to one’s work. The opposite is true. In acknowledging the limitations, you acknowledge that if more can be done, further information can be gleaned. It may remain outside of your abilities (skill/technology) at the moment. Honesty, after all, is the best policy. If you have the documentation of the many times you were not able to fulfill requests due to the limitations of your toolset and/or training, you strengthen your argument for better funding.

Yes, it can be humbling to acknowledge your shortcomings. One must acknowledge and justify assumptions (things out of one’s control). One must acknowledge the weaknesses in the analysis (limitations). One must properly state the delimitations of the analysis (the scope / boundaries within which the analysis is run). Limitations are out of one’s control. Delimitations are entirely within one’s control. Putting all of this in one’s report lets the reader (often the Trier of Fact) know that you are serious about your work, that you’re attempting to work within the bounds of science, and that you have nothing to hide. If the available data supports no conclusion, regardless of what you can “almost see,” any conclusion will be limited by the fact that there is not enough data.

If you’d like to know more about the scientific processing of the digital crime scene, I welcome you to join me in Rhode Island this month … or join us in one of our many training sessions around the world.