“Thou shall peer-review your analysis” is one of the well-known (and often ignored) rules of forensic reporting. Sometimes, this important principle gets poorly translated into: “let’s have a colleague peek into my results”. And so, it may happen that an investigator or examiner will ask a colleague for their opinion before submitting the results.
Issue: Being Objective Isn’t That Easy
Let’s imagine John calling Lucy to his desk and asking her: “Can you help me with this license plate? I can read BC 537 but I can’t seem to get the last two characters, perhaps they are “TT”, and I’m also quite uncertain about the first one.”
What should Lucy tell John?
- Sure thing! Let me help you with my independent review.
- Er… you’ve just burnt me as a potential reviewer of your work.
Well, surely option 1 sounds more polite, but… option 2 is the correct one. By telling Lucy what his interpretation is, John is unwillingly biasing her work and interpretation. Lucy may unconsciously tend to process pixels and select frames in such a way that confirms what John told her. Guess what: this is called confirmation bias.
License plates are just the tip of the iceberg: for example, face comparison is an even subtler field. When enhancing a persons’ face, different processing may easily lead to a significantly different facial appearance. Therefore, it is fundamental that the examiner who processes the video is not shown the suspect’s face beforehand, otherwise, they may tend to unconsciously “tune” the enhancement to make it a match (or vice versa, of course). For example, look at how different the nose of this guy looks in the left and right pictures below: they start from the same footage!
One last but very common example. An analyst is asked to compare some clothing, and is sent the CCTV and the CSI images of the clothing seized from the suspect — at the same time! By seeing what is expected before examining the unknown, they are already biased.
Explanation: Cognitive Bias Is Always Round the Corner
The technical term for the problem explained above is “cognitive bias.” Cognitive bias makes you reach conclusions in a way that is not fully rational or logical because you are influenced by extraneous information that should find no room in a forensic examination.
Psychologists have investigated this subject extensively, and they’ve come up with some categories of cognitive bias. Here we’re listing three of them which are quite common:
- Expectation bias covers the case where the expectation of what will be found affects what is actually found. For example, if you’re asked to investigate an image showing an alien, and you are strongly convinced we’re alone in the universe, you may tend to only account for elements against image authenticity and discount elements in favor of authenticity.
- Confirmation bias is, in some sense, a special case of expectation bias: once they’ve been provided with an interpretation, people tend to confirm it rather than opposing it, especially when the previous interpretation is provided by a famous (and/or higher-in-grade) expert.
- Contextual bias occurs when other information, extraneous to those being considered, steers the outcome of the analysis. For example, performing facial comparison after being informed that the suspect has pleaded guilty may steer the interpretation of available data.
If you want to take a deeper, very instructional tour in the causes of cognitive bias, its effects, and possible mitigation strategies, here’s an online resource you will love: Cognitive bias effects relevant to forensic science examinations. You can find even more detailed treatments in the scientific literature, and the linked document will provide you with a rich bibliography.
Solution: Implement a Mitigation Strategy
Needless to say, there are entire books on strategies for mitigating bias. However, some simple rules could already improve your workflow quite a lot:
- In identification, never reveal the suspect’s face, license plate, etc. to the examiner. This is probably the worst error that you can make, for the reasons we explained above.
- Clarifying the roles within your agency: having a “case leader” or “lead scientist” who builds up the case and then assigns technical examinations to other people is considered a good starting point. The case leader would be informed of relevant contextual or case information, but they will shield the examiners so they don’t get biased by it.
- Allow verification by a second expert, which should work independently, avoiding any communication that could lead to confirmation bias.
Once again, these are just some of the possible mitigation strategies, you will find more of them, together with examples, in the guidance document linked above. We hope that, after reading this post, you’re frightened by the possible impacts of cognitive bias and willing to eliminate it from your casework!
It is easy to misinterpret and incorrectly process imagery and video. This is why any analyst must build these strategies into their forensic workflow. Not understanding the original media, incorrectly processing footage, and having a bias towards a result is a disaster waiting to happen.
