Welcome, dear friends, ready for this week’s video evidence pitfall? Today we’re dealing with one video issue that often gets overlooked: perspective. It’s something we’re all familiar with, but in video analysis, it can play nasty tricks on you.
Issue: you shouldn’t compare the size of things at different depths
A robbery takes place and you get called. When you reach the shop, the owner runs at you yelling a classic statement: “I got him! I have a video!”. Indeed, there’s a video on the CCTV system and the suspect is there, as he walks into the shop (we’ve simulated a balaclava in the picture below). The face is disguised, but there’s good information about clothes. Then you think: well, I’m here in the shop, the suspect is close to the door. So why don’t I measure the height of the door to find out the height of the suspect?
Well, certainly measuring the door is an excellent step towards height analysis. But don’t take it too easy: that would just be the first step of a longer process. Indeed, you can’t directly compare the size of things at different depths because perspective makes farther things look smaller!
And it’s not just a matter of measurements: perspective affects the human view of an event. What you think you see may be affected by perspective. Two planes in the sky may appear to be about to crash into each other when they are actually hundreds of feet apart. For CCTV, it may look as if an action or movement is one thing, but it is not in reality. Moreover, the perspective of the camera must be considered, and this is often very different from a witness perspective. Many cameras are high up and angled down, whereas witnesses will be at ground level and looking perhaps from a different angle. What one person sees may be very different from the camera’s view.
Explanation: perspective makes a difference
I guess everyone reading this blog has come across some “visual trick” images like the one shown below (click on it to view more!). Most of these funny images take advantage of perspective. Things that are farther away from the camera will look smaller in the image, so by placing subjects properly you can create funny situations.
In a very lucky case where objects lay on the same plane, and you have a picture taken with the camera perfectly parallel to such plane, then it would make sense to compare the size of objects. Here is an example of such a lucky situation:
Using this image alone, and without any reference, you’ll never know how long each pen is. But you can at least say they have the same length. If you move the camera away from the plane, still keeping it parallel to the plane, the pens will become smaller, but equally so.
We’ve already learned something: in order to measure something in a picture, we need a reference measurement. Otherwise, we can at most compare sizes.
But things get quickly worse when the camera is tilted (which is the normal situation, especially in CCTV footage). Take a look at the picture below: pens appear to have different sizes because of perspective!
As a consequence, comparing the size or height of things at different depths can be very misleading! For example, are you sure you would be able to correctly rank these cars by their size?
Solution: use dedicated tools, or play it safe!
If you want to take measurements from a CCTV video, there are several possible solutions:
- Single View Metrology (SVM): if you know the size of a reference object, you can use an SVM tool, such as Amped FIVE’s Measure 3D filter. This will allow you to mark some calibration lines in the frame, which are used by the software to “compensate for the perspective”. You then select the reference object and specify its height, and you’re ready to measure other objects lying on the same plane. In the example below, we used the height of the door as a reference, and we measure the subject. SVM provides an objective and scientific approach to measurement.
- Reverse projection, which can be helpful to have a visual validation and cross-reference SVM results. You go back to the scene with a measuring stick, place it in the same position where the suspect was, and then use reverse projection to estimate the suspect’s height (there’s a dedicated blog post here!).
- Other approaches, including laser scanners and 3D animation, can help when conditions are not advisable for the two previous methods. They tend to be more time consuming and expensive, though.
One final remark: before measuring things in a video, remember that first, you may need to fix aspect ratio or optical distortion issues (click on the respective link to read the dedicated blog post of this series!).
