February 2, 2021
Dear loyal readers welcome to this week’s video pitfall post! Today we continue our mission of shedding doubts on whether the aspect of things in your video is actually trustworthy. Last week, we showed the “stretching effect” that is often introduced by various aspect ratio issues. Today, we’re dealing with lens/optical distortion.
Issue: Can You Trust the Shape of Objects in a Video?
Imagine you’re looking for a missing person. You ask the family for a recent picture, and they tell you, “Look, we have a video from our home camera. It shows her just before she gets out for the last time”. Hey, that could be useful, perhaps she still wears the same clothes and that would help the search. “Okay, we’ll take that video!”, you say. And here it is:
Of course, one would be tempted to just “crop, magnify and annotate”, to get something like this and promptly send it out to the media:
And that would probably be just fine for the sake of clothes. But, did you notice that walls and all lines, in general, are bent quite a bit in this image?
That suggests that the girl’s body is also distorted. So she may be appearing thinner than she actually is, and her face narrower than it actually is! Not the best image to send out, if you want people to help find her – what’s going on?
Explanation: Lenses Are Made to Bend Light!
In all imaging systems, from expensive DSLR devices to the cheapest CCTV camera, there is a lens (or a system of lenses) whose job is to fit light coming from the world into a tiny grid of pixels (the camera sensor). Lenses are indeed used to bend light since ancient times, well before photography was invented.
There are entire books about the optical properties of lenses. In a nutshell: depending on the lens design and quality, you may get several kinds of optical distortions in your image. The common trait of optical distortion, which makes it different than perspective distortion (for which we’ll have a dedicated blog post) is that straight lines become curved in the final image, just like we saw in the pictures above. The two main kinds of optical distortion you may encounter are:
- barrel distortion: straight lines are curved outwards from the center, in a shape of a barrel;
- pincushion distortion: straight lines are curved towards the center of the image.
As usual, they’re well illustrated with a picture.
Barrel and Pincushion Distortion
Barrel distortion is normally caused by wide-angle lenses, which are the typical choice for CCTV systems. By using wide-angle or fisheye lenses you can fit a very wide view into the image sensor. So your camera will capture more of the surrounding events. As you can clearly see in the picture at the beginning of this blog post, the camera is able to capture almost the entire room in front of it, with a viewing angle close to 180°. It’s impressive, it’s useful, but it’s also distorted.
Pincushion distortion is instead typical of zoom lenses, especially low-end ones.
From the grid picture above, you may also notice that in both types of lens distortion the effect becomes stronger as you move away from the center of the image. That is a consequence of the fact that the lens is less curved at the center and more towards the outer part. The result is an increasing amount of lens distortion.
Solution: Fix Lens Distortion if You Can, Work Around Otherwise
Optical distortion is something that can be corrected quite effectively since there exist accurate mathematical models for the problem. If you are an Amped FIVE user, you have two restoration filters to compensate for optical distortions: Undistort and Correct Fisheye. Which one should you use?
Well, if you know that the camera mounts a fisheye lens, you are better served by the Correct Fisheye. While the Undistort uses a more general mathematical model. In our example above, we see that the video comes from a Canary camera, which indeed mounts a fisheye lens. With the Correct Fisheye tool, you can indicate to the software three “curved” lines in the image. Then choose one among the Mapping Functions (if you don’t know this lens parameter, just try all of them and pick the one providing the best visual correction). It’s all done:
Notice that the ability to automatically compute correction parameters by selecting curved lines makes the process much more objective. It is certainly good for forensic and investigative workflows.
We could have used Undistort as well. But as you can see below, the correction would have been less optimal in this case:
If you don’t have Amped FIVE or some software that can fix distortion at hand, there’s still something you can do. Check whether the subject appears closer to the center of the image in another frame. This way, the impact of optical distortion would be less marked.
Conclusion
Before we say goodbye, one important comment: filters that correct lens distortion typically assume the optical center to be in the pixel center of the image. That means: never crop the image before compensating for the distortion, as this would make the optical and pixel centers of the image diverge (unless you make a perfectly central crop) and lead to wrong results!