Dear friends, welcome to this week’s tip! Today we’re sharing a tip to make the most of the Perspective Super Resolution filter recently introduced in our Amped FIVE. Right from its release, it’s one of the filters our users love the most, thanks to its ability to automatically track and align license plates even under strong perspective variations! But, you know, here at Amped we’re not happy to provide you with just nice results. We want to give you excellence! So, today’s tip is about further improving the image produced by Perspective Super Resolution… keep reading to find out more!
It’s no secret, forensic video analysis often involves reading license plates from severely degraded CCTV videos. Sometimes you’re dealing with a group of frames showing a rapidly moving car, and you can’t find a frame where the license plate is readable.
Or, you can read something, but with limited confidence, as in the example below.
In these cases, the ability to integrate the information available in adjacent frames is of paramount importance. There are essentially two ways to go:
- Stabilization followed by frame averaging;
- Super resolution techniques.
Frame averaging is often considered a good way to reduce various kinds of noise that may be affecting your footage, but it won’t help much when the resolution is just too low. Super resolution methods, instead, create a larger representation by “pooling” pixel values from multiple frames.
In both cases, however, the ability to track the object’s movements and stabilize them is fundamental. When such movements are not simple “translations”, but perspective changes, like in the sketch below, there’s no way to “superpose” the three license plates by bare rotation and scaling.
The Perspective Stabilization filter takes care of this complex stabilization task and provides input to the Perspective Super Resolution filter, that will compose a higher resolution image by pooling pixels taken from neighboring aligned frames. For the two samples showed at the beginning of this post, this is what we obtain using Perspective Stabilization and Perspective Super Resolution.
In both cases, we see a clear improvement! However, both obtained images are a bit blurred. This happens quite often when integrating information from multiple frames, because the stabilization and alignment steps are not 100% accurate, thus resulting in a smoothened output image. Luckily, Amped FIVE has plenty of ways to mitigate this! Let’s see our favorite two.
If you’ve integrated a rather large amount of frames (e.g., 60, as in our orange-car example above), then simply increasing the Sharpening parameter of the Perspective Super Resolution tool may get the expected result.
Below you can see how blurriness gets mitigated in our example by increasing the Sharpening parameter from 1 (left) to 5 (right).
Unfortunately, this approach won’t give the expected improvement when you’ve integrated a smaller number of frames, because the artifacts introduced by the sharpening operation can get very annoying, as shown below:
This is due to the shortage of reference pixels, which is a consequence of the scarcity of frames. In these cases, we recommend trying the Blind Deconvolution filter instead of using the sharpening feature included in Perspective Super Resolution. This is what we get using blind deconvolution in our case:
Before we conclude, let’s compare the starting pixels (left column) with the enhanced version (right column) for both images:
Okay, for the license plate on the top we’re still unable to read everything, but we can make an educated guess for the three starting characters and perhaps the last two numbers. That could provide enough investigative information to move on with the case.
