The capabilities in Amped FIVE for the correction of lens distortion have recently increased with the new Camera Calibration filters. Supporting filters such as Undistort and Correct Fisheye, Camera Calibration allows for finer control and flexibility, especially with portable cameras or difficult scenes.
Read more in this latest “How-To” blog post.
Hey everyone. In the last update to Amped FIVE, we introduced two new filters to manage Camera Calibration. We summarized the process in the update blog post, and now it’s time to take a closer look.
In this “How-To” article, we will examine the filter settings in more detail and explore other considerations when using the filters.
Before we delve into the “how”, it is essential to consider some other key questions and remind ourselves of the different options available to Amped FIVE users.
Why is Camera Calibration Important?
Ensuring visual authenticity is a vital component in the presentation of video evidence. The process of correcting the lens distortion in the digital representation of a scene is therefore one of the most common restoration functions used by Forensic Video Technicians and Analysts.
Shape, size, and the perception of distance are all severely affected by lens distortions. Furthermore, subsequent analysis for comparisons or measurements would be entirely unreliable without ensuring scene and object authenticity.
What Are Other Methods of Visual Restoration?
All the filters used for restoring lens distortion can be found under the Edit category. The most basic is Undistort.
Using one of the training samples in Amped FIVE that deals with Aspect Ratio, we have applied the required filters in the correct order to restore the interlaced detail and then correct for the aspect ratio error.
Following the image generation model, we must now restore the geometrical distortions caused by the lens.
It is easy to see the curves on the two vertical corners, and also along the ceiling. In reality, these are straight.
Our decision, therefore, to use these lines is objective. We are not being subjective and just guessing. We are using objects in the scene that are known to be straight, and informing the software that the curves should be corrected.
After applying the line data to the filter, the view is restored.
To learn more about Undistort, watch the video below and read this blog post.
There are times when Undistort may not be suitable, or you require much finer control. For these situations, Correct Fisheye could be used.
As we can see in the next image, after evaluating the result of Undistort, the straight edge of the table at the bottom of the field of view is now further distorted.
Using Correct Fisheye and then selecting known straight edges in the scene again results in a corrected view.
Another important benefit of Correct Fisheye is the ability to move the view up, down, and rotate, depending on the investigative requirements.
To learn more about Correct Fisheye, watch the video below.
In both previous filters, objects with known straight lines in the scene have been selected, and the view has been corrected according to those entered parameters.
It may be possible to place an object with straight lines into the scene and then use those. A commonly used object is a pinboard, as seen below, with possible lines selected to correct the distortion.
With these filters, though, there are a few considerations.
- It is you, the user, who is dictating which lines are to be used for the correction. Consequently, different users may select different lines, and therefore, the results would differ slightly. Subsequent measurements would then be affected by these differences.
- Only a single frame is used, and the lines selected may not be evenly placed through the field of view. Staying with lines, we come to the last point. It may be difficult to select lines, and in many places, there may not be any lines at all in the scene.
If we were able to place many straight lines into the scene, the software could identify those lines automatically. This would help solve some of the challenges associated with restoring lens distortion.
This is where Camera Calibration enters, and brings us nicely to the purpose of this post: the “how”.
How to Use Camera Calibration?
The process of calibration has been split into two. It allows flexibility to create Camera profiles in different projects, and then the ability to load those created profiles in subsequent case projects.
- Create Camera Calibration allows for the creation of a camera profile.
- Load Camera Calibration allows for the loading of a previously created camera profile.
Using the same scenario we looked at in the release post, here we have a scene recorded from a cyclist’s GoPro before an incident. There are minimal straight edges, and the lens distortion is difficult to visualize.
Using the camera, with the same settings, we can record a small piece of test footage, capturing a printed grid.
It is important to ensure that the settings are the same. Pixel resolution is fairly simple to match by identifying the size of the evidence footage and then selecting those settings on the camera. Lens settings may not be available, though. With cameras such as sports cameras and dashcams, there may be options such as Ultra-Wide, Action, or Linear. If the lens settings are unknown, then it would be beneficial to record several different pieces of test footage, using all the lens modes available.
For this scenario, the grid does not have to be that large, as we can hold and move the camera around the grid. You can find a good grid example available here. It is very important to print the checkerboard on a rigid flat surface, so that it does not bend.
Move the camera around the grid, capturing it from all angles.
The export of this test footage must be made in the same manner as the evidential clip. Several cameras have save-to-cellphone capabilities. If that was selected in the evidential footage, then that same process should be used.
Treat this footage as an evidential item to preserve its integrity. Make full, concise notes on the camera usage, settings, and acquisition method of the test recording.
If a static CCTV camera is used, and you have moved the grid around the scene, then the acquisition of this footage should be made in the same manner as the original evidence.
The first task after loading this test footage into Amped FIVE is to minimize the number of frames required for calibration. It may be possible to use a single frame. However, it’s best to use a small range of frames, or, as in this example, use Sparse Selector to obtain frames at different positions. The selected frames should contain the entire grid, be in focus, and without motion blur.
Here we have the six frames selected.
Using multiple frames, with different perspectives, sizes of grid, and positions within the field of view, allows the software to calculate more complex distortions. This approach particularly assists in the computation of correction when the optical axis is not in the centre of the image.
We can now select the new Create Camera Calibration filter.
As you can see, from the Test Pattern tab, different pattern sizes are supported.
With handheld cameras, a smaller board, such as A4 or US Letter, will usually suffice. However, when conducting camera calibration on a static camera from a distance, a much larger board will be required to ensure that enough of the grid is replicable after capture and compression.
The width and height parameters represent the internal corners of the horizontal and vertical lines. In our test footage, the board width is “9”.
Let’s click Apply now, and we will look at the other tabs after evaluating the results using the default settings.
The computation is very quick, thanks to having selected only the required frames, and the results initially look promising.
One of the first things to check is that the points have been identified.
In the Output tab, change the Output Mode to “Original”.
Then, move to the Test Pattern tab and check the box to Show Live Points.
This is a good way to verify that the software has identified the grid correctly.
We can now make some adjustments or review other options.
To better visualize the correction, it is worthwhile to increase the canvas size. You can always reduce this later if required. The Slider is under the Output tab.
This enables us to observe the edges of the frame. With this footage, it is now possible to identify that the floor panels on the left and right still have slight curvature.
This is then a good opportunity to review the parameters under the Calibration Model tab.
Under Distortion Type, we have three alternative mathematical models that use polynomials in their calculation to correct distortion. A polynomial is a mathematical expression consisting of variables, coefficients, and the operations of addition, subtraction, multiplication, and non-negative integer exponents. Essentially, it’s a sum of terms, where each term is a product of a coefficient (a number) and a variable.
Polynomial
The Polynomial model uses a polynomial to represent the radial distortion. This model is mainly used to correct small amounts of barrel and pincushion distortion.
Rational (Default)
The Rational model utilizes a rational function (ratio of polynomials) to represent radial distortion, providing a more accurate representation than simple polynomial models, particularly for wide-angle lenses.
Fisheye
The Fisheye model uses a polynomial function of the angle of light rays entering the camera, rather than of the radial distance. This allows correction of the radial distortion of lenses with a very large field of view, possibly 180 degrees or even more.
Next, there is a checkbox for fixing the principal point. When the checkbox is active, the model assumes that the optical center of the image coincides with the central pixel of the image, and this may speed up the computation.
In high-quality cameras, the manufacturing process would normally ensure a very precise alignment between the sensor and the lens. In contrast, cheaper mass-produced CCTV cameras may suffer from lens placement error, where the central point may not be accurate.
It is also important to remember that, whenever you crop your image, you are implicitly moving the optical center away from the image center. Unless, of course, you do a “central crop”, which means removing the same amount of lines on top and bottom and right and left sides of the image. Cropping before camera calibration, though, is not normally recommended.
Finally, you have the various Coefficients parameters.
As we said, polynomials are a sum of terms. By adjusting these parameters, you are instructing the filter on how many terms the Polynomial, Rational, or Fisheye models have to use.
With the Rational model, the Numerator and Denominator Coefficients are adjustable within the slider range. With the Polynomial model, only the Numerator slider is used.
Finally, the Fisheye model uses its own set of coefficients.
In many cases, the default settings will result in a corrected view. However, as we have discovered, the default in this example still had a slight curvature.
In this example, increasing the denominator to “3” produced a much better result. However, as a general rule, we recommend using the minimum number of coefficients that result in a satisfactory correction. Using more coefficients than needed may result in a model that “overfits” the checkerboard region and therefore performs less well in other regions of the image.
Overfitting results are usually easily visible, as the corners of the image “wrap” and reflect portions of the image that are actually closer to the center.
Important: Create Camera Calibration is not an Auto-apply filter. That means, every time you switch to a different model in Distortion Type, remember to click “Apply”. Then, turn to edit the number of coefficients. This will ensure you only edit the parameters that are actually used by the newly selected distortion model.
Before we save this calibration profile, we will briefly look at the mathematical data calculated by the filter, which will be used in the distortion correction.
This is the data that will be used by the Load Camera Calibration filter.
Moving back to the Output tab, we can rename and save it as an AFCC file.
After saving the Project used for calibration, let’s now load our Amped FIVE project that contains the footage recorded by the cyclist.
Load Camera Calibration
We can load our .afcc file into the chain using the Load Camera Calibration filter. You can, of course, have everything in the same project, but that is not required.
Let’s concentrate on this tab, where we select the file and then some basic presentation parameters.
After applying the filter, the camera calibration profile that we calculated using the checkerboard is applied to the incident footage.
Here we can see the before and after views.
For static cameras with a large field of view, you may then wish to utilize the Post-Projection adjustment capabilities within the Post-Projection tab. This is the same as we looked at earlier in the Correct Fisheye tab. It would all depend on the requirement of restoration and where the incident occurred in the field of view.
These new filters are powerful additions to the current lens correction capabilities, especially for sport cameras, dashcams, and Body Worn Video (BWV).
Various checkboards of different sizes will now be invaluable additions to your scene kits, along with aspect ratio spheres, pixel boards, height charts, and measuring equipment.
Many situations may require testing and evaluation. Particularly those camera systems that utilise multi-sensor cameras, and then stitch them together for an ultra-wide field of view.
These are 2 cameras, and may have to be treated as such.
Conclusion
We hope you have enjoyed this tour through the use of the new Camera Calibration filters. We are continuously developing the functions within the filters, so keep watching in future Amped FIVE updates for advancements.
