When performing image and video restoration through filtering, the order of the filter chain does make a difference. Read the scientific paper we recently co-authored with the University of Florence!
Dear friends, we did it again! Amped Software has co-authored a paper published in IEEE Access that sets an important milestone on the topic of the image restoration workflow. The paper specifically addresses the order in which defects affecting an image should be compensated. Keep reading to learn more!
The Restoration Workflow Problem
If you’ve attended our training or followed our blog, you know how much we care about the “scientificness” of forensic image processing. We always remark that image restoration and enhancement have little to do with pressing buttons until you get a nicer image. You have to know what you’re trying to fix and how to fix it properly.
With Amped FIVE offering over 140 filters for image analysis, restoration, and enhancement, a very common question we get asked is: which is the correct order by which those filters should be applied? For example, should I fix the blur or the perspective first, if I have an image suffering from a distorted perspective view and optical blur? As we’ve shown in our blog, the answer is you should first fix the blur and then the perspective. Indeed, the distortion to the object occurred first within the image generation, followed by the out-of-focus lens causing the blur. As exemplified in this image (borrowed from the article), the restoration must reverse the order in which the defects occurred.
While this sounds very intuitive, our diligent students asked us if we could thoroughly justify this. Basic math tells us that a composed function can be inverted by composing the inverse functions in the reverse order.
Unfortunately, as every video analyst knows very well, we don’t have the exact inverse for blurring, optical distortion, etc. We only have some “estimate” of it, which is what Amped FIVE filters provide. It turns out that answering the question in this more realistic setting is not trivial!
Therefore, we joined efforts with professors from Engineering and Mathematics at the University of Florence. After working months on this, we published a paper in the IEEE Access journal presenting our results. The paper focuses precisely on the “blur and perspective” scenario presented above. It shows, both mathematically and experimentally, that using the inverse order is the correct choice. You can see a visual example in the figure below. It is also borrowed from the paper: the restoration order in figure (b) provides a partially readable license plate, while others fail.
Final Note
The paper is open-access, which means you can freely read it without paywalls. Just click on this link: A Proposed Workflow for the Restoration of Image Artifacts in Forensic Applications. We’re deeply grateful to Prof. Argenti from the Dept. of Information Engineering and Prof. Bellavia and Rebegoldi, and their student Alberto Limone, from the Dept. of Industrial Engineering of the Univ. of Florence for all the efforts they’ve put into solving this intriguing problem!
At Amped Software, we believe in research, and we work to provide a scientific basis for all you do with our products. While we publish this article, we have several other research projects going on with Universities in Italy and abroad. So stay tuned!