Dear Amped friends, we’re glad to announce that our research team, in collaboration with prof. Sergio Carrato (University of Trieste), authored a new scientific article! The title is the same as this post’s: “Does Deep Learning-Based Super-Resolution Help Humans With Face Recognition?”.
There is little doubt that AI-based super-resolution images (bottom left) are visually more appealing than bicubic-interpolated images (bottom right). But does that lead to improved recognition accuracy?
We don’t need to convince you that examining and comparing facial images is one of the critical activities in forensic video analysis, which becomes even more challenging when pictures are available at low resolution. In the last years, there has been a proliferation of AI-based super-resolution techniques that achieve impressive results, especially in the case of facial image super-resolution.
Now, let me put it straight and clear: no AI-based super-resolution algorithm is present today in any of Amped’s commercial products! But that doesn’t mean we should not ask ourselves questions about them, experiment with them, and evaluate them.
An interesting question is whether these recent SR techniques could help face recognition made by a human operator. We thus devised a study in which we addressed such a question through a simple yet insightful experiment. We used two state-of-the-art deep learning-based SR algorithms to enhance some very low-resolution faces of 30 worldwide celebrities. Then, we asked a heterogeneous group of more than 130 individuals to recognize them and compared the recognition accuracy against the one achieved by presenting a simple bicubic-interpolated version of the same faces. Results are somehow surprising: despite an undisputed general superiority of SR-enhanced images in terms of visual appearance, SR techniques brought no considerable advantage in overall recognition accuracy.
One of the pictures from the scientific paper. “BIC” stands for Bicubic, while SRFlow and GLEAN are two state-of-the-art super-resolution algorithms. The two boxplots on the left cover the case where super-resolution starts from 20×20 images, while for the two on the right 32×32 images were used. The red segment denotes the median identification accuracy.
So how come better images brought overall no gain in identification accuracy? The general impression is that neural networks sometimes removed peculiar traits and replaced them with “more standard” ones, thus hindering identification. Here’s an example of Tommy Lee Jones’s face: the peculiar shape of the eyes, visible on the left, high-resolution image, is not reflected in the super-resolution image shown on the right. And indeed, people recognized better the celebrity when confronted with the bicubic interpolated image (center).
There were cases, however, where super-resolution brought a strong advantage in terms of identification, for example with Emmanuel Macron’s face.
We published our methodology and results in an open-access, peer-reviewed paper that you can freely access at this link: https://www.frontiersin.org/articles/10.3389/frsip.2022.854737/full
We hope you’ll enjoy it!
