Does Deep Learning-Based Super-Resolution Help Humans with Face Recognition?

Reading time: 3 min

Dear Amped friends, we’re glad to announce that our research team, in collaboration with prof. Sergio Carrato (University of Trieste), authored a new scientific article! The title is the same as this post’s: “Does Deep Learning-Based Super-Resolution Help Humans With Face Recognition?”.

does deep learning-based super-resolution help humans with face recognition

There is little doubt that AI-based super-resolution images (bottom left) are visually more appealing than bicubic-interpolated images (bottom right). But does that lead to improved recognition accuracy?

We don’t need to convince you that examining and comparing facial images is one of the critical activities in forensic video analysis, which becomes even more challenging when pictures are available at low resolution. In the last years, there has been a proliferation of AI-based super-resolution techniques that achieve impressive results, especially in the case of facial image super-resolution.

Now, let me put it straight and clear: no AI-based super-resolution algorithm is present today in any of Amped’s commercial products! But that doesn’t mean we should not ask ourselves questions about them, experiment with them, and evaluate them.

An interesting question is whether these recent SR techniques could help face recognition made by a human operator. We thus devised a study in which we addressed such a question through a simple yet insightful experiment. We used two state-of-the-art deep learning-based SR algorithms to enhance some very low-resolution faces of 30 worldwide celebrities. Then, we asked a heterogeneous group of more than 130 individuals to recognize them and compared the recognition accuracy against the one achieved by presenting a simple bicubic-interpolated version of the same faces. Results are somehow surprising: despite an undisputed general superiority of SR-enhanced images in terms of visual appearance, SR techniques brought no considerable advantage in overall recognition accuracy.

super-resolution algorithms

One of the pictures from the scientific paper. “BIC” stands for Bicubic, while SRFlow and GLEAN are two state-of-the-art super-resolution algorithms. The two boxplots on the left cover the case where super-resolution starts from 20×20 images, while for the two on the right 32×32 images were used. The red segment denotes the median identification accuracy.

So how come better images brought overall no gain in identification accuracy? The general impression is that neural networks sometimes removed peculiar traits and replaced them with “more standard” ones, thus hindering identification. Here’s an example of Tommy Lee Jones’s face: the peculiar shape of the eyes, visible on the left, high-resolution image, is not reflected in the super-resolution image shown on the right. And indeed, people recognized better the celebrity when confronted with the bicubic interpolated image (center).

face recognition

There were cases, however, where super-resolution brought a strong advantage in terms of identification, for example with Emmanuel Macron’s face.

face recognition

We published our methodology and results in an open-access, peer-reviewed paper that you can freely access at this link:

We hope you’ll enjoy it!

Table of Contents

Share on

Subscribe to our Blog

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Subscribe to our Blog

Related posts