If you follow the news from Apple you may have heard that the latest iOS 11 introduces new image and video formats.
More specifically, videos in H.264 (MPEG-4 AVC) are replaced by H.265(HEVC) and photos in JPEG are replaced by the HEIF format.
Files in HEIF format have the extension “.heic” and contain HEVC encoded photos. In a nutshell, a HEIF file is more or less like a single frame encoded H.265 video. Here there is a nice introduction. And, if you want to go more in depth, here there is some more technical documentation.
For people like us, that have been working for years on image authenticity exploiting the various characteristics of the JPEG formats and various effects which happen when you resave a JPEG into another JPEG, this is pretty big – and somewhat worrying – news.
If you want to do image forensics in the real world – not in academia, where the constraints are usually quite different – it means that the vast majority of images you will work with will be compressed in the JPEG format. A lot of filters in Amped Authenticate actually work only on JPEG files, because that’s the most common case. On the contrary, a lot of the algorithms published in journals are almost useless in practical scenarios since their performances drop dramatically when the image is compressed.
JPEG has been on the market for ages, and many tried to replace it with something better, with formats like JPEG 2000 and, more recently, Google WebP. However, with the decreasing costs of storage and bandwidth and the universal adoption of JPEG, it has been impossible to displace. In contrast, video formats and codecs have seen a very rapid progression at the same time, since storage and bandwidth for video is always an issue.
I think this time will be different, for better or worse, since when Apple introduces radical changes, the industry normally follows. This means a lot of work for those of us working on the analysis of image files. Nowadays the majority of pictures are done on a mobile device, and a good part of them are Apple devices so the impact cannot be neglected.
If the HEIC format becomes the new standard, many of the widely used algorithms must be heavily modified or replaced. Don’t hope to save many of those. After all, despite what some are saying, most of the image authentication and tampering detection algorithms don’t work on videos at all. The exception is having a Motion JPEG video modified and resaved as another Motion JPEG video. But that’s a very rare case, and most times the quality will be so low that it will be impossible to use them anyways.
Now let’s see what the situation is like in practice. Continue reading