If you follow the news from Apple you may have heard that the latest iOS 11 introduces new image and video formats.
More specifically, videos in H.264 (MPEG-4 AVC) are replaced by H.265(HEVC) and photos in JPEG are replaced by the HEIF format.
Files in HEIF format have the extension “.heic” and contain HEVC encoded photos. In a nutshell, a HEIF file is more or less like a single frame encoded H.265 video. Here there is a nice introduction. And, if you want to go more in depth, here there is some more technical documentation.
For people like us, that have been working for years on image authenticity exploiting the various characteristics of the JPEG formats and various effects which happen when you resave a JPEG into another JPEG, this is pretty big – and somewhat worrying – news.
If you want to do image forensics in the real world – not in academia, where the constraints are usually quite different – it means that the vast majority of images you will work with will be compressed in the JPEG format. A lot of filters in Amped Authenticate actually work only on JPEG files, because that’s the most common case. On the contrary, a lot of the algorithms published in journals are almost useless in practical scenarios since their performances drop dramatically when the image is compressed.
JPEG has been on the market for ages, and many tried to replace it with something better, with formats like JPEG 2000 and, more recently, Google WebP. However, with the decreasing costs of storage and bandwidth and the universal adoption of JPEG, it has been impossible to displace. In contrast, video formats and codecs have seen a very rapid progression at the same time, since storage and bandwidth for video is always an issue.
I think this time will be different, for better or worse, since when Apple introduces radical changes, the industry normally follows. This means a lot of work for those of us working on the analysis of image files. Nowadays the majority of pictures are done on a mobile device, and a good part of them are Apple devices so the impact cannot be neglected.
If the HEIC format becomes the new standard, many of the widely used algorithms must be heavily modified or replaced. Don’t hope to save many of those. After all, despite what some are saying, most of the image authentication and tampering detection algorithms don’t work on videos at all. The exception is having a Motion JPEG video modified and resaved as another Motion JPEG video. But that’s a very rare case, and most times the quality will be so low that it will be impossible to use them anyways.
Now let’s see what the situation is like in practice.
After downloading the iOS 11 update, I tried to do some testing. What happens is the following.
By default, if your device supports HEIF format (I have an iPhone 7 Plus), it will save the photos in HEIF format. Recent Apple devices support encoding and decoding at the hardware level. You can switch it back to “Most Compatible” in the “Camera” app and it will save them in JPEG. I assume the vast majority of the people will never bother changing the default settings.
But something more interesting happens on the settings of the “Photo” app. There is a setting that tells you whether transferring the files to another device will keep the originals or convert them to a more compatible format. In my case, this setting converts the images to JPEG when transferring them to my Windows 10 PC.
If you dig into the official Apple presentation here, you can see more interesting details. For example, transferring via email or sharing app extensions should always imply a conversion. While for transferring between devices, it depends on the device capability. Transferring through AirPlay, for example, should keep the original format.
I tried to do a few tests. I captured a random picture (not many details, just the door of my office) and then exported it to my PC with the default settings (“Automatic”). It saved a JPEG of 2.34 MB.
Then I went back to the settings and set the transfer to “Keep Originals”. This time I got an HEIF file of 800 KB. This is impressive, as it’s 1/3 of the JPEG file in size. But, to be fair, the picture was pretty flat so the spatial prediction of the new format may have had an easy game. The problem is that in this moment very few pieces of software are able to deal with it.
I then wanted to check the metadata of the JPEG file transferred in the first place and compare it with a picture shot with iOS 10, before the update to iOS 11. The file has exactly the same format features and quantization tables. The only difference is the Exif Software version where the iOS version number is indicated. So, while technically this is not an original file since it has been converted, we can consider it indistinguishable from an original.
On the contrary, if you find a HEIC file, it’s more likely for it to be an original since its support in software is very limited nowadays.
The conclusion of this little analysis is that for now, we can still rely on the good old JPEG for pictures coming from iPhones and other iOS devices, it does not change much than before. However, we must be prepared for the future and see how the HEIC format impacts this.