Have you ever heard “Dulcis in fundo“? It’s a Latin saying that means: “leave the sweetest for the end”. And this is what we’ve done with Authenticate this year! Just before this challenging 2021 reaches its end, we’re sharing a new Amped Authenticate update introducing VPF Analysis. Keep reading!
VPF Analysis for Video Double Encoding Detection
Detecting double encoding is an important step towards integrity verification. Indeed, virtually all videos are encoded during acquisition. If integrity is preserved, that should be the only encoding step. Instead, whenever a video is processed in any way, including simple recompression or conversion to a different codec, that normally implies one more encoding step at the end of the processing.
The VPF Analysis aims at revealing whether a video shows traces of double compression. When this is the case, it provides an estimate of the Group of Picture (GOP) size of the previous encoding. Before you ask, VPF stands for Variation of Prediction Footprint. The meaning of such a name will be explained below.
VPF Analysis is based on the fact that, when a video is recompressed, it will normally happen that frames that were Intra-coded (I-frames) during the first compression get re-encoded as Predicted (P-frames) or Bidirectional (B-frames) during the second encoding. This is visually shown in the figure below. If a video was firstly encoded with a fixed GOP size G1 = 30 and it is then re-encoded with a GOP size G2 = 50, the 30-th frame of the video will turn from being an I-frame to being a P-frame. The same will happen for most of the subsequent I-frames.
It has been observed that when a previous I-frame is re-encoded as a P- or B- frame, there’s an alteration in the choice of macroblocks (MBs) type, for all macroblock-based codecs, as illustrated below. Several Skip MBs (S-MBs, green) and some Predicted MBs (P-MBs, blue) become Intra MBs (I-MBs, red) when frame 30 is reached. Then, the previous proportion of MBs type is restored in frame 31. Here’s the variation of prediction!
For a video where a fixed-size GOP was used during the first compression, we can then observe a periodical artifact in the number of I-MBs, P-MBs, and S-MBs. Detecting this periodical artifact is a way to reveal that the video has been encoded twice. Estimating the period allows to reveal the GOP size of the previous compression.
The VPF Analysis tool implements the algorithm presented in the recent scientific paper Video Integrity Verification and GOP Size Estimation Via Generalized Variation of Prediction Footprint. It can work under the following hypotheses:
- The first compression was carried out using a fixed GOP size.
- The GOP size used in the first and second compression are not identical.
- The first GOP size is not a multiple of the second GOP size (e.g., first GOP = 30, second GOP = 15), since in this scenario all original I-frames would be re-encoded as I-frames, and no VPF effect would appear.
- A macroblock-based codec (MPEG-2, MPEG-4, H.264) was used for the first and second encoding. VPF Analysis does not support HEVC/H.265 videos since they don’t use macroblocks.
Amped Authenticate’s implementation of the VPF Analysis has been validated against the same datasets presented in the mentioned paper. It has obtained consistent performance in terms of double encoding detection accuracy.
To access the VPF Analysis tool, click on the Video Tools menu and choose VPF, and then Double Encoding Detection.
As you can see, this tool is quite straightforward to configure. Just provide the input video, choose whether you want to see a plot besides the classification results, and hit Run.
The threshold parameter is set by default to 0.6, a value that resulted in a low (~2%) false-positive rate on the validation dataset. There’s also a Batch Processing button, allowing you to analyze all videos in the same folder as the provided one.
Let’s go practical with an example. We have this surveillance video which is wrapped into an executable file.
We drag it into Amped Replay and export two versions of it. One, using Export Original Video as AVI (we call it “stream-copied.avi“), and one using Export Original Video as MP4 ( we call it “transcoded.mp4“). The difference is that the former will only replace the container, without re-encoding the video. While the second will transcode it, thus changing pixels. Despite none of the two videos using the original container anymore, one is still loyal to the original visual content, while the second has undergone one more compression step and is necessarily less good. Let’s see how VPF analysis can help reveal who’s who.
Let’s go back to Amped Authenticate and run the VPF analysis on these two videos. Here’s the output for the stream-copied.avi file:
As you can see, no double encoding is detected, and that’s correct! The video has not been re-compressed. Let’s now run the analysis on the transcoded.mp4 file:
This time we do find traces of double compression (Row 2)! This is telling us that the transcoded.mp4 video is a recompressed version of an original video, whose GOP size was 16 (Row 5). And indeed, if we look at how the original GOP structure of the proprietary video was (you can do this with Amped FIVE’s Advanced File Info), we see that the original stream was encoded with a GOP size of 16!
So, summing up, Authenticate allows you to blindly detect video re-encoding, with no need for any reference database or reference file to compare against.
There’s more than the result table, though. When a video is doubly compressed, looking at the plot of the VPF signal you should be able to spot a periodic behavior quite evidently. This is the plot for the transcoded.mp4 file above, and a clear periodicity of 16 can be observed.
You can also use this plot to inspect the usage of I, P, and S macroblocks. It has to be noted that, for all macroblock plots, values corresponding to the position of intra-frames of the last encoding are replaced by copying the value of the preceding frame. This is needed in order not to pollute the periodicity estimation with the position of the last compression’s I-frames.
The VPF Analysis works even when the first and second encodings were carried out using different codecs. However, when bidirectional frames (B-frames) are used in one or both encodings, the VPF can still work. However, detection performance will not be as high as in the case where no B-frames are used. You can find more details in the related paper.
Improved Shadows Filter
Since its initial release, the Shadows filter gathered lots of interest from our users. The possibility of using it on recaptured images (e.g. scanning of a paper picture) and of single video frames makes it one the most robust filter in Authenticate.
Now, after using it for a while, our users asked for an improvement: for high-resolution images, wedges were sometimes too thin to be seen without zooming. Therefore, we’ve introduced a new parameter in the filter called Line Thickness.
This new parameter brings in two advantages:
- You can customize the thickness of all wedges, so that you can create visually compelling images such as this one:
- Even if you do nothing, the default line thickness will automatically be adapted based on the image resolution.
If you customize the Line Thickness parameter, this will be reflected in images in the report as well.
Don’t Delay – Update Today
If you have an active support plan you can update straight away by going into the menu Help>Check for Updates Online within Amped Authenticate. If you need to renew your SMS plan, please contact us or one of our authorized distributors. And remember that you can always manage your license and requests from the customer support portal.
Happy Holidays from the Amped Team!
