Dear friends welcome to this week’s tip! Today we take inspiration from a question that we receive from time to time from our users. It basically goes like this: “How come consumer video players like VLC sometimes appear to play the video better than Amped FIVE or Amped Replay?”. If you’re curious about the answer… just keep reading!
Right from its beginnings, Amped has been pursuing a crystal clear mission: becoming the one-stop-shop for any needs related to image and video analysis and enhancement. The company motto, Justice Through Science, helps to clarify our mission to deliver products that are based on the scientific method. Now, acting scientifically requires you to:
- be aware of what you do,
- give the reasons for what you do, and
- use validated tools.
How does all of this relate to the “VLC” question? Let’s see it with some examples!
Scientific Tools Require Scientific Methods
First and foremost: resampling. Imagine you’re given a surveillance video that happens to be playable with VLC. You then notice that VLC opens up in a small window… that’s because the video resolution is very low.
One is naturally tempted to drag VLC’s window corner to enlarge the player… and it works! The video actually gets as big as your screen allows.
But how did that happen? As a consumer video player, VLC simply aims to provide a pleasant image. It will enlarge your video using interpolation, which means, it will compute new pixel values from the actual pixels. There’s nothing wrong with resampling, but if you’re doing video forensics, it must be done intentionally. You must save all the information about which algorithm you used and with which parameters.
Moreover, resampling should be placed at a certain point in the processing chain, which the analyst should accurately choose. For this reason, Amped FIVE and Amped Replay will not compute new pixel values when you enlarge the video with the magnifier (or scroll with the mouse wheel). They will simply make current pixels larger (so-called “nearest neighbor” interpolation) so that you can fully understand the amount of information that is present in your evidence. When you decide that interpolation is needed, you’ll readily find it in the Resize and Smart Resize filters under the Edit category.
Metadata Can Be Misleading
Let’s now move to another case: we’re working on a .MOV video file. Once opened with VLC, it shows this urban scene, with people walking on a street.
“Hooray!”, we say. It’s decoded by VLC so this will be an easy one for FIVE! We thus drag the video into Amped FIVE, and we’re presented with this:
It’s upside down! Why is that? Well, it’s easy: the guy who took the video was holding the camera upside down, so the video was recorded on the camera sensor upside down as well. But the camera was aware that it was being held in the “wrong” position, so it simply added to the video container, information about how to process the video and make it play “better”. This is easily spotted using the Advanced File Info tool in Amped FIVE:
Now, when VLC reads the video, it will take that “suggestion” into account and automatically rotate the video. Amped FIVE (or Amped Replay) will instead show pixels the way they are actually encoded. They let the user make a decision about whether they should be rotated or not.
Now, you may be wondering: “Okay, but why don’t Amped products do what the container tells it to do”? There are two reasons:
- Amped products prioritize forensic soundness, while VLC and consumer players (understandably) prioritize a “click and watch” approach. We believe forensic software should not make decisions in place of the analyst, as it could get something wrong or it could add a processing step of which the analyst is not aware, and this is not good.
- If you know video containers, you also know you shouldn’t trust them so much!
Let us show you what we mean with point 2, with another example. We’re given another video, this time coming from a CCTV system with analog cameras. (Somewhat surprisingly) we see VLC can play it, so we happily launch it:
Let’s look at how it appears in Amped FIVE:
We promptly notice a first, big difference: in Amped FIVE the vehicles show the typical interlacing artifact, which was not visible in VLC.
Full Analyst Control vs Hidden Automation
That’s because VLC, by default, automatically deinterlace videos when they need it. But it’s done in silence, so if tomorrow you’re asked or challenged about which deinterlacing algorithm you used… what will you say? You maybe even don’t know that you deinterlaced, because the software did it without any warning!
In Amped FIVE, it’s just a matter of firing up the Deinterlace filter: with just one click, you get full control of the deinterlacing algorithm to use (the default one is normally OK), and full documentation and repeatability. I wouldn’t trade in these two things for one click. But there’s more than that. You may have noticed that this video has black bands on the vertical sides. If we draw a rectangle around them, we’ll see they’re both 576 high (as the whole frame) and 8 pixels wide.
No image was ever recorded there, so the actual samples’ width is 720 – (8 x 2) = 704. Now, if we take a look at the container, we’ll notice it has information about how it was digitized, the Sample Aspect Ratio, which is 12:11 (if you want to know more, take a look here).
Let’s put this two information together: 704 x 12 = 8443 samples in width and 576 x 11 = 6336 samples in height. And if we compute the ratio, we have 8443/6336 = 4/3, which is the standard PAL aspect ratio. And so? If we want to display the video correctly, we need to deinterlace it, remove the two bars on the sides using the Crop tool, and finally use the Aspect Ratio tool, set to 4:3.
A normal video player doesn’t care at all about the black bars on the sides. It will just read the Display Aspect Ratio from the container and stretch the video, in this case incorrectly.
Final Note
We could continue with more examples: some video players will “automatically” deblock or filter videos to improve their appearance, or perhaps they will silently retain the image filtering settings that you used on Monday when you reopen the software on Friday to work on a different video (e.g., VLC works this way). It’s nothing serious if you’re watching a movie, but it can ruin someone’s life if you’re doing video forensics.
We hope this tip explained the reasons why we tend to not automate too much in our products. We want the user to be in control, and we offer our training to get them ready to make use of such control. Something consumer video players simply don’t provide.
