Dear friends, welcome back! Let me just say it was so nice to meet many of you at the Amped User Days last week, even though virtually. Ok, back to us: welcome to a new chapter of the Video Evidence Pitfalls series! Today we’re dealing with multiplexed videos: we’ll briefly look at different kinds of multiplexing, how to recognize them, and how to bring your video exhibit back to a “normal” form. So keep reading!
Issue: Often Video Seems Corrupted
How often do you happen to load a video only to see a mishmash of hardly playing pixels? In such situations, one typical reaction is shouting out loudly: “Damn it! It’s corrupted!”, and dismissing the file. Actually, the video is rarely corrupted. More often, the issue is that you’re not playing it correctly, especially when the proprietary player is not available or not running for some reason.
One of the possible causes for a “seemingly corrupted” video is multiplexing, which means that your file contains video streams from multiple cameras intertwined together, as happens in the example below. We can observe a pattern that goes like a clean picture, then a bunch of “corrupted” frames, then another clean picture, then corrupted frames, and so on.
Sometimes, instead, you may find that every single frame looks right, but they are played in a round-robin fashion, so that frame 0 is from a camera, frame 1 from a different camera, etc. Often it will not even be a regular pattern, since cameras may send different amounts of frames depending on what’s happening in the scene (e.g., presence of motion, etc.). Here’s an example:
We know our users, so we know there’s nothing more frustrating to them than feeling the data is there and not being able to access it. How can we deal with such situations?
Explanation: Multiplexed Videos Need Dedicated Processing
Most surveillance systems feature more than one camera. All cameras, though, are typically connected to a single Digital Video Recorder (DVR). The DVR will digitize (if needed) the incoming videos, and then encode and store them all to drive simultaneously. If you’re lucky, the DVR simply provides footage from different cameras into separate files when you export original recordings. That’s the ideal situation, of course, since you don’t have to disentangle anything. Quite often, however, DVRs resort to multiplexing techniques, so as to store frames from different cameras into the same video file. And as usual, DVR producers adopted many possible, and often proprietary, ways of dealing with multiplexing. The main two approaches are:
- Data-based multiplexing, as shown in the first of the examples above. In this case, the file contains one single (technically valid) video stream. The stream is made of many groups of frames (GOPs), and each group consists of a keyframe (or I-frame) followed by many predicted frames. While keyframes can normally be decoded independently, for P-frames the decoder needs to know how to link them to their reference, and this information is garbled. As a consequence, as we saw in the example above, common players will be able to show a few correct frames (the key-frames) but they will not be able to properly show all the P-frames (which normally account for more than 95% of the total frames, so you’re losing quite a lot!).
- Image-based multiplexing is conceptually much easier to grasp: the DVR will store a video where frames are encoded in a standard way, and they indeed play correctly. Unfortunately, however, consecutive frames show the content of different cameras, thus breaking the continuity of each view. All you can achieve by looking at that is a big headache!
Solution: Use the Right Tools
When facing a multiplexed video, you may either use the proprietary player, which knows how to demultiplex data or try to separate streams without resorting to the player. We already presented in a dedicated post the possible pitfalls hidden in proprietary players, so we won’t list them again here. We’ll just mention that many players may not have the capability to extract the desired stream in a standard format without re-encoding it, which means losing quality. So let’s take a look at the other way, extracting streams from the file.
Easy to guess, data- and image-based multiplexing need two completely different handling techniques. (Spoiler: Amped got you covered in both cases).
For data-based multiplexing, the only viable way is to work at the file level and find a way to isolate separate streams. Doing so allow you to demultiplex videos without even re-encoding them, which is definitely good for your forensic workflow. However, doing this process manually is quite a nightmare, as it may take hours to build up a few seconds of a video. A much simpler way to go is dragging your file into Amped FIVE or Amped Replay: with good chances, our video conversion engine will be able to separate streams and provide you with an individual file for each camera, and do so in a few seconds (no re-encoding means also very fast processing). Take a look:
Can you imagine an easier way? Actually, I can: if you use Amped Replay it looks even easier 😉
If dragging your video does not work, and you have an active SMS plan, just contact our support team and we’ll do our best to help you.
For image-based demultiplexing, Amped FIVE provides the Demultiplex filter, under the Frame Selection category. There’s a dedicated Amped FIVE Filter Friday video, linked below for your convenience!
If you’re only interested in one single camera, however, you may also consider using the Auto Selector filter: you’ll be able to set one frame as the reference and let the filter keep only visually similar frames.
That’s all, folks! If you ever had to manually deal with a multiplexed video, you’ll surely appreciate the automation that Amped solutions bring in. And remember: if you drop in the file and it does not work, then use the Support Portal to send it to us: we’ll let our experts analyze that specific pattern.