The Macroblocks filter was introduced in the recent Amped FIVE Update 7620 and it is a very simple tool to use. The key here, though, is understanding and interpreting the presented data correctly. Let’s take a closer look into the world of MPEG Macroblocks and Motion Vectors.
It’s important to first go over some MPEG basics.
What I am about to detail is a highly simplified explanation. The various flavors of MPEG, and then the huge array of profiles and terms within each one, means that any attempt to cover all aspects would lead to confusion! With a knowledge of the basics, you will be able to research and source any specific details required that are relevant to your individual task.
Three Types of Image
In MPEG streams, the video is made up of three different types of image. They are stored in groups called a GOP – A Group of Pictures.
- I (Intra) – These are completely refreshed, new data. They only reference itself and do not rely on any other image in order for it to be decoded correctly. The first image in a GOP must be of this type.
- P (Predicted) – These use one or more previously decoded reference images in order to construct a full picture.
- B (Bi-Directional Predicted) – These use previously decoded reference images and future displayed images to be constructed. As a result, the order that the images are encoded and decoded will be different to that displayed.
Example of a 6 frame GOP
Here we have 2 examples of a 6 Frame GOP.
The example on the top does not utilize B frames. Each Image is displayed in sequence and the order that they are coded and decoded is displayed at the bottom.
The lower example does utilize B frames. They are in the correct display order but note how they have been coded and then how they should be decoded. Coded picture 1 is actually frame 4 (Remember we start at 0). As a result, this needs to be decoded before frames 2 and 3.
It is possible to have an I frame only Mpeg, but these are not utilized very often within surveillance video due to the storage requirement – They would take up too much space. If a requirement was for a new image, all the time, they would not use Mpeg, they would use Mjpeg or another compression type.
It is also possible to have a single stream with a non standard GOP. It could place an I frame when the scene changes beyond a certain level, rather than at a specific number of frames.
More common though is that the recordings have only I frames when motion is detected and then at other times, it reverts to utilizing P and B frames.
The Macroblock Filter
Each image, regardless of its type, is made up of blocks. These blocks, consisting of 8px x 8px, are created through the use of 4 smaller blocks (4px x 4px) that are derived by Discrete Cosine Transform (DCT) coding.
Four of these blocks, 16px x 16px, make a macroblock. These macroblocks can be encoded in a number of different ways depending on the complexity and referencing of the item being encoded.
In an Intra Frame, all the macroblocks will be newly encoded. In Predicted and Bi-Directional Frames, the macroblocks can be newly encoded, copied, moved or changed.
The Macroblock filter allocates a color to a block if a specific condition is met.
Purple and red macroblocks
In this I frame, I have a mixture of purple and red macroblocks. The red shading identifies macroblocks that have been divided up in such a way to support gradients in light and color. Whereas the purple shaded macroblocks are coded in their entirety.
Gray and green
I have mainly gray in this P frame, with green and then a single purple and red. (I have enlarged the image to get a better view!)
Now, I hear a few people saying, ” hang on, what’s the gray?” Well, these have no condition set. They are merely copies from the previous frame. The encoder has detected no change in color or luminance.
The green blocks are referenced, and predicted, from a previous frame. The encoder has not been able to reference two blocks so these have been newly encoded.
You will also see macroblock partitions, visualized with the lines and crosses. These allow for more accurate coding and movement prediction by dividing the blocks.
Consider a static camera on a roadway. There is no traffic, but then a car enters the scene. Most DVR’s are not configured to automatically place an I frame when the scene starts to change. As a result, the next I frame may be when the vehicle is halfway through the field of view. When certain new information is detected the DVR will newly encode that part of the frame. Prediction will then be used to copy and move that data through subsequent images. How does it move these Macroblocks? Through the use of Motion Vectors!
The Motion Vector
Let’s look again at the videos I discussed during the introduction to this filter in the original blog post.
and now with the Macroblocks and Predicted Forward Motion Vectors visualized…
The small arrows indicate the location of where the block originated from and its new location.
There is a word, often mentioned, and it is very important in Forensic Analysis… “Prediction”. Has an object been predicted?
Take a look at the left edge of the circle as it enters the image. The encoder is predicting this information based on the previous frame.
In this next small example, I can see where the divided predicted blocks near the center originated from.
Finally, after looking at all these colorful blocks – why are they a benefit to us?
By identifying the macroblock type and their history, it may be possible to validate their usefulness when conducting pixel level interrogation for comparisons, height or motion analysis. In contrast, you may decide to not use a specific image as the detail has been created after a number of predictions and the reliability could be questioned.
More recently I have used the tool to explain why enhancement is not possible as it has identified that the blocks covering a required license plate are purely predicted. In contrast, for another vehicle travelling along the same path, enhancement was possible as this time it included an I frame that had captured sufficient detail to progress through subsequent P frames.
Finally, visualizing the blocks and using the add grid filter can assist in authenticating some video as original. If other blocks are seen that are offset or distorted, this can be as a consequence of transcoding and/or rescaling.
Using the Filter
Now that we understand the macroblock types and the importance of the motion vectors, using the filter is actually very simple.
After an original MPEG file has been loaded, the filter can be found under the ‘Verify’ category. Note that if the file has been recompressed or transcoded, the information shown will be relative to the last version, not the original one.
A new file will be created in the format of your choice. You can select to show the Macroblock type and then one of the three Motion Vector visualizations.
After the file is created, a great way to preview the file is by using the Mixer filter.
My advice, as always when learning and understanding new features such as this, is to test it yourself with some MPEG files before you need to analyze footage in a real case.
- 352 x 288 (Mpeg4 Part 2)
- SD H264
- HD H264 with B frames
It is worth highlighting again that this information is only acceptable when you are working on an original encoding. If your file has been transcoded since capture then you will see the block types and vectors from the transcode and not the original.
Keep watching for more posts relating to the big Amped FIVE Update 7620 soon. Subscribe to this blog by email, or use any of the social media links that are at the top of this page to get notified as soon as they are released.