Protecting Evidence: Lossless Data Extraction in Forensic Video Conversion

Reading time: 15 min

This latest post on video conversion explores the critical issue of lossless data extraction from proprietary formats.

Hi everyone, welcome back to this series on Video Formats and Conversion. The feedback so far from this series has been great. We hope the information on video conversion, along with the learning opportunities here has helped you.

In “Protecting Evidence: Lossless Data Extraction in Forensic Video Conversion”, we will be looking at:

1. Video Conversion Purpose: The main goals of video conversion are to enable full analysis of multimedia data and ensure presentation authenticity through restoration and enhancement while preserving evidential integrity.
2. Identification and Extraction: Identifying the video format goes beyond file extensions, as different formats can share the same extension. Correct extraction of video, timestamps, and other metadata is essential to avoid data corruption and ensure accuracy.
3. Container Formatting: After extracting the multimedia data from proprietary formats, it’s placed into an open and compatible container (e.g., MKV). This process preserves data integrity and ensures it can be analyzed and processed further.
4. Convert DVR Options: Amped FIVE provides utilities like Convert DVR to handle various conversion types. Users can choose between copying the stream (lossless) or transcoding (changing the codec), depending on the requirements for analysis or playback.
5. Batch Conversion: Batch Convert DVR allows for processing multiple files or directories at once, saving time and effort when dealing with large amounts of video data from the same source or format.
6. Concatenation: Video files from the same source, such as hour-long clips, can be joined into a single video for easier handling and analysis. This process simply copies streams into a new container without re-encoding.
7. Transcoding: When a proprietary codec prevents direct stream copying, transcoding is necessary to convert the video into a playable format. Transcoding can also be used to make files compatible with specific playback software.
8. Reindexing and Reformatting: These processes are used to fix issues with video navigation or timing, particularly when modern codecs are placed into incompatible containers like AVI. Reindexing or reformatting the video into a more suitable container (e.g., MKV) can resolve several problems.
9. Fallback Procedures: If the proprietary format isn’t recognized by Amped FIVE, fallback methods can be used to extract some frames, providing a quick look to assess if further investigation or acquisition is needed.
10. Encrypted/Proprietary Formats: Completely proprietary or encrypted formats pose significant challenges for forensic analysis, as they may require specific players or passwords. Solutions include analyzing different export options (e.g., screen capture, BMP stills) to obtain the most reliable version of the footage.

Purpose

In the series so far we have looked at proprietary data, and then codecs and formats. We have learned how video evidence, when correctly acquired, may not be structured to facilitate full forensic analysis. We also now understand the importance of video codecs and their link with the container format where they reside.

Using the information we now have, we can investigate how we get from proprietary data to a standard. This ensures we retain the evidential integrity of the visual evidence.

Video conversion is another term and process often misunderstood. You can convert something but at the same time ensure that nothing is lost. Also, after that process, it may then be possible for the data to be more thoroughly analyzed.

Consider a proprietary CCTV viewer application with all the video embedded inside.

We cannot ensure image authenticity or evaluate timing accuracy. Saving individual images from several cameras would take far too long and then those images may have been pre-processed by the viewing application. Even when Export options are available, what are they doing?

We must:

Identify

Identification of the format is not the same as simply reading the file extension. File extensions are not controlled. Any developer can name their file type with any extension. Consequently, there could be several different formats, requiring very different data extraction techniques, but all having the same file extension.

Extract

The first thing to extract is the unformatted video. Rather than one big chunk of data, where it may be all mixed up, we need to separate the camera streams. This ensures that other videos, the date and time information, or other metadata do not corrupt and interfere with the correct decoding.

Next, we have the date and time information. This must not only be extracted but referenced to the frame numbers in the video.
With other formats, there could be audio data too.

Format

After the data is extracted, it must be formatted correctly, using an open standard. The video therefore may be placed into a compatible container, such as Matroska (MKV). For the date and time, our .time file format is simply open text with Frame Number > Frame Type > Date Time. We will take a closer look at this during the next post on decoding.

Decode

The final stage is the presentation of that data, such as loading it into the viewer within Amped FIVE.

Video Conversion

Video conversion has two main aims:

1. To enable the complete analysis of the multimedia and associated data (time).
2. To enable the further processing of the multimedia, ensuring presentation authenticity through restoration and enhancement.

There are several other reasons why it may be necessary to convert a video file. When starting with a standard codec and format, there may be a requirement to change it to facilitate viewing or transmission.

Also, there may be errors within the file caused by its writing method or formatting. Moving the stream into a new container format may correct the errors.

We will go through some of the more common conversion processes using various examples. This breakdown will help you with this common stage of Forensic Video Analysis.

We will be using Convert DVR within Amped FIVE. You may remember from a previous article, we briefly looked at this message.

Whenever you drag in a file to Amped FIVE, it gets analyzed. If the file is unknown, or the data matches a known proprietary format, this message will appear.

You can use Convert DVR directly if you wish, from the Utilities menu.

The message box above gives you various pathways.
The first is the ability to analyze the file before you make any decision on conversion. You can open the file in Advanced File Info, and then decide on your next actions. For example, if you remember in the last post on codecs and formats, we looked at an AVI that contained a video using a proprietary codec. If you dragged an AVI into FIVE, and this box appeared, it may be worthwhile understanding what’s inside it first, before moving onto the conversion stage.

Next, we have three options.

1. Yes (Choose Settings) – This opens up Convert DVR so you can select or check the settings.
2. Yes (Last Settings) – This bypasses the opening of Convert DVR and starts the conversion process using the last settings entered.
3. Attempt direct loading and bypass any format analysis and stream extraction.

We will be looking at direct loading in the next post in this series. For all the files, we will be selecting – Yes (Choose Settings).

Single SEC File

The evidence consists of a single .sec file, along with a video player. The file is loaded into Convert DVR automatically after selecting the option in the previous message.

There is an option to change the initial output filename if required. Other data, however, will be added to the output filename during the video conversion process, so it may not always be required.

There are a couple of dropdowns. We will look at those further throughout this post, but for this file, the settings are correct.

The “Conversion Type” option is important.
Copy Stream means exactly that. Copy the multimedia stream, do not change it.
Transcode means change it.

So, “Copy Stream if possible, or else Transcode”, means don’t change it unless there is no other option.

We can now move to the next tab.

Also, we need to decide what format to place our copied stream into.
We learned all about formats in the previous post, and you can see that they are all selectable here.
There are a couple of others designed for some edge cases, but you should recognize most of these.

As we have started with a proprietary container, and we wish to extract the video stream into an open format, we will select MKV. We will look at the other tabs a bit later. For now, we can press OK, and let the Amped Engine do its job.

The first stage is the identification of the format. There are several indicators within the file, and these allow Amped Engine to correctly navigate the data and identify what is required.

Next, we have the extraction of the data. This is a key stage and complies with any legal requirements for “The Lossless Extraction of Data from Proprietary Formats”.
This data is placed into separate files, such as video or time data.

We learned previously that multimedia should be formatted correctly and containerized. This is the next step.
We have extracted the video stream from the proprietary container and thereby removed the risk of data contamination during the formatting process. Now, we can turn to a multimedia framework for this task.
We use FFmpeg to mux the extracted stream(s) into the selected container format.
A log of this formatting is also retained.

Finally, the files can be decoded. The default action after formatting is to load the files into Amped FIVE for viewing, analysis, and processing.

From this single proprietary file that does not include audio, four other files have been created.

1. Extracted Video
2. Extracted Timestamp
3. Formatted MKV
4. Format Log

When loaded into FIVE, you will notice that the .time file is also loaded as usable data within the chain.

Another thing to notice is the filename. Appended to the original name is -sc-converted. The SC stands for Stream Copy. The file will also have the .MKV file extension.

Single DAT File

This time we have a single file with a .dat file extension. Many files originate from surveillance systems with this file extension, but they can have very different formatting! Keeping the same settings, and just converting this file to MKV with Stream Copy, reveals multiple cameras inside the same file.

Along with the 8 extracted video streams, we have extracted an audio stream, and also the data timestamps.

This allows us to look at the audio settings under Convert DVR briefly.

If audio is identified, it will either be placed directly into a compatible container, or will be transcoded to a lossless format. The transcoding is often completed to allow resampling. This ensures synchronisation with the video even if audio samples had been dropped during the original encoding process.

Before we move on, let’s look back at the video streams in this single .dat proprietary container. In a previous post in this series, we learned that if the streams are not extracted correctly, then it may be possible to decode. However, the presented frame may be referencing the wrong previous frame.

In the image below, we have used another player that did not identify the streams.

The video appears corrupted. In reality, it just needs the Amped Engine to extract everything correctly before formatting it all for accurate decoding.

Finally, let us consider another type of proprietary data. This is the executable type we mentioned at the start of this post. There are many like this, with the video player and all the video streams wrapped up inside a self-launching application. Our process is the same. Drag it in, or load it into Convert DVR and let the Amped Engine analyze, identify, extract, and then format.

Multiple Files

There are several ways of managing multiple files, and they can be either proprietary, standard or even both. Your decisions will be based on the structure you are starting with and perhaps any further requirements.

Here we have a directory of files. These are all from the same location and extracted during the same acquisition.

After dragging the first one into FIVE, and selecting Convert DVR, we can use the “Files to Convert” dropdown option. From there we should select “All files with the same extension in the same directory”.

This will then mirror any selected settings and apply those to every file in that directory.

But what if we had multiple directories or multiple file types?
This is where Batch Convert DVR comes in.

Batch Convert DVR allows much more flexibility in your file management.
As you can see, you can select an initial directory or drive, and simply select the files you require to be converted using the checkboxes.

Next, you can select the Output Folder Structure. Think back to all our Dat files earlier. Using Batch Convert DVR would allow you to place each output into a single folder. This could be useful if each Dat file had 16 streams!

Finally, as shown above, you can convert, whilst removing all the files from folders. All the files get prepended with the original folder name. This ensures individualization of the filenames if the directories contain different files with the same name. This comes in useful when you need to complete further processing on a specific camera for instance.

In the first pass, you may want to convert, individualize, and place all into a single folder.
In the next pass, you may want to take all the converted hour files related to a single camera and concatenate them into a single file.

You can learn more about Batch Convert DVR in this Amped FIVE update blog post.

Concatenation

Here we have 4 standard files.

Each one has a single H264 video contained within the MP4 format. The filename relates to the hour of time of each file, so 15 starts at 15:00 hrs.

By selecting the concatenation option within Convert DVR, we can place all of these into a single MP4 container with the video lasting 4 hours. It’s very quick as the streams are simply copied one after another into a new container.

When concatenating:

1. Use the same output container if possible to reduce the risk of any formatting differences. In this case, MP4.
2. If the source files are proprietary, conduct some analysis on a couple of video conversions first to ensure matching parameters. You cannot concatenate files with different resolutions or frame rates.
3. If Transcoding, supplemental files may be required to be created first. Convert DVR will manage this process if required.

Transcoding

So far, for most of the examples here, we have Stream-Copied. Remember, that most of our written files will then have the “sc” in the filename to denote this.

There are times though when we need to change the codec, and this is called Transcoding. All files will then have “tc” appended to the filename.
There are two main reasons why Transcoding would either be the only option available or be selected by you as a choice. The first is when certain proprietary codecs have been detected within the source file. Some of these can be decoded, but are unable to be placed into a new container due to their structure. If Convert DVR detects this issue, then Copy Stream will not be available as a Conversion Type.

Remember that if you need to conduct analysis, restoration, or enhancement on the resulting file, select a lossless codec such as FFV1.

The next is that you simply need to get the files playable by someone else, on another system, and they are restricted in the software they have. In this case, it will be you, the user, deciding to Transcode.

Within the Transcode tab, you have all the options available for Format, Codec, Quality and Hardware Acceleration.

We looked at hardware acceleration in the previous post when writing a video, but here we have another option. You may remember that when writing a file, it can only be completed as fast as the pixels can be created. As such, if intensive filtering has been applied, there may be no benefit of using hardware.

However, within Convert DVR, this is a simple in > out process. As such, the benefits of hardware acceleration can be huge.

The added Intel Quick Sync Video Encoder here utilizes the full benefit of Intel CPUs. It would benefit departments to run tests to see what acceleration method is best on your workstations if you have both CUDA GPUs and Intel CPUs.

Fallback Procedures

The final tab we will look at is Fallback Procedures.

We understand the importance of the early stages of an investigation and the frustration that arises when you have video evidence that Amped FIVE does not recognize.

As we learned earlier in the series, manufacturers tend not to consider the forensic requirements of the video they create. Therefore, there is no list of CCTV formats. It’s not even possible to accurately list CCTV extensions as even these are not registered, with multiple different formats using the same one. We can only add support to Amped Engine by working together.
If you receive a format or conduct an acquisition, and obtain formats that are not supported, please let us know. It not only helps you but the entire Amped community worldwide.

In that interim stage, you do have some options though. It is sometimes possible to get some frames out of unknown proprietary formats. This may be enough for you to identify that a further acquisition is required, or that the camera’s field of view would not have captured an incident.

There is a full blog post on Fallbacks we suggest you read. This post includes all the methods of skipping bytes to known video headers and the creation of the configuration file so it can be saved, shared or utilized within Batch Convert DVR.  

Fixing Scrubbing Issues

In the next post in this series, we will look at video decoding within Amped FIVE. When a video is loaded, and you attempt to navigate or scrub the video within the Player bar, you may sometimes observe freezing or be unable to go directly to a specific frame number.

With many CCTV exports utilizing a standard container, it is rather common for issues to arise during the formatting process. It could be that the option for AVI, or ASF is selected on the DVR, instead of the native proprietary format. Issues may also occur when a network or mobile device is used and the formatting is completed after the transmission process. The system does not transcode the footage, but as it places the original stream into a standard container format, errors can occur. Timing often becomes a problem during this process but also, sometimes the indexing can get a little mixed up.

We see this most commonly when modern codecs, such as H264 or H265, are used by the recording device but are placed into the AVI container. As we learned previously, this can cause a few issues.

Within the Video Loader, there is a button to load the file directly into Convert DVR.

ReIndex

Re-indexing is the simple case of taking the stream out of the container and placing it back into a matching container format. So, for a HEVC video inside an AVI, you simply Stream Copy into another AVI. In the muxing process, a new index is written automatically within the container.

ReFormat

Re-formatting is, as you may have guessed, the process of taking the standard stream out of a standard container and placing it inside another compatible container format. Therefore, our HEVC video inside the AVI gets moved into an MKV container. In this process, it is newly formatted.

There is another very common reason for reformatting. We have talked a lot about modern codecs being placed into the AVI container format. One of the (many) downsides to this is that although the frame timing may be in the file, the container format is unable to present that information to many analysis tools.

We use the frame analysis functionality within FFprobe to ensure transparency and cross-tool verification. We also run calculations on the data and scan the output for inconsistencies, such as missing coded picture values.

In the frame analysis above we have HEVC that was forced into the AVI container by the system manufacturer at the time of acquisition. The PTS data is unable to be read from within the container. This results in the N/A values seen, and no calculations are therefore possible. However, taking the stream out of the container, and placing it inside another more suitable one will usually ensure the data is readable.

Both reindexing and reformatting are very quick processes that make no changes to the stream data. They can provide a swift solution to problematic frame navigation or analysis.

Unable to Convert

We have discussed, over several posts, many of the challenges that surveillance video brings. Then, we have looked at the solutions available in the lossless extraction of data, followed by the various video conversion methods discussed here. We have also looked briefly at fallback procedures in an attempt to obtain some workable video whilst you send the format to us at Amped Software.

There is one final group that must be mentioned. These are completely proprietary and/or encrypted. These cause huge problems within the legal world. Passwords get lost. Applications that control the decryption and playback may need to be installed, requiring Sandbox or Virtual Computer environments. The video cannot be extracted, analyzed, or interpreted correctly or easily. Some may offer a form of video export, but they will often transcode it, thereby changing the data completely.

As we have learned, even the proprietary players may not display the footage correctly. Therefore, a screen recording may only give you a version of the video.

There is no simple solution to this problem. Reporting on the limitations of any “version” of the video may suffice, but establishing what the best version is will take research and testing.

• BMP still images
• Compressed video export
• Uncompressed video export
• Screen capture (lossless)

All of these may give you slightly different results, that can be compared and contrasted using Video Mixer in Amped FIVE. It may then be required to use different versions to answer different questions or complete other tasks. 

With encryption, it’s sometimes easier to reacquire the footage with the DVR encryption checkbox unchecked! However, with complete proprietary footage, that is not possible. Luckily, they are now few and far between. Working through the options, testing, and then reporting on the limitations is often the easiest solution.  

Finally

File analysis, processing decisions, and then video conversion is a vital first step in an investigation. Managing all the source data and making the preparations to enable the investigation to continue can take time. Sometimes individual files need to be converted first before further decisions are made on entire directories.

Spend some time to learn all the options, get to know what is within each tab and the selections available within the dropdowns.

The options available to you with Convert DVR, and then Batch Convert DVR, allow full control of the data inside proprietary formats. No need to export from poor player applications or screen capture a version of it. Amped Engine is designed specifically to get you the data you need quickly and accurately.

In the next post in the series, which will also be the last, we will look at the next stage – decoding. It’s not just a matter of hitting play, you have options on how to play it!

Until then, stay safe.