Dear friends, welcome to this week’s video evidence pitfall! In this post, we’re focusing on a crucial element of forensic video analysis: timestamps. Timestamps allow us to locate in time what’s shown in a recording, or reference an event to a specific moment in time. Although virtually all surveillance systems do record timestamps, you should be aware of several pitfalls in accessing and interpreting them, so keep reading!
Issue: Timestamps May Be Off, Wrong or Missing
All investigators know that finding “when” something happened is one of the most important elements of their work. Luckily, this information is typically provided by most video surveillance systems, and also by most consumer devices such as smartphones, cameras, etc. However, things are never as smooth as they seem.
Take a look at the example below: we have three different timestamps for the same frame! Two are overlaid over pixels, and their time differs by several hours. Plus, Amped Replay was able to decode the proprietary timestamp embedded in the file, which is shown over the player bar (number 3). The difference between timestamps 1 and 3 is one second, which can be meaningless or paramountly important, depending on the kind of investigation. How did we get there? And which one should we trust, if any?
- This data appears to have been written into the video using a common surveillance font and design.
- This data appears to be an overlay, however it also has been written into the video image.
- This data is being read from the date and time information held within the file.
The most common explanations for different visual timestamps are IP Cameras and Files exported using a player or client. Firstly an IP camera may encode a date and time onto the data, and then when that data gets placed into a file, the DVR adds another. Next we have the scenario where a video is exported using a Video Management System. This may add on the date and time information from the system and transcode the video as a new file.
Here is another example: the exhibit is a proprietary video file and Amped Replay has managed to decode and display it. Unfortunately, however, we don’t see any timestamp information anywhere.
Explanation: Proprietary Formats and Lack of Documentation Make Everything Hard
Let’s try to put some order in timestamp issues. First of all, the information can be provided in several ways:
- Some systems may simply overlay the date and time over pixels in the recorded frames. This means you lose forever the video content that was below the timestamp (it’s replaced by the date and time), but at the same time, it ensures that if you can play the video then you can also view the timestamps. Also, the association of timestamps to frames is done once and forever by the recording system, which is good.
- Most systems will store timestamps as metadata inside the video file. Unfortunately, however, they will not be put there in a standard way most of the time. In such cases, the DVR’s proprietary player will be able to decode and display timestamps, while any other software will have to “reverse engineer” the proprietary file to locate and decode them. “That’s great, then: we can just use the player!”, I hear you say. Well, true, but the player often fails in showing the actual recorded pixels. So you can use the player for decoding timestamps, but you still want to consider directly importing the proprietary file in your analysis and enhancement software, if the software allows. We’ll see later how you can handle this case with Amped products.
- Some systems store metadata in a dedicated subtitle track.
So, just as it happens for video frames, the first issue is being able to find and decode metadata. But once this is done, the question turns to: can I trust them? Well, several comments are in order…
- DVRs often rely on an internal system clock that is set by the user and does not update automatically. We all know that digital clocks, especially those in cheap hardware, naturally drift over time, so if the shop’s owner had set up the system in 2012 and never touched it again, chances are that the clock is off by a sensible amount. Not to mention power-offs: the internal battery of the DVR (if there’s one) may be faulty after years of service, so a single shutdown may lead to timestamps being reset to some “default initial date” decided by the manufacturer.
- Then we have time zones: are the timestamp values provided by the DVR stored in GMT format? Do they provide a value for time zone compensation? Are they stored in the local time zone? This information is often left to your imagination!
- In time-critical cases, such as collision investigations or speed analysis, timestamp precision and accuracy is also a concern. Do timestamps reach the millisecond level, or do they stop at seconds? And can we trust them?
Solution: Document During Acquisition, Use the Right Tools for Analysis
As usual in forensics, you need three things to reach a sound result: evidence, competence, and tools. What does all of that mean in our specific case?
“Evidence” means that if the information is not available at all, there’s not much you can do besides resorting to different kinds of analyses. So if the DVR is not writing timestamps you may try to obtain a timestamped recording from another surveillance system nearby, then look for events that allow synchronizing the two recordings. It will not be millisecond-accurate, but better than nothing. More fancy ways could involve techniques such as Electrical Network Frequency (ENF) analysis.
“Competence” means that you must know what you need to do and how to do it. First and foremost, when you reach a DVR, always remember to write down as many details as possible, which of course includes the DVR’s clock value vs the real acquisition moment as available from reliable sources (e.g., this one).
“Tools” means that you need the proper hardware and software to work as a professional. Using Amped products, you can directly import the proprietary files generated by most DVRs. For many formats, our products will also decode and store the embedded timestamps, and load them automatically in your project. The picture below shows what we obtained by simply dragging the .exe file into Amped FIVE and letting it do the rest: video and timestamps were extracted from the proprietary (executable!) file and both loaded.
When proprietary timestamps cannot be extracted, you still have a backup solution: use the proprietary player to display the timestamp of the first and last frames, and manually provide such information to Amped FIVE using the Add Timestamp filter. Everything will stay tidy and well documented.
We could go on for pages and pages talking about timestamps, and still, we would not cover all the possible variants (scary word nowadays) and issues. But hopefully, what we’ve said is enough to raise awareness.