Infrared: Don’t Trust the Colors of Most CCTV Footage at Night

Dear friends, ready for this week’s Video Evidence Pitfall? Today we’re talking about infrared (IR) images, how they could be misleading, and some potential IR-related issues that may involve even “normal” videos. Keep reading!

Issue: you can’t trust any color in infrared images

How many times have you seen this kind of “washed out” image?

These are typical infrared images, and, as we will see, you should not use them to get any color information. To convince you, let me show you a couple of pictures. On the left, you have the visible light version of a shirt, on the right the IR version of the same shirt in the same scene and same everything.

I think the pictures above are quite self-explanatory! So, if you’re watching an IR video and you see a car passing by, the fact that the car appears bright or dark in the video is not telling you anything about the color of the car. Too many times I’ve heard investigators say “and then at minute 3:12 you see that guy wearing a dark jacket passing by”. Hopefully, they meant that “the jacket appeared dark in the IR video”, and they did not restrict searches to guys wearing a real dark jacket 🙂

Notice that, from the image gallery above, it emerges that you should not even get “relative” information about colors. I mean that looking at an IR image, you cannot even say “regardless of the real color, this object is at least darker than this other one”.

Explanation: what is infrared?

What we call “infrared” waves are just a part of the electromagnetic spectrum, just like what we call “visible light”, X-ray, gamma rays, radio waves, etc.

Original author: Philip Ronan, Gringer

Now, as you can see in the figure, the infrared part of the spectrum is way larger than the visible light part. Indeed, there are further sub-divisions of the infrared spectrum: we have near-IR, short-IR, middle-IR, etc., as shown below:

Source website: https://sites.google.com/a/globalsystemsscience.org/digital-earth-watch/

Now, as far as our normal “night vision capable” CCTV cameras are involved, we normally deal with near-IR waves, which are reflected by objects, similar to visible light, and are then captured by the camera sensor. (As a side note, there exist also cameras capturing mid-IR and far-IR, but they work completely different: they capture radiations emitted by objects due to their heat and are thus called “thermal cameras“.)

As the pictures above show, near-IR waves are a different part of the spectrum than visible light. You’ll agree, then, that IR waves do not bring any information about the “color” of objects, they are more likely a “different color” themselves! That explains why you should never make any conclusion about the color of objects based on what you see in an IR image. It would be as meaningless as saying that an object is pink because you can cook it in the microwave.

So, until now, we have quite a clear justification for our takeaway rule: don’t get any color information from infrared imagery. But there’s one more thing about infrareds that is worth sharing.

Near-IR waves don’t need a special camera sensor to be captured: the normal imaging sensor that we also have in our smartphone would capture IR waves as well. To avoid them, camera makers put a filter on top of the sensor that stops IR light, so that the sensor works similar to our eyes and capture just visible light. When you have a CCTV camera that works in both “day mode” (visible light) and “night mode” (infrared), this is typically obtained by using a switching filter: at daytime, the filter is put in front of the sensor so IR light is blocked, and you see normal colors (notice I’ve said “normal”, and not “real”: we’ll discuss in a separate post how much colors get killed by video compression in CCTV systems). At nighttime, the filter is removed and the camera captures IR light (indeed, this kind of camera often has LEDs around the lens that throw infrared light in the scene: you can’t see it, but the camera can).

Original image by Amirecoy

Now, it may happen that the filter’s switching mechanism breaks or the filter becomes defective (you wouldn’t envy a surveillance camera’s life, working 24/24 with rain, sun, ice, etc). If this happens, it may be that when working in daylight mode the camera sensor captures a mixture of both visible and infrared light! Unfortunately, this has a devasting effect on the reliability of colors, and this is quite dangerous since you think you’re seeing the normal color of objects, but they’re not reliable at all. Here is an example:

Detail of an image from https://raspi.tv/2013/pinoir-whats-it-for-comparison-of-raspicam-and-pi-noir-output-in-daylight

In a case like the one above, it’s rather easy to understand something could be wrong because we have a prior expectation about how trees should normally look. Unfortunately, there’s not a general rule to understand when you’re in a visible+IR light case. It’s just something to keep in mind as a possible explanation for strange effects such as “hey, why does everyone in this video seem to be dressed in the same color!?”.

Solution: you can still make good use of IR images

Before we say goodbye, let me share a piece of good news: while IR imagery is not good for understanding the color of objects, you can still make good use of it.

First of all, you can, of course, compare the appearance of objects in several IR images captured by the same camera. Besides that, however, there’s also a safe way of comparing IR and visible light images! You simply have to rely on something that is well captured in both domains. Let’s consider again the picture below: notice how the different fabrics are even more distinguishable in the IR image than in the color image. The shape and position of such elements can be soundly used for comparison, although they appear different because of different image formation modes.

That’s all for today! We hope you’ve found this issue of the Video Evidence Pitfalls series interesting and useful! Stay tuned and don’t miss the next ones. You can also follow us on LinkedIn, YouTube, Twitter, and Facebook: we’ll post a link to every new tip so you won’t miss any!