Skip to main content

Experimental Validation of Amped Authenticate’s Camera Identification Filter

Marco Fontani

Reading time: 10 min

We tested the latest implementation (Build 8782) of PRNU-based Camera Identification and Tampering Localization on a “base dataset” of 10.069 images, coming from 29 devices (listed in the table below). We split the dataset in two:

  • Reference set: 1450 images (50 per device) were used for CRP estimation
  • Test set: 8619 images were used for testing. On average, each device was tested against approximately 150 matching images and approximately 150 non-matching images.

Contents hide
Experiment 1: Source Device Identification on the Base Dataset
Experiment 2: Camera Identification in Presence or Rotation, Resize and JPEG Re-compression
Experiment 3: Camera Identification in Presence of Cropping, Rotation and JPEG Re-compression
Experiment 4: Discriminating Different Exemplars of the Same Model
Experiment 5: Investigating the Impact of the Number of Images Used for CRP Computation

It is important to understand that, in most cases, we could not control the image creation process. This means that images may have been captured using digital zoom or at resolutions different than the default one, which makes PRNU analysis ineffective. Making use of EXIF metadata, we could filter out these images from the Reference set. However, we chose not to filter out such images from the Test set: we prefer showing results that are closer to real-world cases, rather than tricking the dataset to obtain 100% performance.

Using the above base dataset, we carried out several experiments:

  • Experiment 1) testing the system on images “as they are”
  • Experiment 2) camera identification in presence or rotation, resize and JPEG re-compression
  • Experiment 3) camera identification in presence of cropping, rotation and JPEG re-compression
  • Experiment 4) discriminating devices of the same model
  • Experiment 5) investigating the impact of the number of images used for CRP computation.

Test results are shown using Receiver Operating Characteristic (ROC) curves and their Area Under the Curve (AUC). If you’re not familiar with ROC and AUC, you may want to take a look here.

Table 1: complete list of devices in the dataset. By clicking on device name, you will be sent to the corresponding page on DPreview.com

MakeModelTypeRelease yearResolution
AppleiPhone 4Phone20115MP
BenQDC4330Compact?3MP
CanonPowershot A75Compact20043MP
CanonPowershot s2isCompact20055MP
CanonPowershot SD630Compact20066MP
CanonIXIUS v2Compact20022MP
CasioExilim ex-z60Compact20066MP
CasioExilim ex-z75Compact20077MP
CasioExilim ex-z70Compact20067MP
DigCamSub-6MPCompact? 
FujifilmFinepix A350Compact20055MP
HPPhotosmart 320Compact20022MP
Nikone-coolpix-s570Compact200912MP
NikonD70DSLR20046MP
NikonD80DSLR200610MP
NikonD100DSLR20026MP
NikonD300DSLR200712MP
Nikone2500 (Coolpix 950)Compact19992MP
Nikone885 (Coolpix 885)Compact20013MP
NikonE7900 (Coolpix 7900)Compact20057MP
NikonE880 (Coolpix 880)Compact20003MP
Nokian95Phone20075MP
Olympusu760Compact20077MP
OlympusE300DSLR20048MP
PanasonicLumix dmc-tz3Compact20077MP
PanasonicLumix dmc-fs15Compact200912MP
PentaxOptiosCompact20033MP
PremierDC-3320Compact20013MP
SonyCybershotCompact?2MP

Experiment 1: Source Device Identification on the Base Dataset

Using the Base dataset, we ran experiments and stored the obtained PCE value for each matching case (that means, comparing an image to the CRP of its originating device) and each non-matching case (we were comparing an image to the CRP of a different device). Figure 1 shows how PCE values are distributed for the non-matching cases and matching cases. We see that PCE values are higher than 100 in most matching cases, while they spread in a seemingly exponentially decreasing way between 0 and 25.

Bar chart displaying the distribution of PCE (Peak Correlation Energy) values used in PRNU-based camera identification. The x-axis represents PCE value ranges, and the y-axis shows the number of images. Red bars indicate non-matching images with low PCE values clustered near zero, while green bars represent matching images, with a large spike in the final bin labeled “[100, ∞)”, indicating strong correlation. The chart visually separates authentic camera matches from mismatches based on statistical thresholds.
Figure 1: histogram of PCE value for matching cases (red bars) and mismatching cases (green bars)

Intuitively, what we need to do is to set a threshold for the PCE value. Then, when comparing an evidence image to the CRP that supposedly generated it, we’ll compare the obtained PCE against the threshold: if it’s above the threshold, we’ll say the image is compatible with the device, otherwise not. Using our experimental data, we can decide where to set the threshold. Choosing too low values will result in “compatible!” quite often, so we’ll have many true positives (which means that a matching image is correctly classified as matching), but we’ll also have many false positives (non-matching images erroneously classified as matching).

Choosing too high values would bring the opposite problem: we’d remove many false positives, but we’d also lose many true positives. This is where the ROC curve comes in handy. We let the threshold span a large set of possible values and annotate the true positive and false positive rates associated with each value. Then, we consider false positive rate versus true positive rate for each threshold value, as in Figure 2. We see that for a threshold value of 26, the classification algorithm yields a 94% true positive rate for a 0.1% false positive rate. Keeping in mind that the Test dataset contains images that may have been zoomed, this result is undoubtedly positive overall.

Receiver Operating Characteristic (ROC) curve for PRNU-based camera identification, showing performance across 2,695 positive and 2,900 negative image tests. The x-axis represents the false positive rate, and the y-axis shows the true positive rate. AUC (Area Under Curve) is 0.99, indicating high accuracy. A red threshold marker at PCE = 26.0 is annotated on the curve, highlighting the decision boundary used to separate matching from non-matching results.
Figure 2: ROC curve for the base dataset

Table 2 shows the performance achieved by the system on each device, using 26 as the threshold. We notice that the Panasonic Lumix DMC-TZ 3, together with the Nikon Coolpix S 570 and the outdated Premier DC-3320 devices are the most critical to the system (true positive rate below 90%). For other devices, however, performance is greatly appreciable. Noticeably, on our Test dataset, the true negative rate never gets lower than 97%, and it is 100% for 17 devices.

MakeModelSource identification performance
AppleiPhone 499% (TP: 98 | TN: 100)
BenQDC433099.5% (TP: 100 | TN: 99)
CanonPowershot A7598% (TP: 98 | TN: 98)
CanonPowershot s2is100% (TP: 100 | TN: 100)
CanonPowershot SD63099% (TP: 100 | TN: 98)
CanonIXIUS v2100% (TP: 100 | TN: 100)
CasioExilim ex-z6090.9% (TP: 82 | TN: 100)
CasioExilim ex-z7593% (TP: 91 | TN: 95)
CasioExilim ex-z7095.9% (TP: 92 | TN: 100)
DigiCamSub-6MP97.0% (TP: 95 | TN: 99)
FujifilmFinepix A350100% (TP: 100 | TN: 100)
HPPhotosmart 32099.5% (TP: 99 | TN: 100)
Nikone-coolpix-s57084.3% (TP: 69 | TN: 100)
NikonD7099.5% (TP: 100 | TN: 99)
NikonD8099.5% (TP: 99 | TN: 100)
NikonD10099.0% (TP: 100 | TN: 98)
NikonD300100.0% (TP: 100 | TN: 100)
Nikone2500 (Coolpix 950)97.0% (TP: 94 | TN: 100)
Nikone885 (Coolpix 885)98.0% (TP: 96 | TN: 100)
NikonE7900 (Coolpix 7900)98.5% (TP: 100 | TN: 97)
NikonE880 (Coolpix 880)98.5% (TP: 100 | TN: 97)
Nokian95100.0% (TP: 100 | TN: 100)
Olympusu76098.0% (TP: 98 | TN: 98)
OlympusE30097.0% (TP: 100 | TN: 94)
PanasonicLumix dmc-tz376.8% (TP: 54 | TN: 100)
PanasonicLumix dmc-fs1592.4% (TP: 87 | TN: 98)
PentaxOptios100% (TP: 100 | TN: 100)
PremierDC-332084.8% (TP: 70 | TN: 100)
SonyCybershot100% (TP: 100 | TN: 100)
Table 2: device specific source identification performance

Experiment 2: Camera Identification in Presence or Rotation, Resize and JPEG Re-compression

In this experiment, images are processed and saved as JPEG before entering the source identification algorithm. Processing includes a combination of resize, rotation and lossy JPEG compression. The parameters characterizing each operation are shown in the following table.

ProcessingParameters
ResizingBilinear algorithm, resize ratio randomly sampled between 50% and 150%
RotationRandomly chosen between 90, 180, 270 degrees
JPEG compressionRandomly chosen between [60, 70, 80, 90, 100] (standard JPEG quality)

The overall performance is shown in Figure 3: while there is a sensible loss in terms of true positive rate compared to test on the Base dataset, the source identification system is still reliable, with an AUC of 0.94, and a true positive rate over 80% when allowing only 1% of false positive rate. If we stick to the same threshold for the previous experiment (26), the system yields 81.5% true positive rate for a 0.2% false positive rate.

ROC curve illustrating PRNU camera identification performance on 2,689 positive and 2,900 negative images with varying resolution (0.5x to 1.5x), rotation (90° to 270°), and JPEG compression (quality 60 to 100). The plot displays the true positive rate vs. false positive rate, with an AUC (Area Under Curve) of 0.94, indicating strong detection accuracy. A red threshold line at PCE = 20.0 denotes the decision threshold used for classification.
Figure 3: source identification performance in presence of resize, rotation and re-compression

In order to understand the impact of each processing, we first restrict the analysis to those images that did not undergo any resize. Results are in Figure 4. We see that the AUC increases, and we get a true positive rate of 92% for a false positive rate of 1%. Since these values are very close to those in Experiment 1, we can argue that rotation and re-compression do not hinder significantly the performance of the source identification system.

ROC curve for PRNU-based camera identification using 229 positive and 300 negative images, with consistent resolution (1.0x), image rotation between 90° and 270°, and JPEG compression quality between 60 and 100. The graph shows the true positive rate against the false positive rate, achieving a high AUC (Area Under Curve) of 0.99. A red line marks the classification threshold at PCE = 30.0, indicating excellent discrimination performance.
Figure 4: results in presence of rotation and re-compression only

Now, let’s compare results when images are resized, rotated and compressed at slight or moderate quality (Figure 5). We notice that stronger JPEG compression negatively affects the performance of the system, leading to approximately a 10% loss in terms of true positive rate.

ROC curve evaluating PRNU camera identification performance on 1077 positive and 1100 negative images, with varied resolution (0.5–1.5x), rotation (90°–270°), and JPEG compression range (90–100). The plot shows true positive rate versus false positive rate, highlighting a classification threshold at PCE = 32.0 in red and achieving an AUC (Area Under Curve) of 0.96, indicating high model accuracy in distinguishing source camera.

ROC curve for PRNU-based camera source verification on 1612 positive and 1800 negative images with resolution scaling (0.5–1.5x), rotation (90°–270°), and JPEG compression range (60–80). The graph plots true positive rate versus false positive rate, indicating a classification threshold of 20.0 in red and an AUC (Area Under Curve) of 0.93, reflecting strong identification performance despite lower image quality.
Figure 5: results in presence of resize, rotation and slight (top) or strong (bottom) final compression

We find it interesting to compare the effect of down-scaling (top plot in Figure 6 to that of up-scaling (bottom plot in Figure 6). It is evident that device identification is harder to achieve when the image is downscaled (73% true positives for 1% false positives), while upscaling does not affect performance significantly (89% true positives for 1% false positives).

ROC curve illustrating PRNU-based camera identification accuracy on 1,459 positive and 2,100 negative images under transformations including 0.5–1.0× resolution scaling, 90°–270° rotation, and JPEG compression (60–100 quality). The red threshold line is marked at 20.0, and the AUC (Area Under Curve) score is 0.91, indicating high model performance despite moderate compression and resolution variability.

Receiver Operating Characteristic (ROC) curve showing PRNU camera identification results for 1,459 positive and 1,100 negative images with transformations including 1.0–1.5× resolution scaling, 90°–270° rotation, and JPEG compression levels from 60 to 100. The red threshold line is set at 21.0, and the AUC (Area Under the Curve) score is 0.98, indicating excellent classifier performance under these moderate transformation conditions.
Figure 6: comparison between down-scaling (top) and up-scaling (bottom)

As to the rotation factor, Figure 7 shows that it is rather uninfluential, meaning that different rigid rotations do not influence the accuracy of the system.

ROC curve for PRNU-based camera identification using 917 positive and 1,100 negative images with transformations including resolution scaling from 0.5 to 1.5, fixed 90° rotation, and JPEG compression range of 60–100. The threshold is set at 20.0, and the AUC (Area Under the Curve) is 0.93, indicating strong detection performance under moderate rotation and compression scenarios.

Receiver Operating Characteristic (ROC) curve displaying PRNU-based source camera identification performance with 894 positive and 1,000 negative images. Transformations include resolution scaling (0.5–1.5), 180° image rotation, and JPEG compression between 60–100. The threshold is set at 32.0 and the AUC (Area Under Curve) is 0.93, showing reliable classification despite challenging rotational distortion.

ROC curve showing PRNU camera identification performance with 878 positive and 800 negative images. Conditions include image resolution between 0.5 and 1.5, 270-degree rotation, and JPEG compression levels from 60 to 100. The curve highlights a threshold of 20.0 and an AUC (Area Under Curve) of 0.95, indicating high accuracy despite extreme rotational transformations.
Figure 7: impact of rotation factor

Experiment 3: Camera Identification in Presence of Cropping, Rotation and JPEG Re-compression

This experiment is similar to Experiment 2, but resize is substituted by crop. Images are processed applying cropping, followed by rotation and final JPEG compression. Parameters for each processing step are as in the following Table.

ProcessingParameters
CroppingFrame cropping, randomly chosen in the interval 0-20% of the image size
RotationRandomly chosen between 90, 180, 270 degrees
JPEG compressionRandomly chosen between [60, 70, 80, 90, 100]

Results are shown in Figure 8. We see that performance is in line with those of the resize case (see Figure 3).

ROC curve for PRNU-based camera identification showing results from 2,689 positive and 2,900 negative images. Test conditions include image rotation between 0° and 270°, cropping up to 20%, and JPEG quality between 60 and 100. The curve shows a red threshold line at 27.0 and an AUC (Area Under Curve) of 0.96, indicating strong classification performance under cropping and compression variations.
Figure 8: results in presence of cropping, rotation and compression

Similar to the previous case, the final JPEG compression plays an important role: when the last compression is at good quality (left plot of Figure 9), the effect of cropping on performance is very limited. On the other hand, when the compression is moderate (right plot of Figure 9) the system loses around 10% in terms of true positive rate.

ROC curve for PRNU-based camera identification using 1,107 positive and 1,600 negative images, with test conditions including cropping up to 20%, image rotation from 0° to 270°, and JPEG compression quality between 90 and 100. The chart highlights a threshold value of 32.0 in red and an AUC (Area Under Curve) of 0.98, demonstrating high classification accuracy under minimal compression and moderate cropping.

ROC curve for PRNU-based camera identification evaluated on 1,582 positive and 1,300 negative images, with transformations including rotation from 0° to 270°, cropping up to 20%, and JPEG compression quality between 60 and 80. The red label indicates a threshold of 27.0, and the curve achieves an AUC (Area Under Curve) of 0.94, reflecting strong detection performance under moderate compression and limited image alteration.
Figure 9: influence of slight (top) and moderate (bottom) JPEG post-compression in presence of cropping

Interestingly, the amount of cropping is not as relevant: as Figure 10 shows, the fact that a small (left plot) or relevant (right plot) amount of cropping is applied does not influence performance significantly.

Receiver Operating Characteristic (ROC) curve for PRNU camera source identification using 1,077 positive and 1,400 negative image samples. Tested under transformations including image rotation from 0° to 270°, cropping up to 5%, and JPEG compression quality ranging from 60 to 100. The plot indicates a threshold of 27.0 and an AUC (Area Under Curve) of 0.97, demonstrating high model accuracy and robustness against mild image modifications.

ROC curve showing PRNU-based camera source verification with 1,096 positive and 1,000 negative images. Test conditions include rotations between 0° and 270°, cropping from 15% to 20%, and JPEG compression quality levels from 60 to 100. The red threshold line is set at 24.0, and the Area Under Curve (AUC) score is 0.95, indicating strong performance and reliable discrimination under moderate image alterations.
Figure 10: impact of the amount of cropping (top plot: small amounts – bottom plot: relevant amounts)

Experiment 4: Discriminating Different Exemplars of the Same Model

This experiment investigates if the source identification system can discriminate images generated by different exemplars of the same brand and model. We tested 5 different models, two exemplars each, as explained in the following Table:

MakeModelTypeYearResolutionExemplars
AppleiPhone 4sPhone20118MP2
CanonEOS 10DDSLR20036MP2
CanonEOS 40DDSLR200710MP2
CasioExilim EX-Z70Compact20067MP2
OlympusD-560Compact20033MP2

During experiments, 40 images were used for each device to create the reference pattern; all remaining images from the same exemplar were used as the positive set. The negative set is composed by all available images coming from a different exemplar of the same brand and model, plus a random choice of images coming from different devices.

As Figure 11 shows, performance is totally in line with those in Experiment 1. This experiment suggests that the source identification system effectively discriminates images coming from different devices even when they belong to the same brand and model.

ROC curve for PRNU-based image authentication using 537 positive and 1,303 negative samples. The test evaluates source camera identification accuracy with a threshold set at 50.0. The chart indicates high performance with an AUC (Area Under Curve) of 0.99, showing excellent true positive rates and minimal false positives under strict verification conditions.
Figure 11: performance when devices of the same brand and model are considered

Experiment 5: Investigating the Impact of the Number of Images Used for CRP Computation

One key step in source device identification is CRP estimation. When the device is available, the best practice is to take some shots of a flat wall or a sunny sky (so-called “flat-field” images), with no zoom and with little or no compression, and then use them for SPN estimation. However, the device is not available in many cases: a reliable SPN estimate can still be obtained also from “common” images, but in this case, a greater number is recommended.

In this experiment, we investigate how the number of images used for SPN estimation impacts system performance. In our dataset, for most devices, we have natural images only, while flat-field images are available for three of them (Table 3). Notice that, for this experiment, we excluded the Lumix DMC-TZ3 device as it behaved as a negative outlier in Experiment 1, and would complicate interpretation of results.

MakeModelType
AppleiPhone 4Flat-field
CanonPowershot A75Flat-field
CasioExilim ex-z70Flat-field
BenQDC4330Natural
CanonPowershot s2isNatural
CanonPowershot SD630Natural
CanonIXIUS v2Natural
CasioExilim ex-z60Natural
CasioExilim ex-z75Natural
DigCamSub-6MPNatural
FujifilmFinepix A350Natural
HPPhotosmart 320Natural
Nikone-coolpix-s570Natural
NikonD70Natural
NikonD80Natural
NikonD100Natural
NikonD300Natural
Nikone2500 (Coolpix 950)Natural
Nikone885 (Coolpix 885)Natural
NikonE7900 (Coolpix 7900)Natural
NikonE880 (Coolpix 880)Natural
Nokian95Natural
Olympusu760Natural
OlympusE300Natural
PanasonicLumix dmc-fs15Natural
PentaxOptiosNatural
PremierDC-3320Natural
SonyCybershotNatural

Table 3: list of devices used in Experiment 4, specifying the type of images available for SPN estimation.

We repeated Experiment 1 separately for these two groups, while iteratively increasing the number of images used for estimating the SPN, moving from 5 to 50. This means that, at each iteration, a new group of images was

added to SPN estimation, while retaining those used in the previous step. Results are in Figure 12 for the case where natural images are available, and in Figure 13 for the case where flat-field images are available.

Plots highlight two relevant facts:

  1. When natural images are used for SPN estimation, performance improves significantly until 30 images are used, then converge.
  2. In the case where flat-field images are used for SPN estimation, performance converge is much faster, so that using 20 or 50 images does not make a relevant difference.

The above facts are totally in line with the theory about SPN estimation. It is well known, indeed, that a better estimate is obtained when reference images have little variance (as flat-field images do). On the other hand, when natural images are used, it takes more of them to reach the same SPN pureness (which leads to better device identification). Intuitively, this is because there is more content-related information to be “averaged away”.

From a practical point of view, this experiment suggests that, if natural images are used for SPN estimation, at least 40 images should be employed. This number can be lowered to 20 when flat-field images are used. We should consider, however, that for the flat-field case our dataset contained only three devices, which does not allow us to reach strong conclusions.

ROC curve comparing source camera identification performance using different quantities of natural reference images for PRNU analysis. The graph displays AUC values for 10, 20, 30, 40, and 50 reference images, with identification accuracy improving from an AUC of 0.977 to 0.992 as the number of reference images increases, demonstrating the positive impact of additional reference data on verification performance.
Figure 12: ROC curves obtained using different amounts of natural images for SPN estimation

Graph showing source identification performance based on varying numbers of flat reference images using PRNU analysis. ROC curves indicate increasing AUC values with more reference images: 10 (0.991), 20 (0.992), 30 (0.993), 40 (0.994), and 50 (0.994). Results demonstrate that even a limited number of flat-field reference images yield high identification accuracy in digital image forensics.
Figure 13: ROC curves obtained using different amounts of flat-field images for SPN estimation

 Marco Fontani

Marco Fontani is the Forensics Director at Amped Software, a software company developing image and video forensic solutions for law enforcement agencies worldwide. He earned his MSc in Computer Engineering in 2010 and his Ph.D. in Information Engineering in 2014. His research focused on image watermarking and multimedia forensics. He participated in several research projects funded by the EU and EOARD, and authored/co-authored over 30 journal and conference proceedings papers. He has experience in delivering training to law enforcement and provided expert witness testimony on several forensic cases involving digital images and videos. He is a former member of the IEEE Information Forensics and Security Technical Committee, and he actively contributed to the development of ENFSI’s Best Practice Manual for Image Authentication.

Subscribe to our Blog

Receive an email notification when a new blog post is published and don’t miss out on our latest updates, how-tos, case studies and much more content!