Probably, Tip Tuesday aficionados have already understood the trend. We’re alternating tips for Amped FIVE and Authenticate every week. Well, it is true. And it is intentional! Today we’re showing you some static and dynamic tracking tips in Amped FIVE.
Tracking an object is a basic, yet
Regardless of the goal, good tracking is essential to the success of your processing. That is why Amped FIVE features several different ways to track your object of interest:
- Manual tracking
- Static tracking
- Dynamic tracking
Pidgeon Stabilization
Since most FIVE users have to deal with faces and license plates every day, we’re using a different example for this Tip: pidgeon stabilization! Let’s see how tracking can be used to stabilize the top-positioned pidgeon in this video (notice that both the pidgeon and the camera are moving):
Manual Tracking & Static Tracking
Manual Tracking doesn’t leave much to your imagination. You draw a rectangle around your object in the first frame. Then, by pressing “U” on your keyboard you’ll have such selection assigned to Frame 0 and be carried to the next frame, where you will move the rectangle to follow the object, press “U” and iterate until the end. Tedious? We know. Keep in mind, however, that Amped FIVE will interpolate the selection between frames. So, you are free to specify the object once every few frames, not for every single one.
Manual Tracking can be an easy solution when you are doing redaction and you want to follow an object through a few frames. Normally, we suggest you try Static and Dynamic tracking first!
Static Tracking will do most of the job for you. Just select your object in the first frame as before. However, but now our algorithm will try to automatically follow it in the next frames. If the algorithm fails, you will see the whole frame move abruptly (in case you’re doing Local Stabilization), or your red circle is randomly going somewhere (if you are annotating), indicating that more work is needed. This is indeed what happens in our example.
In such a case, you may want to move to the first “problematic frame” and help the algorithm by selecting the object and pressing U. This way, you are telling FIVE “psst, the object is indeed here!”. We’ve added four selections covering the first 50 frames.
Let’s see how it goes:
We see that now the tracking succeeds until frame 90, then gets lost. Now, in many practical cases, a single initial selection is sufficient. Sometimes, your helping hand is needed a couple of times. But there are cases in which the tracking keeps failing, or too many “manual hints” would be necessary, as with our pidgeon. Why is that?
Likely, the reason is that the aspect of your object is changing too much, so that the algorithm cannot match its pixels with those in the following frame. A strong change in perspective, shape, or lighting complicates the match. In fact, Static Tracking always compares the object to “how it was” in the first frame. Thus, if the object is moving a lot, like in our example, after a few frames it will have accumulated a huge difference! This is why Static Tracking could not help us with pidgeon stabilization.
Dynamic Tracking
This is where Dynamic Tracking enters the game. Instead of using your first selection as reference, the algorithm will update the target object at each frame, so to better capture its changes in shape, perspective and illumination. This comes at the cost of a higher computational effort (read: slower processing). Sometimes it causes the stabilized object to move slightly (nothing, when compared to the “epic fail” caused by a tracking failure).
Let’s try Dynamic Tracking on the previous example. If you place your mouse over the pidgeon at the beginning of the video and leave it still, you’ll see that the pidgeon is finally static (it remains in the position of your mouse).
At this point, you may be asking: “Why not always use the Dynamic Tracking, then?”
As usual, there is a tradeoff:
- Static Tracking has the advantage of being more precise (if tracking fails on one frame, the next one is not affected), at the cost of limited robustness;
- Dynamic Tracking on the contrary is more robust to variations, but less precise because errors in one frame may propagate to the next.
Conclusion
Let’s conclude with this week’s takeaway. The strategy we recommend for tracking is to first look at how the object evolves in time. Does its aspect change significantly? If not, try the Static Tracking first. Possibly, add a couple of additional selections if needed. If that doesn’t work, just click on the drop-down menu and try the Dynamic Tracking. It will, hopefully, do the trick! However, if you are still unlucky, you can still add some manual selections every few frames to help the algorithm (even when it’s running in the Dynamic mode).
We hope you enjoyed this week’s Tuesday Tip! Stay tuned and don’t miss the next ones!
