Motion Tracking Made Easy

Today, I want to focus on a fun but nerdy topic, something I have struggled with for a long time: Motion Tracking.

There are really two types of motion to track, the movement of an object, and the movement of the camera. And it can be challenging to pull off correctly.

What exactly is motion tracking, you ask?

Well, when a camera or a subject moves inside a live-action scene, and a CG artist (such as myself) needs to add a 3D element to that scene, such as a background or a weapon, we have to track the motion in that scene in order to make a believable interaction between the live-action elements and the digital CG ones. Normally this is very difficult, but luckily there is software out there makes it easy!

I use a program called Boujou, but there are many tracking options out there, from other stand-alone programs to plug-ins. They tend to be expensive, thanks to the complicated programing and math involved, but the more expensive, the more you can do and the better the result, typically. I’m just glad I don’t have to know the math involved.

Essentially what I have to do is take a completely flat, 2D sequence of images (remember it’s 24, 30, or 60 frames a second for your average video) and tell a program to figure out how the camera or object moves in 3D space. In essence, I am extrapolating the 3D motion in the scene from a series of 2D images. In order to make this work, I need to find and mark several locations in the scene that the computer can then follow as long as it can. The more locations marked, the better the outcome. And, to get the best data, you need foreground, mid-ground, and background points all tracked so the data has a variety of points to sample from.

The locations, called trackers, can be placed either automatically or manually. If the scene you are working on has enough details of “high contrast,” the program can place dozens of trackers by itself and then follow the motion of those trackers. However, if there aren’t enough locations it can find, or if there are objects like mirrors or screens in the scene that can mess up the tracking, then you have to place them by hand. Once the trackers are placed and calculated, the program gives you a solution, called a “solve”.

This solve is then applied to a 3D scene comprised of nothing more than the trackers and a camera. The great thing about Boujou is it does a really good job of picking the best tracking points and making a good camera solve. There’s more to it then JUST that, like camera distortion and scene placement. Not to mention the camera data is useless inside Boujou by itself. It needs to be exported into another program like After Effects or Maya. I have been using it for Maya, and I have to say it does an amazing job creating a Maya scene with the trackers and the camera.

The best part for me is once you have the camera and tracking data, you can then put it in your scene, and as long as you set up your image sequence correctly, you can get an amazing result right in Maya! The funny thing is, while from the camera the scene looks perfect, if you look outside the camera, the illusion of the scene is broken.

In short, programs like Boujou do an amazing job of tracking the motion of the camera or objects in a video, and converting that into 3D data for use in visual effects and animation. This is an essential part of the job, and doing it all by hand is impractical and often impossible. The best solution is to use the computer to triangulate the location and movement of the camera or objects in a 3D space and then extract that data and use it in your scene.

Spending the extra time to learn and use motion tracking software will greatly improve any VFX project and looks really impressive.

Lately I’ve had several colleagues request I blog about specific topics. I plan to write more about those topics in future posts, and greatly appreciate the suggestions. If anyone has more suggestions, please send me a request and I will take a look at it.