Game Changing Algorithm Turns First Person Videos Into Incredibly Smooth Hyperlapses

Researchers at Microsoft just changed the POV video game — there’s no doubt about that. Using a newly developed algorithm, they transform long, boring first person videos shot with helmet cams into super smooth hyperlapses that look like they were shot with a Steadicam.

The quality of the results really defy description. What begins as a long and tedious first person video of, say, someone rock climbing, turns into a steadicam-smooth roller coaster ride up a mountain… and it’s all software based.

Here is a technical explanation of what the algorithm is actually doing in order to obtain these results:

Basically, the software uses the video to recreate the camera’s path on a 3D depth map, picking a smooth path through that map, and reconstructing what the camera on this path would see using multiple frames to reconstruct each frame.

Doing this, they’re able to get much smoother results than standard stabilization techniques that deal with a single frame at a time. A marginally smooth, nauseating video turns into something not just watchable but downright impressive.

They even remove any sections where the camera stays still for too long, so the movement of the camera can stay smooth and continuous.

Screen Shot 2014-08-11 at 8.52.49 AM

The final results aren’t always perfect, you can see some issues that arise from combining several frames that don’t always match as well as they should, fixing for distortion and white balance. But it’s leaps and bounds beyond what the ‘naive’ time-lapse that uses every 10th frame can do.

If you haven’t already, definitely watch the video at the top to see the technique in action and hear a brief explanation, check out the second video if you want to get into the technical details, and if you’re really intrigued, you can read the full paper by clicking here.

And get excited, because while no specific timeline has been laid out, creators Johannes Kopf, Michael Cohen and Richard Szeliski say they are working hard to put the tech into a Windows app.

Discussion