Microsoft makes cool stills from lousy video
At Microsoft's TechFair this week, researchers show how a surprisingly sharp panorama can be created from a rather wobbly video image.
MOUNTAIN VIEW, Calif.--Ever wish you could get a decent still image from that so-so video clip you took on your digital camera?
That ability may not be too far away. At Thursday'son Microsoft's campus here, researcher Neel Joshi showed several cool approaches of using short video clips and burst-mode photography to create some pretty nifty pictures and panoramas.
In one example, Joshi used a hazy view of Mount Ranier and tracked the motion of the atmosphere to take the haze away and create an image that was far sharper than what was visible with the naked eye. In another example, he showed how a wobbly video of Microsoft's Building 99 could be turned into a surprisingly sharp panorama.
At 30 frames a second, the 30-second video has about 900 still images, though only about 80 were used to create the panorama.
"It's only taking about 10 percent of the data, but it's probably the best 10 percent," Joshi said.
Those first two efforts used computers to remove the motion and leave in place the part that was static--the mountain and the building. A third effort use the reverse approach. The researchers took a video of someone jumping into the base of a waterfall and created a really neat stop-motion shot that showed the person jumping at several points along the path, while keeping the background steady using a sharp composite created from all the video stills that had the same background.
Part of the science behind all this is that the images from a video clip can be more than the sum of the blurry parts. In each frame, there can be one part that is sharper than others. Combining that can offer some pretty dramatic improvements.
To really get a sense for this, watch the two video clips I've embedded here and then see the still images that were created. For more examples of what the researchers were able to do, check out the photo gallery embedded above.
This is a hazy view of Mount Rainier:
And this is the still image that resulted:
And here is the video of Building 99:
And the panorama that was created:
Joshi, who is based at Microsoft's research lab in Redmond, is also working on some cool stuff that he plans to present at the Siggraph show in July. That paper looks at how to looks at how one might be able to use the accelerometer and gyroscope in a cell phone or camera to detect what's behind image blur and potentially sharpen things further. At a separate conference this summer, Joshi plans to present a paper on creating a de-blurred panorama.