Inspired by bugs, start-up seeks night vision

Seeing in the dark is hard, but some insects can manage it. NocturnalVision uses some of their methods for its own low-light video tech.

Henrik Malm of NocturnalVision speaks at the Image Sensors Europe conference.
Henrik Malm of NocturnalVision speaks at the Image Sensors Europe conference. Stephen Shankland/CNET

LONDON--Every researcher from Isaac Newton on knows well the advantages of seeing farther by standing on the shoulders of giants. Some Swedish researchers, though, are seeing better by standing on the shoulders of tropical bees.

A Swedish start-up called NocturnalVision wants to help cameras see in the dark better. To do so, it took inspiration from Megalopta genalis, the bee, and other insects active at night, Henrik Malm, a professor at Sweden-based Lund University and co-founder of the start-up, said in a talk at the Image Sensors Europe conference here.

The researchers are working to address a common problem with digital image sensor technology: the image-degrading speckles called noise that show up in low-light conditions.

Image sensors work better with more light, but people want to take photos and videos in dim conditions such as a party or in a restaurant. Cameras can amplify the information they record, transforming a dark image into one that shows what's going on in the scene, but doing so also amplifies the noise.

To cope, NocturnalVisioin takes an approach it calls spatiotemporal summation. In a nutshell, it analyzes what's going on across each frame of an image--the spatial component--and what's going on from one frame to the next--the temporal component--to try to intelligently direct the noise reduction process.

So, for example, with a man walking across a videocamera's field of view, the company's process detects his motion. That information can help inform whether one pixel is changing from dark to light because of the random fluctuations of noise, which which case it should be suppressed, or the arrival of the man into that part of the frame, in which case the change should be preserved.

The comparison of a frame taken recorded with a videocamera a night shows the original frame at left, the amplified signal at the center, and the image with NocturnalVision's noise reduction applied at right.
The comparison of a frame taken recorded with a videocamera a night shows the original frame at left, the amplified signal at the center, and the image with NocturnalVision's noise reduction applied at right. Henrik Malm, Lund University and NocturnalVision

"I think the noise reduction is pretty strong," said Malm, who is head of technology at NocturalVision.

The technique also adjusts automatically to local differences across the frame, so, for example, details in a dim area can still be seen even after a very bright light source intrudes into another area.

Trying to discern dim details while avoiding the blinding effects of bright lights sounds like a problem night-time drivers have, and indeed, Toyota is interested in the researchers' approach.

The carmaker has rights to the technology for automotive applications, Malm said, and NocturnalVision has rights elsewhere.

Megalopta genalis, a small tropical bee that inspired NocturnalVision's approach.
Megalopta genalis, a small tropical bee that inspired NocturnalVision's approach. Henrik Malm, Lund University and NocturnalVision

The start-up is interested in consumer cameras, camera phones, surveillance cameras, and perhaps military applications, he said.

The company is seeking venture capital and also is in talks with Sony Ericsson, Malm said.

One difficulty with the company's process is its computational requirements. It works by a mathematical analysis of a multi-frame sequence, wending its way through an entire video. That means for real-time work, there's a delay of about six frames before the image processing can kick in.

In addition, the mathematics involved take some processing. Right now, the company can produce five frames of video per second at a resolution of 640x400 pixels, Malm said--much slower than the frame rate cameras today employ.

However, the process can get a helping hand from a graphics chip; the company is currently using an Nvidia GForce 880GTX for its work.

"This algorithm is well-suited for parallel programming on a GPU or on other parallel architectures," Malm said. With newer hardware, it's likely the algorithm could work in real time--in other words, for video cameras that shoot at the ordinary rate of 30 frames per second.

About the author

Stephen Shankland has been a reporter at CNET since 1998 and covers browsers, Web development, digital photography and new technology. In the past he has been CNET's beat reporter for Google, Yahoo, Linux, open-source software, servers and supercomputers. He has a soft spot in his heart for standards groups and I/O interfaces.

 

Join the discussion

Conversation powered by Livefyre

Don't Miss
Hot Products
Trending on CNET

CNET's Christmas Gift Guide

'Tis the season for a gadget upgrade

Check out these 9 tablets you'll want to bring home for the holidays.