Inspired by bugs, start-up seeks night vision
Seeing in the dark is hard, but some insects can manage it. NocturnalVision uses some of their methods for its own low-light video tech.
- Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
LONDON--Every researcher from Isaac Newton on knows well the advantages of seeing farther by standing on the shoulders of giants. Some Swedish researchers, though, are seeing better by standing on the shoulders of tropical bees.
A Swedish start-up called NocturnalVision wants to help cameras see in the dark better. To do so, it took inspiration from Megalopta genalis, the bee, and other insects active at night, Henrik Malm, a professor at Sweden-based Lund University and co-founder of the start-up, said in a talk at the Image Sensors Europe conference here.
The researchers are working to address a common problem with digital image sensor technology: the image-degrading speckles called noise that show up in low-light conditions.
Image sensors work better with more light, but people want to take photos and videos in dim conditions such as a party or in a restaurant. Cameras can amplify the information they record, transforming a dark image into one that shows what's going on in the scene, but doing so also amplifies the noise.
To cope, NocturnalVisioin takes an approach it calls spatiotemporal summation. In a nutshell, it analyzes what's going on across each frame of an image--the spatial component--and what's going on from one frame to the next--the temporal component--to try to intelligently direct the noise reduction process.
So, for example, with a man walking across a videocamera's field of view, the company's process detects his motion. That information can help inform whether one pixel is changing from dark to light because of the random fluctuations of noise, which which case it should be suppressed, or the arrival of the man into that part of the frame, in which case the change should be preserved.
"I think the noise reduction is pretty strong," said Malm, who is head of technology at NocturalVision.
The technique also adjusts automatically to local differences across the frame, so, for example, details in a dim area can still be seen even after a very bright light source intrudes into another area.
Trying to discern dim details while avoiding the blinding effects of bright lights sounds like a problem night-time drivers have, and indeed, Toyota is interested in the researchers' approach.
The carmaker has rights to the technology for automotive applications, Malm said, and NocturnalVision has rights elsewhere.
The start-up is interested in consumer cameras, camera phones, surveillance cameras, and perhaps military applications, he said.
The company is seeking venture capital and also is in talks with Sony Ericsson, Malm said.
One difficulty with the company's process is its computational requirements. It works by a mathematical analysis of a multi-frame sequence, wending its way through an entire video. That means for real-time work, there's a delay of about six frames before the image processing can kick in.
In addition, the mathematics involved take some processing. Right now, the company can produce five frames of video per second at a resolution of 640x400 pixels, Malm said--much slower than the frame rate cameras today employ.
However, the process can get a helping hand from a graphics chip; the company is currently using an Nvidia GForce 880GTX for its work.
"This algorithm is well-suited for parallel programming on a GPU or on other parallel architectures," Malm said. With newer hardware, it's likely the algorithm could work in real time--in other words, for video cameras that shoot at the ordinary rate of 30 frames per second.