Big Blue could monitor borders, shoplifters, moose

High-tech digital-surveillance system from IBM Research has far-reaching possibilities.

Candace Lombardi
In a software-driven world, it's easy to forget about the nuts and bolts. Whether it's cars, robots, personal gadgetry or industrial machines, Candace Lombardi examines the moving parts that keep our world rotating. A journalist who divides her time between the United States and the United Kingdom, Lombardi has written about technology for the sites of The New York Times, CNET, USA Today, MSN, ZDNet, Silicon.com, and GameSpot. She is a member of the CNET Blog Network and is not a current employee of CNET.
Candace Lombardi
3 min read
A new surveillance system that analyzes video for intruders, or even a specific license plate number, is now ready for the mainstream market, IBM announced Tuesday.

The Smart Surveillance System (S3), available in conjunction with IBM's digital-video surveillance services, can analyze real-time video while it is being digitally recorded and stored over an Internet Protocol network. Video stored both online and offline can also be retroactively searched for certain characteristics.

"This is not the HAL 9000. There are also much less evil reasons for using this.
--Charles Palmer, manager, IBM Research

While IBM does not name specific security customers, the surveillance system has been in use by several governments, law enforcement agencies, airports and some businesses, according to Charles Palmer, director of IBM's Privacy Research Institute.

"This is not the HAL 9000. This is not able to do full face recognition or car type, unless they are different-color cars," Palmer said. "There are also much less evil reasons for using this. One retailer, for example, wanted to know where empty parking spots were."

In another field test, S3 was used in the self-checkout line of a grocery store to identify different types of fruit and their corresponding prices.

But IBM has also been in talks to possibly manage an S3 system in spots along the U.S.-Mexico border.

"Clearly, those folks have been up here several times to look at it, and every time they do backflips down the hall. It's a long border. It's really hard to get that data back and forth because it has to be wireless, or the critters will eat it, and you also have large birds of prey perching on things like cameras," Palmer said.

IBM's system connects surveillance video with smart software that can detect and index what you tell it to in real time. The system performs attribute-based searches on stored video clips for specific objects or actions, or can be set to sound alarms when those things come across the screen.

S3 is similar to surveillance software developed by San Francisco-based 3VR, but it adds the ability to integrate with other recognition software, such as plug-ins that identify license plates.

In one instance, S3 was used to identify customers who walked into a store entrance without a package but then approached the returns desk with a package.

S3 can also backtrack the path of an object entering a particular area. In a video feed of an airport tarmac, for example, S3 can electronically draw a line around a particular area of the screen, then backtrack the path of anyone entering that secure area of interest. A rule can be set so that alarms go off if the person walking into that secure area did not enter from a predetermined point of entry, Palmer said.

To address privacy concerns, S3 can redact things like people's faces and license plates with a black box or blur, Palmer said. The redaction takes place when the video goes to an IP network to be stored, though the entity doing the surveillance has to request that that feature be in place. The video can then only be unredacted by an authorized person if, for example, an incident occurs, and the video needs to be searched for a particular time span, Palmer said.

If the data is being stored offline, however, then the redaction feature is most likely not being used, Palmer said.

But S3 doesn't keep tabs on just humans; it can track animals too. During its field testing, IBM Research was asked by one group to create an S3 system for detecting when moose migrated into a particular area.

"It's not a big area, but (it's) one we can do. We can certainly tell the difference between a moose and a deer. People can too, but you need to keep them awake," Palmer said.

The S3 could also potentially be programmed to identify things as specific as skin color in humans.

"I can imagine they could do it for people. I suppose it's possible, but it's never been something we'd been asked to do. The only color-based things have been cars and airplanes," Palmer said.