Big Blue could monitor borders, shoplifters, moose

The Smart Surveillance System (S3), available in conjunction with IBM's digital-video surveillance services, can analyze real-time video while it is being digitally recorded and stored over an Internet Protocol network. Video stored both online and offline can also be retroactively searched for certain characteristics.

While IBM does not name specific security customers, the surveillance system has been in use by several governments, law enforcement agencies, airports and some businesses, according to Charles Palmer, director of IBM's Privacy Research Institute.

"This is not the HAL 9000. This is not able to do full face recognition or car type, unless they are different-color cars," Palmer said. "There are also much less evil reasons for using this. One retailer, for example, wanted to know where empty parking spots were."

In another field test, S3 was used in the self-checkout line of a grocery store to identify different types of fruit and their corresponding prices.

But IBM has also been in talks to possibly manage an S3 system in spots along the U.S.-Mexico border.

"Clearly, those folks have been up here several times to look at it, and every time they do backflips down the hall. It's a long border. It's really hard to get that data back and forth because it has to be wireless, or the critters will eat it, and you also have large birds of prey perching on things like cameras," Palmer said.

IBM's system connects surveillance video with smart software that can detect and index what you tell it to in real time. The system performs attribute-based searches on stored video clips for specific objects or actions, or can be set to sound alarms when those things come across the screen.

S3 is similar to surveillance software developed by San Francisco-based 3VR, but it adds the ability to integrate with other recognition software, such as plug-ins that identify license plates.

In one instance, S3 was used to identify customers who walked into a store entrance without a package but then approached the returns desk with a package.

S3 can also backtrack the path of an object entering a particular area. In a video feed of an airport tarmac, for example, S3 can electronically draw a line around a particular area of the screen, then backtrack the path of anyone entering that secure area of interest. A rule can be set so that alarms go off if the person walking into that secure area did not enter from a predetermined point of entry, Palmer said.

To address privacy concerns, S3 can redact things like people's faces and license plates with a black box or blur, Palmer said. The redaction takes place when the video goes to an IP network to be stored, though the entity doing the surveillance has to request that that feature be in place. The video can then only be unredacted by an authorized person if, for example, an incident occurs, and the video needs to be searched for a particular time span, Palmer said.

If the data is being stored offline, however, then the redaction feature is most likely not being used, Palmer said.

But S3 doesn't keep tabs on just humans; it can track animals too. During its field testing, IBM Research was asked by one group to create an S3 system for detecting when moose migrated into a particular area.

"It's not a big area, but (it's) one we can do. We can certainly tell the difference between a moose and a deer. People can too, but you need to keep them awake," Palmer said.

The S3 could also potentially be programmed to identify things as specific as skin color in humans.

"I can imagine they could do it for people. I suppose it's possible, but it's never been something we'd been asked to do. The only color-based things have been cars and airplanes," Palmer said.