Facial recognition face-off: Three tools compared

Last week's Picasa software update from Google brought with it a neat trick--facial recognition. But it wasn't the first free consumer photo-editing software to find faces. In January, Apple unveiled the latest version of iLife, which included an updated version of iPhoto that could detect and recognize faces in your photos. And this time last year, Microsoft released an updated version of its Windows Live Photo Gallery desktop software that could find faces inside of photos, though it couldn't (and still can't) recognize who's in them.

So, how do these three stack up? To figure that out, we put them to the test. Using 500 sample photos on fresh installs of each program, we tracked around how long each of the tools took to process all the photos, as well as some notable hits and misses from each.

To be fair, our results may not scale, or match the experience you will have. For one, we're using a test bed of photos that's almost entirely 12-megapixel JPEG files, whereas some people may be shooting smaller or larger files that may be in different formats and contain large groups of people--something that can slow these programs down. You're also likely to have a whole lot more than 500 photos sitting around on your computer; we certainly do.

Note: Adobe's PhotoShop Elements software (for Windows | Mac), which also includes a facial recognition feature was not included in this roundup since it's a paid application. Technically iPhoto is as well, but we included it since it comes free on all Macs.

The apps and workflows

iPhoto

iPhoto is the only product of the bunch that's Mac-only. It comes bundled with all new Macs, but the latest version (which includes face detection) must be purchased as a software upgrade if you've got iPhoto '08 or lower. We've included it in this roundup as a free product since it comes bundled with all new Macs.

Face scanning in iPhoto happens automatically, but it's largely a manual process, requiring users to "train" the system to recognize certain faces. The program took around nine minutes to scan through our 500 test photos and when it was done it didn't offer up any suggestions of photos with faces in them.

Instead, users are required to click on a photo with a face in it and hope the program picked it up. If it has, users can simply type the name in--which will auto complete if the person is in your Mac address book. If someone's face was not found, but you can see it in the photo, you can manually contain the face inside of a box, then tag it with their name.

After you add names to just few photos, iPhoto's system begins to piece together others that look the same--although it doesn't learn as fast as it does for photos where it already found the faces. In my testing, it only took two photos to get it to offer up some more suggestions. If those suggestions are correct, continuing to add them was just a matter of a few clicks.

iPhoto's system for doing this isn't perfect though. As far as workflows go, iPhoto's requires a fair number of steps. As it offers up suggestions, you need to click an additional button to get into the "confirm name" mode. There's also not a way to skip directly to its suggestions without first scrolling down past the photos you've already confirmed as that person.

Its one grace is that like Picasa, it's fairly easy to bulk accept or reject iPhoto's suggestions--but doing so isn't made immediately evident from the software. The normal system requires one click to accept the person as a match, whereas two clicks reject it. This works great for a handful of photos, but if you're going through hundreds of shots it's a pain. Apple's solution is to have you drag your mouse around the photos you want to accept, while holding the option key while you do it sets all of the photos to be rejected. This makes it easy to blow through a handful of photos at once.

Picasa 3.5

Google's Picasa 3.5 is the first software version of Picasa to feature facial recognition and sorting. It's also one of the easiest programs to use out of the three we tested.

Picasa begins to scan for photos as soon as you fire it up for the first time, as well as any time new photos are added. For us, it took 14 minutes once it started scanning to when it finished, however having recently tested this on a larger library, those times don't necessarily scale. Some users may have to leave the program running for a few days for it to finish scanning.

Once it's done, it presents you with an array of faces that can be claimed by name. These names can be created on the fly, or if you're signed into your Google account--pulled from your Google Address Book.

Now here's where Picasa offers one of the best user workflows out of all of these services. Once you've identified a few photos as the same person, it begins offering up suggestions of other photos where they may have appeared. And instead of making you click through one at a time, it has speedy ways to bulk accept or deny its suggestions.

It does this two ways: one is to give you a little alert with an orange question mark next to their name. It also lets you skip directly to those recommendations while ignoring the items that you've already confirmed as that person--something iPhoto does not. It also lets you bulk confirm all of them at once, which can be a huge time saver instead of clicking the little yes and no buttons that sit below each suggested photo.

Windows Live Photo Gallery

Windows Live Photo Gallery is Microsoft's free Web-enabled desktop photo library management, editing, and sharing software. It's largely the same as the Windows Photo Gallery software that ships with Windows but adds a few extra features like a photo syncing between computers, a panoramic stitching tool, batch resizing tools, and facial detection.

Being the oldest software with facial detection in this bunch did not mean Windows Live Photo Gallery was the slowest. In fact, in our tests, the software came out the fastest, scanning through our test photos in just under eight minutes.

That was unfortunately the fastest part of the process though, since "detection" is about all it does. As for recognizing who is in those photos and putting them into groups, the legwork gets put entirely on the user. This wouldn't be so bad, except for the fact that built-in tools for organizing require that you go through each photo one at a time to name each face it's found.

You can click on multiple photos with the same person and drag them over to your source list of "people tags," but this does not automatically assign whatever face the program found to that person. This leaves you with an additional tag the system has not yet assigned.

As for manually assigning people tags to faces it's found, if you're signed into your Windows Live ID it can pull in names from your online address book. This speeds up the process (assuming you have the people in there already), but here again, the user workflow has not been well thought out.

Instead of surfacing recently-used contacts to the top when beginning to type a name, it consistently puts things in alphabetical order. If you have a large address book, or friends with slightly similar names this really stinks. It's also annoying if you're going through a "roll" of photos from an event where you have many of the same people.

One thing Windows Live Photo Gallery does well is offer users a way to categorize their friends into folders. Actually, this needs to be done from somewhere like Hotmail, or another service that lets you access or edit your address book. But the results are great. You can very quickly sort out friends and family from business contacts and coworkers. In Picasa and iPhoto you're stuck with an alphabetical source list.

Winner: Picasa

If we were to choose one of these apps to help churn through an enormous library, Picasa is the way to go. It has, hands-down, the best workflow for finding faces, as well as alerting users to when it had found people in new photos. It's also the only program out of the three we tested that's cross platform, meaning you can use it on either a PC or Mac and have an identical face recognizing experience.

Face detection hits and misses

Hits

Something surprising that just kept happening with all three pieces of software was it picked up small faces in photo frames and posters, as well as people behind reflective glass surfaces.

This happened on a number of occasions but was most apparent when using Picasa since it shows you all of the possible faces it picked up inside of one menu. Many times these were people far off in the background, or in spots so dark that the photo had to be brightened up a bit for us to see that there was a person there.

Also impressive was that all three pieces of software did not pull up photos of the pets that were included in some of the photos from our batch of test shots, as well as from an auxillary test done with 20 photos of the same cat. Blog MacLife had a different experience--at least for iPhoto, and was able to get the software to discern between two cats.

Misses

It should be noted that all three of these programs (and facial recognition in general) has an incredibly hard time with sunglasses, hats and hooded sweatshirts, and to a lesser degree, things like regular glasses and facial hair.

Just like the Unabomber, hiding half of your face with large sunglasses and covering up your hair with a hood is a surefire way to have all of these apps be unable to identify you. That was certainly the case for some of our test photos. We threw in more than 50 shots that included large sunglasses--none of which were picked up as recognizable.

Also problematic were features that were face-like that got picked up by the software, as well as facial expressions. Wide smiles, open mouths, and upturned eyebrows threw off all three pieces of software. Again, this is one of those areas where the software's workflow can make these misses less bothersome.

What was more of a problem though, were the programs not being able to pick up people's faces if turned sideways--be it a profile shot, or someone with a tilted head. For posed snapshots this isn't a big deal, but a good number of our photos featured people who weren't looking at the camera. Out of three programs, Picasa handled that the best, simply highlighting them as faces, even if it was unable to tell whose they were.

Other things to look out for include slightly obscured faces, like people in mid-bite at a meal, as well as faces that are slightly out of focus. This isn't a big problem with point-and-shoot cameras, but if you have an SLR and are using a lens with a shallow depth of field, someone just slightly out of focus can render their face undetectable to the software.

Conclusions

These three pieces of software each offer different approaches to photo management and organization, and face detection is just a small part of the many things they do. It's also hard to truly stack them together, since two of them can only be run only on one kind of operating system (iPhoto for Mac and Windows Live Photo Gallery for Windows).

That said, what matters in the end is the workflow--or how you're able to go through the suggestions it comes up with and use it to help organize and categorize your photos. For that Picasa is the clear winner.

Out of all three products Picasa required the least amount of time and effort for us to properly go through our photos and feel confident that we had tagged them all. It also had one of the best systems for accessing and editing information about stored online contacts by basically sticking the Google contact manager into the program as its own window.

In second place is Apple's iPhoto, which can also make tagging large batches of photos relatively easy, but in the end it lost out to Picasa due to the training required to even begin the tagging process. It also got taken down a notch on the user friendliness scale for not providing more notice when it had come up with more possible face matches.

And in last place was Windows Live Photo Gallery, which to be fair, is limited to face detection and not recognition. My hope is that future versions will take a page from Google's efforts and focus less on the feature itself as much as what users need to do in order to use it for organizational purposes. As it stands the tagging process is just too manual, and without the quick tagging shortcuts and optimizations like Picasa and iPhoto have, users just get muddled down with too many button presses to make it worth their while.

Has your experience differed from ours? Let us know in the comments.

Note: The machine used for testing was a late-2008 Apple MacBook with a 2.4GHz Intel processor and 4GB of RAM. For testing on Windows, the same machine was used but in Boot Camp running an RTM build of Windows 7 Ultimate.

Correction: This post initially misstated the lack of a feature to bulk accept or reject facial recognition matches in iPhoto. The software does contain the feature.