With cameras and object recognition, GE and Samsung are bringing smart kitchen appliances to the US that might actually be useful.
I spent 15 minutes trying out the newest version of the Samsung Family Hub smart fridge at Samsung's CES booth this year. It was just me, the fridge, and version 5.0 of the software that runs the 21-inch touchscreen embedded in its door.
What that new software does, among other things, is use the fridge's built-in camera and some back-end AI to interpret food items on the shelf as data. Once the Family Hub understands that an avocado is, in fact, an avocado, the fridge can do things like make recipe recommendations, plan your weekly menu automatically, track your food inventory and even send cooking instructions to a connected oven.
Not all of the new Family Hub functions worked perfectly in my brief, unsupervised hands-on, but the step forward was evident enough that it appears that the smart kitchen might actually be getting smart enough to be useful.
We've seen this kind of data interpretation in other smart home products. Google's venerable Nest Learning Thermostat claimed from the start that it would learn your behavior to adjust the heat in your home automatically. Mattress-maker Sleep Number introduced a smart bed that communicates with devices like a Nest thermostat and a Fitbit fitness band to inform you about the patterns of other things in your life that correlate with you getting a good night of sleep.
Back in the kitchen, the June Intelligent Oven relies on object recognition to automatically start cooking your steak when you put it in the oven. GE also demonstrated its own food recognition capability at CES this year with its updated Kitchen Hub over-the-range microwave.
The rush to interpret smart home data into something meaningful is just the latest front in the campaign among technology companies to bring everything they can inside and outside of your home online. Call it smart home technology, ambient computing, or whatever next year's buzzword for it happens to be, the end result is that once everything is connected, these companies believe they can offer more compelling, more convenient technology products and services informed by the data they gather along the way.
Where the basic, transactional smart home functions (turn the lights on and off with your phone, turn your AC off when your thermostat detects that you've left the house) are well understood at this point, the next step in bringing all of these fixtures online needs to do more than turn your phone or your voice into a simple remote control. All of those simple commands generate data. It appears that the industry is now ready to put that data to work in the kitchen.
I went back to Las Vegas this week to check in on the annual Kitchen and Bath Industry Show, known as KBIS. Like CES, KBIS also takes place across the many halls of the sprawling Las Vegas Convention Center. If you're into backsplash tile and cabinet hardware, this is the show for you.
You'll also find large appliance manufacturers here, but unlike CES, the focus of KBIS is much more US market-driven. Bosch was here, but the AI-driven, camera-equipped refrigerator it demonstrated at CES was nowhere to be seen since it's still a Europe-only product. Electrolux was also here at KBIS, but while it has demonstrated camera-equipped ovens at Germany's IFA tech conference (it doesn't exhibit at CES), its lineup at KBIS only had the basic remote oven-heat functionality we've seen for years.
Even LG , which every year shows off all manner of because-they-can technology at CES and IFA (the cocktail ice fridge is cool), had little at KBIS beyond the basic smarts we've seen for years. We know it has next-level smarts in the works for kitchen from a chip announcement last May, but we haven't seen that technology emerge in a shipping product yet.
In the US this year, it appears that GE and Samsung will be the standard bearers for bringing the smart kitchen forward. Samsung's new Family Hub and GE's new Kitchen Hub microwave were the highlight of each vendor's respective KBIS booths. Samsung has the more complete approach, but GE is on to something as well.
I like Samsung's strategy because it starts at one of the places where food comes into your home. You can back the cycle out further back to dietary needs, or general food preferences, and new Family Hub software will allow you to tailor its recipe recommendations based on those factors, but for me, my best intentions when I go grocery shopping, even with a list built around a weekly menu, are often corrupted by whatever might look good on the shelf at that moment.
Once I pay for all of that food and bring it home, that's when my family's cooking for the week gets real. The most functional scenario would be a system that logs what I've purchased at the moment I buy it, supported by a kitchen full of cameras and sensors in both my refrigerator, my cabinets and wherever else I store my food to monitor my total food inventory and generate and maintain a weekly menu, recipes and the following week's shopping list in real time. We're not there, and I'm not sure I'd want to be yet without robust end-to-end data security, but short of that overall scenario, Samsung's Family Hub 5.0 software appears to be a pretty good next step.
The object recognition in Samsung's fridge isn't perfect yet. When I played with it at CES, it mislabeled a can of seltzer when I shifted its position next to a can of tomatoes. It can recognize the difference between an orange and an apple, but a Samsung spokesperson at KBIS admitted that there's still a lot of learning to do before the Kitchen Hub can make more subtle differentiation like telling parsley from cilantro. Its camera also only captures the main shelves of Family Hub. It can't see inside the crisper drawers, the door shelves, or the freezer. That's a lot of fridge left unscanned, not to mention the dry goods in your pantry.
The immediate solution will rely on you to either fill in those gaps manually in Samsung's app to improve the recommendation engine, or to simply live with the imperfect suggestions, and adapt and ignore as needed. We're probably years away from perfection. Put your hands-on the Family Hub though, and you can see the concept start to come together. I hate menu planning. If Family Hub 5.0 works when Samsung distributes it to new and existing fridge models later this spring, even its partial solution is welcome.
GE's approach is focused more directly on food preparation. Like the Family Hub fridge, GE's Kitchen Hub over-the-range microwave (and the vent hood-only version released this part year) is built around cameras and a large touchscreen front-and-center in your kitchen.
You'll have a few camera options with the Kitchen Hub microwave. It ships with three cameras built-in, one facing out for making video calls, another facing down to capture what's on your burners, and yet another inside the microwave itself. At KBIS, GE also demonstrated a fourth-camera option attached underneath a cabinet, aimed down at an adjacent cutting board. Both the primary down-facing camera and the accessory cam will have object recognition capability.
The down-facing cameras do a few things. They can recognize food items that you put in front of them and recommend recipes based on that data. The burner cam, in combination with the cook settings if you have a connected GE oven, can also tell you when the food on your stove top is done. GE envisions the camera and the touchscreen working in combination to help you become a better cook. Once the Kitchen Hub identifies ingredients, it can recommend recipes and then display step-by-step instructions for how to prepare them. It can pull those ingredients from the camera, or from an inventory that you enter manually.
I asked GE's Shawn Stover, executive director of SmartHome Solutions, about incorporating refrigeration, but he doesn't believe the technology there is robust enough yet. "Too many bad inputs can cascade into other problems," he told me. He did say GE is exploring partnering with grocery stores to link up your weekly purchases with the Kitchen Hub, which might be one way to save you from the labor of manual entry. He also described a possible future scenario where an AR overlay on the Kitchen Hub screen shows you exactly how to cut a steak you've plopped down under the camera.
Current-generation Kitchen Hub owners will get to try the object recognition for themselves when the software update goes out in the second quarter this year. The Kitchen Hub over-the-range microwave and the accessory camera will ship in late 2020 for an as-yet-undetermined price. The vent hood-only model costs $1,200.
I don't anticipate either GE's or Samsung's solution will sway skeptics of the smart kitchen or the smart home in general this year, although credit to both companies for bringing new functionality to existing products. In Samsung's case, even the first generation Family Hub refrigerator, now five years old, will get the object recognition software update. The more important point is that both companies are going beyond the basic transactional smart home functions and doing something innovative, and even potentially useful with data they generate by bringing household devices online.