Toyota gives robots a soft touch and better vision to amplify our intelligence

Toyota Research Institute's latest robots can't be fooled by clear packaging or mirrors. That's a breakthrough.

Brian Cooley
Brian Cooley Editor at Large
Brian Cooley is CNET's Editor at large and has been with the brand since 1995. He currently focuses on electrification of vehicles but also follows the big trends in smart home, digital healthcare, 5G, the future of food, and augmented & virtual realities. Cooley is a sought after presenter by brands and their agencies when they want to understand how consumers react to new technologies. He has been a regular featured speaker at CES, Cannes Lions, Advertising Week and the Publicis HealthFront. He was born and raised in Silicon Valley when Apple's campus was mostly apricots.
Expertise Automotive technology, smart home, digital health. Credentials 5G Technician, ETA International
3 min read

Thanks to decades of sci-fi movies, it's easy to think of robots as human replacements: Metal arms, hinged legs and electronic eyes will do that. But the future of robots may center on extending and boosting us, not so much replacing us. While that concept isn't entirely new, it came more sharply into focus recently when I visited Toyota Research Institute's headquarters in Los Altos, California. That's where robots amplifying humans is the mission. 

TRI robot with Max Bajracharya and Brian Cooley

Don't be fooled by the humanoid layout of many robots. Their real value may be in amplifying more than replacing us. Toyota Research Institute's Max Bajracharya shows CNET's Brian Cooley one of its development robots.


"People here are passionate about making robots truly useful," says Max Bajracharya, VP of robotics at TRI and veteran of Google's robotics unit, Boston Dynamics and NASA's Mars rover team. Today he develops robots in mock-ups of home kitchens and grocery store aisles. "How can we really help people in their day-to-day lives?" TRI isn't charged with getting robots on the market but with figuring out the problems preventing that from happening.

One tangible example is a soft touch sensing gripper developed at TRI that allows a robot to handle delicate things the way we do, with a progressive, nuanced sense of pressure and recognition.

TRI robot soft touch gripper

The softer side of robots: TRI has developed soft touch robot "hands" that can not only handle things elegantly but also identify them.

Toyota Research Institute

These soft grippers are like padded hooves with cameras on the inside that can figure out the perfect amount of force needed to grip something and can also help identify an object, essentially by touch. "Replicating our body is incredibly complex if you think about how many sensors we have" in our skin and the superlative "wetware" processor in our skull that makes sense of them all, according to Bajracharya.

This work in soft touch is about more than just grabbing and holding. AI and smartphone pioneer Jeff Hawkins subscribes to a theory that much of how we understand the world is achieved by comparing "memory frames" of how we've moved through it, either via literal touch or through virtual contact with concepts like liberty and love. That kind of learning about the world seems to have strong echoes in TRI's work. 

TRI has also given its robots the ability to understand clear or reflective surfaces, something anyone with a cat knows can be remarkably hard. Transparencies and reflections can confuse a robot into thinking something is or isn't there when the opposite is true. From grocery aisles full of clear containers to homes full of mirrors, the places where this breakthrough can advance robotic relevance are numerous.

TRI robot handling transparent object

Recognizing a transparent object as actually being there is a sensing task that humans take for granted and one that robots are tricked by.

Toyota Research Institute

But the bigger goal remains a symphony of people and robots. "What you don't see today is humans and robots really interfacing together, which really limits how much robots can amplify humans' ability," says Bajracharya. That's where TRI turns AI on its head with something they call IA, or Intelligence Amplification. Simply put, it views robots as leveraging humans' superior intelligence and multiplying it with a robot's superior abilities in strength, precision, persistence and repeatability. Toyota has been focused on enhancing human mobility since a major announcement in 2017 by CEO Akiyo Toyoda at an event in Athens that I helped moderate

All of this brings up the relatability of robots: I've long felt that they will need to be as relatable as they are capable to achieve maximum adoption, since humans can't help but anthropomorphize them. That doesn't mean being as adorably useless as Kuri, which took CES by storm in 2017 before vanishing 18 months later, but it does mean establishing some kind of relationship. "Some people actually prefer a robot because it's not a person," says Bajracharya, speaking of robots in home or health care settings. "But some people are very concerned about this machine in their environment" due to fear of the unknown or concerns that it will displace human workers. 

The history of robotics is still only at its preface. But nuanced skills that amplify humans' savvy about the world while robots take over some of what dilutes our time and effort appear to be a formula for the next chapter.