Envision Glasses Use AI to Help the Blind Read Documents, Find Objects

Speaker 1: I'm a bit of a kid in a candy store. I love my gadgets and, uh, I love being independent. Speaker 2: These AI smart glasses are helping people who are blind or visually impaired. Hear the world around them. They're called envision glasses and they can read documents, identify people and objects and help with navigation. You wear the device like any regular pair of glasses, but these speak aloud what's around you, Speaker 3: Chicken and pumpkin [00:00:30] soup Speaker 2: With a few simple taps on the side, you can navigate a menu to scan, text, identify people, or describe a scene Speaker 3: 1541 sprinter Amsterdam central in Speaker 2: Vision can read and translate handwritten printed or digital text in over 60 languages. Carthic Conan is Envision's. Co-founder Speaker 4: What envision glasses essentially does is just takes in all the visual informations around. You tries to process that information and then speaks that out, uh, to the user Speaker 2: Envision is developed [00:01:00] on the enterprise of Google glass. Yes, it turns out Google glass is still alive in case you're not familiar. Google first unveiled these smart glasses back in 2013, they were supposed to let people take calls, send texts, snap pictures, and look at maps all from the headset, but they never hit store shelves. A few years later, a Google started working on an enterprise edition of the glass, which is what envision is built on their wearable and slim design makes them a perfect fit for this. Use. The glasses have a thin frame and [00:01:30] a small camera and speakers on the right hand side to relate information to the wearer. There's also a touch pad where users can tap to access the menu of features. Speaker 4: So these are the glasses, do a one finger double tap to get some information like the time and the battery level and the wifi and so on. Speaker 5: It is 1459. Today is Friday 25. March battery level is at 83%. Speaker 4: Then I do a one finger swipe to go through the different options in the glasses. Speaker 5: Breathe, Speaker 4: [00:02:00] Read. It's basically a category that contains all the different read functionality on the glasses. Speaker 5: Call. Speaker 4: Call is the video calling feature. Speaker 5: Identify, Speaker 4: Identify, basically contains all the different identification features Speaker 5: Fine. Speaker 4: Find this where we have all the different fine based features. Like you wanna find a particular object in environment, find people or just have the glasses speak out. Every single trick that I can see in your surroundings. That's something that you can do with the find. So I'm gonna quickly show you how the read category works. I've got a couple of [00:02:30] books here with me and I'm gonna use the instant text feature to quickly recognize those books. Speaker 5: Home greed, scan, text instant text guy, Kawasaki, PR of social media and head fits Patrick power tips. So we're users Speaker 2: Over the last several years, there's been a handful of other companies that have also created similar products in 2019. For example, Google launched the lookout app, which helps people who [00:03:00] are blind or low vision, identify food labels, find objects in a room and scan documents and money. Be my eyes is another app that connects people who are blind or visually impaired to side volunteers who can help them be live chat. What sets envision apart is that you don't have to hold up a phone to get help. The fact that it's a wearable makes it more intuitive since the camera is right next to your eyes. Other headsets like eSight help people with low vision and legal blindness. See the world more clearly. Whereas envision speaks aloud. What's around someone. [00:03:30] Mike May who's been using envision for a few weeks, says the glasses free up his hands and make it easier for him to use a cane or walk a dog Speaker 1: I've been at absolutely amazed in walking down a street. How much print is in the world? That side of people will probably ignore and take for granted if they're not really interested, but I've found fascinating. Uh, it's narrating nonstop when you're walking down a busy street about signs that are on the side of a bus or a taxi or on the out of a [00:04:00] building or on the ground or on a sign. And there's this, uh, whole stream of, of information. Speaker 2: In fact, during our interview may was in Hawaii and said he brought his envisioned glasses to explore the area. Speaker 1: I wanna get the, the scenery mode to tell me what's around. And then to go exploring on this, um, resort and find out what's around and I'll be using the glasses to help with that Speaker 2: Envision glasses became available in 2020 though. The pandemic of course, made it difficult [00:04:30] to reach new customers. But since then, the team's been working to make the glasses more widely available and people can now get them in 20 countries. You can teach envision to recognize face by having people take selfies in the app and then enter their name. After that the glasses can speak aloud. Their name when they're in frame Speaker 3: Looks like Alex from finance. Speaker 2: Another popular feature is ally, which connects users to their friends and family. Through private video calls. You can place a call directly from the glasses and get help with navigating [00:05:00] public rotation or making a recipe. For example, Speaker 4: They give you this, this unique first person perspective of the one that you're actually helping. So the allies can see directly from the camera on the glasses. They can hear the audio from the, uh, from the glasses user as well. And then the glasses user can hear the ally Speaker 2: May use the ally feature when he was going to a brewer recently and had a hard time figuring out where to Speaker 1: Go. I called up one of my colleagues, Evelyn and said, uh, what do you, what do you see? And she described the environment [00:05:30] to me and she said, oh, there's a sign that says private party. That's where we're meeting. So I pushed through those curtains, went into the room and then she told me where tables were and just kind of gave me the lay of the land. Speaker 2: Envision is also working on a feature that would let users connect with volunteers through ally, not just people they know the company also just rolled out some updates to make its glasses easier to use. Now, if someone holds a document and it's out of frame, the glasses will talk them through how to move that document so they can scan it. Speaker 4: [00:06:00] The next feature I want to go ahead and show you is scan, text, Speaker 5: Scan, text, move, document up, or your head down. Speaker 4: So I had a document like pretty much in frame. And then the moment it detected the entire document, it took a picture it process. And all of this stuff happens both on device and on the cloud Speaker 5: From one of the world's leading data scientists, a landmark tour, science of idea flow offering revolutionary interest. Speaker 2: Another recent addition is third party [00:06:30] app support, which lets other developers make their services available on envision. One of the first partnerships is with an app called cat reader, which lets envision users identify over 100 currencies, Speaker 6: 10 Swiss Frans, five American dollars. Speaker 2: The company is also working on adding voice commands to the glasses that's slated to become available later this year, Speaker 4: Users will be able to go ahead and use it very similar to how they use it with CI or Google assistant, right? Just say, Hey envision, can you read this? Or can you who's in front of me [00:07:00] and so on that makes it even more unobtrusive and even more handsfree than what it is right Speaker 2: Now. Now the glasses do come with a steep price tag of $3,500. You can order them through the envision website or from one of the company's distributors. You can also opt to use the envision app, which cost $20 for a one year or $99 for a lifetime subscription. The app can also scan text and tell you about your surroundings using your phone's camera con. And his co-founder started building the app in 2016, after visiting [00:07:30] a school for the blind in India, several students there said they wish they could go out or read a book independently. Speaker 4: We could see that at every Luke and cranny, especially in a country like India, where you don't have a lot of accessibility, uh, awareness, right? Uh, they felt extremely limited just because they didn't have access to this sort of visual information that's around them. Speaker 2: They started working on the envision app as a side project and within months had hundreds of users on the beta version, but their budget ran out and they thought it was time to move on. So they sent an email [00:08:00] saying they were shutting down the app. Speaker 4: The next day we wake up and we see like, I think 400 odd emails, all of them saying like, why are you shutting this down? Like this is, this is what we've been talking to everyone about. And this is exactly what we need and you shouldn't be shutting this down and it really made us reconsider. And so that's when we decided to start envision as a proper company, Speaker 2: They got some help from a local incubator to make it through the early days and quickly launched the app as a subscription service [00:08:30] that helped bring in some needed revenue since then envision has expanded its team and product availability. And last year raised over $1.6 million to build up its F looking ahead, Envision's goal is to bring other apps onto the glasses like IRA, which connects people to trained agents who can see what's around them using their phones camera. An integration with envision would mean users could connect to IRA directly from the glasses. Instead. Envision is also in talks with navigation apps to try to bring their services onto the glasses. Speaker 4: We see what [00:09:00] envisions building the platform that we're building a sort of this platform that anyone who's in the assistive technology space and they're building apps. They can easily come onto the envision glasses and build as well. Speaker 2: In the meantime, the glasses are already achieving that early mission of giving visually impaired people or independence. Speaker 1: I do really like the, the feeling that a kid get with. Aha. I did it myself.