Microsoft wants to help put disabled people on equal footing

My first brush with accessibility tech was with closed captioning.

Like most people, I took it for granted and mostly ignored it as a setting on my TV. But one day, when I was fed up pausing my movie for the billionth time because an ambulance was blaring down our street, I decided to turn it on. Over time, it changed the way I watch TV -- to the point where I miss it when it's not there.

This is the ideal for people like Jenny Lay-Flurrie, Microsoft's head of accessibility. She doesn't just want the world to accommodate people with disabilities, she wants technology just to get better, and as a result benefit disabled people.

One example is live transcription, something Microsoft's been working on for years. During the company's Build developer conference last week, it showed a program that can identify people in a meeting, transcribe what they're saying and even pull out to-do items from the conversation. Of course, many people in the disabled community benefit from that idea, but so would I as a reporter.

"It's so bloody logical," she said.

That drive has already helped Microsoft create some amazing stuff. One example is a wearable motor designed to sense the shaking experienced by a Parkinson's patient and then shake in an opposite motion, allowing them to do simple things like sign their name or hold a cup of coffee. It also developed a free app called Seeing AI, which describes whatever you put in front of it -- whether that means reading a menu at a restaurant for you or identifying how much money you're holding.

And Microsoft's Xbox group last year released a feature called Copilot last year, that lets people use two controllers to play with one character. That made it a hit with parents and young kids who wanted to play games together. It also helped disabled gamers more comfortably use two controllers in different positions, or even mix a hacked controller with special buttons along with a standard one.

This year, it followed up with the Xbox Adaptive Controller, which is designed to plug into nearly any button, joystick or pressure sensitive tube that's been designed to help disabled people use technology and play games more easily.

• Read our story about the new Xbox Adaptive Controller for accessibility.
  

Lay-Flurrie, whose deafness set in at a young age after a bout with measles followed by a series of debilitating ear infections, usually has an interpreter Belinda "Bel" Bradley with her. But that live-transcription technology is getting so good that Lay-Flurrie says she sometimes doesn't feel like she always needs Bradley with her.

To which Bradley quickly chimed in, "I'm figuring out how to dismantle it."

Here are edited excerpts from my conversation with Lay-Flurrie last week, just before visiting Microsoft's "Studio B" building in Redmond, Washington, to see its new Xbox Adaptive Controller.

CNET: Why do you think this trend of accessibility is finally happening? The tech industry's existed for 30 years or more. Why only now?

Lay-Flurrie: The World Health Organization's defines "disability" as a mismatch between the individual and the environment in which they're in.

I'm no not broken. Some may disagree. But for the most part I'm not broken. I have a disability that gives me a mismatch. That's a beautiful design principle.

How can technology actually be much more of a bridge between an individually environment they're in? It doesn't just empower them, or me in a deaf scenario or someone in a mobility scenario, but also it empowers everyone. It's better design, it's better tech, it empowers everyone.

If you look at history there was quite a bit going on. We've been in this space for 20 years. Bill Gates started it in '97. When I joined Microsoft  13 years ago, there were six groups that were humming and buzzing. There's now 15.

Has there always been beauty? No. In any 20-year journey you get a lot of ups and downs. But actually, there's been quite a lot happening in multiple different arenas.

What you're seeing is this heat up and passion around accessibility. I think it's an incredible techie nerdy opportunity for us.

A lot of this stuff still feels like an afterthought. It happens but it's an afterthought. I'm curious, what needs to be done where these things feel like they're even in terms of how much development effort and time and energy is being put into them? Or is it already there and I'm missing it?

Lay-Flurrie: There's always more that you can do. Where I differ with you is that we've moved to the principle of inclusive design, where it's not done at the end. It's actually done as part of the design process of thinking through  how that product's going to work for a human, including the part of them that's diverse -- whether that's gender, disability or anything else. And embracing that is a part of design and following that through the whole of the dev cycle. It means that you don't just get an accessible product, you get a differentiated product that can benefit way beyond.

Where do you think the tech industry as a whole is with this stuff right now?

Lay-Flurrie: This is not an area where we compete. In the US, the unemployment rate's double for people with disabilities. You're not going to compete when you've got that kind of demographic going on.

And none of us would say we've reached an end destination, because if you walk with someone for a day, and as a designer, as an engineer, as a UX, whatever your skill is, you will see a thousand opportunities where technology could have a play.

There's a lot more to do. It's not ubiquitous yet. It's not something that everyone knows about. There is so much in Windows or in Office, people have no idea that's just right there. We have something called Accessibility Checker in Office 365. And we moved it right next to spell check. It just checks your document to see, you know, how inclusive your document is.

In what way?

Lay-Flurrie: So if you're sending out a Word document or an email, if you put picture in there but you haven't put a description behind it, it will tell you, "you're missing a description, dude. Please go fix it."

I'm terrible about that.

Lay-Flurrie: AI is going to help, because it will grab the image and write a description for you. But we put that accessibility checker next to spell check, and improved 5x overnight.

A lot of people have no idea what's there.

Watch this: How an Xbox controller helped this disabled Iraq War veteran play again

02:07

Sensors seem to play a big role in accessibility. So how do you deal with issues like privacy? Because it feels like you have to break a lot of the molds of security and privacy to make all this work.

Lay-Flurrie: Everything we do has the same bar of scrutiny, security and intent. That data is personal data and we're not going to share it.

All that data is yours.

I'm not thinking so much about Microsoft, I think of the everyone. I mean, if you look at companies that should know better about privacy, they say, "well we weren't looking in this direction, we didn't realize, we were focused on other threats" and suddenly it all goes really poorly. And so that's what I'm curious about. Good intentions can still end up badly if it's not thought through enough. And you can never think through everything.

Lay-Flurrie: I think there is making sure from a pure technology perspective that you deal with the data in the right way. Those are principles we uphold.

But also, we have so much respect for the people we're working with. Most of us are people with disabilities in some way or impacted by it. We would never disrespect those individuals.

If we look to the far future, what does it look like, in terms of your job?

Lay-Flurrie: I got back to that principle: A mismatch between the individual and the environment in which they're in. Technology's a bridge. It should be ubiquitous.

Could you have a more inclusive society because that technology is not on dramatically different looking machines -- it's all on a device, and people are just an adapting or using it with a different mouse or with different software or a different setting?

And I think it's our job to give the right foundation, a platform and the capability, to adapt in that way. I want to see technology be much more that bridge.

Let's have the focus on that talent that's just walked in the room that may be a subject matter expert in data science. Let's let technology be the enabler.

So I look at the future in terms of what can we change, what can we empower, and then back technology into that.

One of the blockers with interviews is you may need interpreters. You may not necessarily be able to write a schematic on a whiteboard. In the future, suddenly it can be like, "the tech's here, leverage it however you want."

"Oh, you need an interpreter? Let's video one in and get one for you."

"Oh, you need a slightly bigger room? Well we'll just shift this wall with a power button -- done."

"You need the lighting a bit lower? You can control it yourself."  

You just think about all those scenarios that can be so prohibitive for an interview. And tech could solve all of them.

It's not a conversation about accommodating someone with a disability for new talent coming in the door. It's about "have you got the skills that I need for my job."

It's a more inclusive society I think we can empower.

A Better Xbox: Microsoft has designed an Xbox controller meant to help disabled gamers play like the rest of us.

Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.