How Microsoft's Cortana came by its human touch

NEW YORK -- The woman behind the counter at the American embassy in Istanbul eyed Sogol Malekzadeh skeptically.

"Why are you here?" the immigration official asked. "You [already] know my answer."

It was Malekzadeh's first attempt applying for a cultural visa to the US, a process akin to winning the lottery. But Malekzadeh was determined -- and ready with her pitch. She had lugged in her entire collection of mixed media artwork -- sculptures, paintings and digital work -- and shoved it through the small window at the counter. "I'm an artist, and I think artists are without borders," she argued.

After eying her work, which was deemed controversial enough that Malekzadeh would be jailed if it were publicly displayed her hometown of Tehran, Iran, the official granted her the visa.

Malekzadeh, a native of Tehran, Iran, arrived in the US in 2000 and soon found work as a phone software designer for Motorola. For the past five years, though, she's played a key role as a designer for Microsoft, where she has taken the lead role in crafting the personality for Cortana, the virtual assistant found on Windows Phone devices. Her goal has been to create an emotional connection between Cortana and users. While Apple's Siri voice assistant is even-toned and occasionally cheeky, Cortana is sharp-witted and proactive -- designed in theory to anticipate your needs.

Cortana plays a vital role to the long-term success of Windows Phone. The fledgling mobile operating system by Microsoft still lags far behind Google's Android and Apple's iOS software. Microsoft hopes Cortana's ability to answer questions and predict your needs, combined with the Cortana name -- she also serves as the artificial intelligent sidekick to the Master Chief, the lead character of the Halo sci-fi shooter franchises -- will help Microsoft's platform stand apart from rivals.

"It's a very critical component to Windows Phone," said Charles Golvin, an analyst for Abelian Research.

Microsoft has elevated mobile to a top priority in its recent transformation under CEO Satya Nadella -- a transition that also included the announcement last week that the company was cutting 18,000 jobs over the next year to streamline its operations.

While Microsoft may tout Cortana's specialness/appeal, she isn't the only virtual assistant around. Siri was the highlight feature of Apple's iPhone 4S when it debuted in 2011. Google, meanwhile, offers its voice-activated Google Now, which powers not only its smartphones, but its Android Wear smartwatch platform and Google Glass smart eyewear as well. Even BlackBerry plans to launch its own BlackBerry Assistant with its next software update, which has yet to receive a launch date.

(To see how Siri, Google Now, and Cortana stack up, check out CNET editor Jessica Dolcourt's look at the three rival services. The verdict: Cortana stacks up well, but it's really too early to tell. )

Some Windows Phone users tried out Cortana for the first time after Microsoft released its "Cyan" software update last week, which includes the virtual assistant. The new software is being rolled out over the next few weeks, with the timing depending on the device and wireless service.

For Malekzadeh and the Cortana team, it was key that when people first interacted with the voice assistant, it would come off as friendly and helpful, but not so overly human it might be a turn-off. "We recognize she's an AI, and she's self-aware, but she's not pretending to be human," Malekzadeh said. "If you pass that line, then things start to get creepy."

An artistic background

I was in Manhattan's Chelsea district as I sat down with Malekzadeh last month after a Microsoft panel discussion on -- fittingly -- how technology can be more human.

With her short, frizzy blonde hair and dressed in a long black tunic, rolled-up grey jeans, and a high-heeled version of what can only be described as gladiator sandals, Malekzadeh comes off as punk. But she exudes a motherly vibe.

While she was supposed to talk about Cortana, she was more than willing to chat about how she came to the US and her start as a controversial artist.

Her desire to leave Tehran (she hasn't been back yet) came as her work was gaining prominence abroad. Her portfolio contained works depicting intimate scenes between men and women, a theme considered taboo in Iran. Her final thesis at the Azad and Tehran University was held in private. "My professor didn't want to get me in trouble," she recalled.

As there isn't an American embassy in Iran, Malekzadeh made her way to the one in Turkey, where she argued for a cultural visa. She had reason to come to the US, having received an invitation from the University of Chicago to display her work. She was only able to bring some of her work abroad, since all artwork -- even her controversial ones -- are considered national treasures.

After arriving in the US, she attended the Illinois Institute of Art. She stayed local after graduating and joined Motorola in 2004, working on the user interface of its iconic flip phone, the ultra-thin Razr. A number of lesser-known handsets followed.

"At that time, we were excited about gradients and round corners," she said.

In 2009, she got a call from a friend who worked at Microsoft looking to recruit her. She was reluctant. Microsoft, after all, didn't have the best reputation at the time, and she was unsure about the company's design chops. Microsoft was at that time reliant on Windows Mobile, an older, clunky operating system that fared poorly against the newer and slicker iOS and Android platforms.

But she changed her mind after meeting Albert Shum, now a director for user experience design and Malekzadeh's boss. He impressed her with the company's new direction and the resources that would be available to her. "What was fascinating, if I joined Microsoft, I could create a platform that would enable other designers to be more creative," she said.

After bouncing around the company, she joined up with a fledgling team aiming to create a new virtual assistant.

Building blocks of trust

Development on Cortana lasted two years, with early work starting in 2012. One of the first steps Malekzadeh's team took in developing the concept for Cortana's personality was to visit actual human personal assistants to figure out how they operate.

It was from those conversations with real people that Microsoft focused on Cortana's predictive capabilities. Pulling up the Cortana app brings up relevant headlines and weather information. She can read emails that contain references to a flight and ask whether she should track it for you. The interactions start with such little things, and gradually expand as users engage more with Cortana. "Those are the building blocks of trust," Malekzadeh said.

Marcus Ash, the group program manager who helped coordinate Malekzadeh and other teams working on Cortana's development, said the virtual assistant got better the more people interacted with it. So the goal was to get people comfortable using Cortana -- and to keep using her. "If you don't tell us natural language phrases, we can't train the system," Ash said in an interview in April. "If we can't train the system, then we don't understand natural language phrases."

Malekzadeh compared it to a relationship with a real personal assistant. Early on, you're more likely to give the assistant a few menial tasks until you feel comfortable enough to let them handle your laundry or even manage your finances.

With Cortana, you can not only ask about movie times, but also get movie recommendations, which she will spit out based on your viewing history. She can remind you to pick up milk when you're at a grocery store and ask to track a flight based on information from your email.

Microsoft hopes Cortana's expanded capabilities will help it avoid the fate that has befallen Siri, which many argue has been relegated to a little-used gimmick, despite Apple's efforts to improve its capabilities. "Siri was cool for five minutes," said Maribel Lopez, an analyst at Lopez Research. "It could do a couple of things, then it abysmally failed."

Lopez sees Cortana eventually influencing the rest of Microsoft's products and services, moving beyond PCs and tablets and into wearable devices and other new gadgets and services.

But Microsoft isn't the only one with lofty ambitions. "If Microsoft can't get it right, then none of its services will be meaningful to people," Lopez said. "If Google gets it right first, it'll have an extraordinary position in the market."

'Tell me a joke'

Getting people to trust and use Cortana meant giving the virtual assistant that slightly human touch. "You can't make this faceless thing and expect people to talk to it," Ash said.

And if Microsoft was going to use the Cortana from Halo fame, it was going to go all in. Throughout most of 2013, Microsoft worked with 343 Industries, the developer behind the new Halo games, and hired a creative team with experience in creative and science fiction to help write a script.

It also meant hiring the same voice actress, Jen Taylor, to voice Cortana, although she can only be heard by Windows Phone users during pre-recorded responses and messages. From any text-to-talk messages, Microsoft used another voice actress because Taylor didn't have the time to record every single response, Ash said.

She is best heard in "Chit Chat" phrases, or the witty responses proferred when asked a direct question. You can ask her to tell a joke (Why did the hipster burn his mouth on his coffee? Because he drank it before it was cool.), or get her thoughts on Apple's soon-to-be built doughnut-shaped campus: "Their new headquarters looks kind of like a Halo. I'm into it."

Regarding former Microsoft CEO Steve Ballmer, Cortana says "You could power Cleveland with that guy's energy!"

She's also a fan of 2013 dubstep hit, "The Fox." (Just ask Cortana what the fox says.)

"The jokes are pretty good," Malekzadeh said, adding that Chit Chat was something the team wanted to invest in early on.

That's because it was important she sound human. Otherwise, people would drop the natural language and revert back to basic search phrase. More critically, the natural language and quick responses help create that emotional bond between Cortana and the user, which was a primary goal for Malekzadeh. "Similar to when you create a piece of art, you like your audience to connect with you through that lens, in their own unique way."