How Go shaped a crowdsourcing business career (Q&A)

After the devastating January earthquake in Haiti, which left hundreds of thousands dead, many more injured, and more than a million homeless, the country's infrastructure lay in ruins. But because Haiti's cell phone network inexplicably survived, text messaging quickly became a way for people to put out pleas for assistance.

Not long after the disaster, the country's largest cell network provider set up the shortcode 4636, allowing Haitians to text a call for help. But with thousands of texts flooding the system, and a need for many of them to be translated into English, relief workers needed a technology backbone to help with the work flow.

And that was a niche that a San Francisco start-up called CrowdFlower was able to fill.

CrowdFlower has established itself as a leading provider of crowdsourcing resources for helping businesses and other organizations solve large problems. Among its clients are Microsoft, Princeton University, and O'Reilly publishing, and it had previously created an iPhone app called Give Work that makes it possible for Americans with spare time to assist in making sure that work being done for Western clients by Kenyan refugees under a program coordinated by the nonprofit Samasource is accurate.

In Haiti, CrowdFlower's technology made it possible to route the thousands of text messages to the proper aid workers, to get them translated quickly, and to ensure that the people sending the texts had the best chance of getting what they needed. Once CrowdFlower's technology was implemented, the average time to translate, map, geocode, and categorize a text fell to less than two minutes.

CrowdFlower's CEO, a Stanford engineering graduate named Luke Biewald, took a trip to Haiti in the middle of this process to help get the technology set up properly. This was not about making money. It was about trying to make a difference, and for Biewald, who had studied in Japan and who promotes the idea that people doing useful work helps their mental health, being in Haiti and trying to help the people there was an education in itself.

Biewald recently sat down for an interview to talk about his experiences in Haiti, his education, and the unlikely game that has helped shape his business career.

Q: Welcome to the second installment of "45 minutes on IM." Thanks for taking the time to talk to me. I want to jump right in. So, tell me, how did you become an expert-level Go player?
Biewald: How did you know about that?

OK...it's in your official bio on your site.
Biewald: I've always loved games. When I was in high school I read a book about Go and realized it's the best game. I was an exchange student in Japan mostly because I wanted to play a lot of Go, and in the year I was there, there wasn't much else to do. There was a chess master that said, "If aliens exist they might play chess, but they certainly play Go."

Nice. What do you like about Go?
Biewald: It has incredibly simple rules and is a beautiful game, the only well-known game that artificial intelligence hasn't solved. There's something beautiful that happens when simple rules lead to a complex system. It also teaches me patience and composure and handling ambiguity.

You're reminding me of the main character in the book "Shibumi," by Trevanian. You haven't also lived in the Basque region of Spain and been a world-class adventurer and assassin, have you?
Biewald: I love "Shibumi." And I wish....So, no one has ever asked me this, but I think it's helped a lot with my business, or at least it's shaped my style of management.

How so?
Biewald: Well, Go is all about handling ambiguity. For example, do I want a region of the board to be well defined or undefined? Beginners stress out too much about leaving territory unclaimed or stones half-captured. But you learn often that's advantageous. I think a big part of running an early stage start-up is not freaking out about things being left undefined.

Well, before we get to your business, I wanted to ask you about something else. I saw that in a previous position, your title was senior scientist. That's a pretty cool title. Did you wear a white lab coat?
Biewald: I loved that title. Computer scientist is not as cool as a materials scientist, but I think it's accurate. Mostly, I did experiments all day long. I was working on artificial intelligence algorithms to make better search results.

OK, let's talk crowdsourcing. Without talking about Crowdflower, tell me if you think crowdsourcing is really able to make a big difference in the world, or whether it might be more of an overblown trend.
Biewald: Sure. Obviously, I think the answer is yes--especially since crowdsourcing is becoming so broadly defined these days. I think an interesting question is, has crowdsourcing already made a big difference? Certainly I use the fruits of crowdsourcing all day long. Probably most of us on the Internet do.

Give me an example of how an average person would use it.
Biewald: Without user-flagging, most sites with user-generated content would be overrun with junk. Although you might say they are anyway. But it could definitely be a lot worse. Remember newsgroups? I think there are a lot of great tools for small businesses these days-- 99designs, voice123, getsatisfaction, etc., that are helping a lot of small businesses do more and get off the ground with less capital. And if you use Netflix, I'm sure their recommendation algorithm has benefited from the contest they ran. And it sparked so much great research, I bet most other forward-thinking sites with a recommendation engine also have improved because of it. And if you've ordered a pizza from Pizza Hut, you've used LiveOps, which I think of as crowdsourcing in some sense also because they use distributed call centers so people can work from home and take calls, orders, etc.

Was your experience with things like you've just been mentioning a big factor in your thought process in creating Crowdflower?
Biewald: Definitely. I think a lot of these practices work surprisingly well, especially ones with monetary incentives. Mechanical Turk pays people to do small tasks and relies on the requester to pay the person or not pay the person. There's no trusted relationship here, it's all brokered anonymously, and yet most of the people working genuinely want to do a good job and most of the requesters pay most of the people that do good work for them. And you might say, well, it's a marketplace where it's in people's best interests to preserve their reputation. But I still think people are irrationally--in the economic sense--willing to cooperate and work together even without social pressure.

How did that all come into play with Crowdflower?
Biewald: Big businesses have huge amounts of data that needs processing and they benefit tremendously from the scale of crowdsourcing, but they assume the quality won't be high enough. Or they're--rationally--unwilling to take a risk on unknown quality work. They think the kind of person who would want to do a random task anonymously won't generally do a good job. But, one, they're wrong about that. And, two, we can build automated systems to get very high-quality work, even if the worker population has some people who either don't understand the task or are actively trying to sabotage the system. So, obviously I'm not the only one to notice this, but I saw an opportunity here.

Well, let's talk a bit about Haiti. Talk about what you were doing there, and about your experience there.
Biewald: Sure, that's a great example of this. Right after the earthquake a brilliant guy named Josh Nesbit convinced ComCell and DigiCell to set up a shortcode that Haitains could message emergencies to. It was 4636. It turns out the cell infrastructure was mostly intact and was back up right after the earthquake. They advertised the number on the radio and had a huge response rate. The problem was the U.S. State Department didn't have a lot of people who spoke Kreyol available.

So people would text things like "I need water" or "I'm nine months pregnant and I need a doctor," and the Red Cross, Coast Guard, etc., wouldn't be able to get the messages translated for a long time. So Robert Munro, a friend of mine, and a linguistics Ph.D. student at Stanford, was working on automated translation. But it's too difficult--text messages are too idiosyncratic and the messages are too important to screw up.

So they started collecting volunteers in the Haitian diaspora around the world. But you can imagine the logistical nightmare here of routing text messages, making sure that everyone is working on a different message, and all in real-time. And it wasn't just translation--the messages had to be classified and also geocoded to be sent to the right agency and give the agency the right address to go to.

I realized CrowdFlower had exactly this technology. Actually, the State Department looked a lot like other customers we've had: a sudden spike of work, but not enough people to handle it, and also cares a ton about quality and short latency. So it was a really nice fit. And in fact we're still doing it months later.

Nice.
Biewald: One of our partners, SamaSource, had set up an outsourcing center in Haiti. So now most of the work gets sent there, which is even more awesome because it gives badly needed jobs to people in Haiti.

For you personally, it must have been strange to be in Haiti, a Westerner visiting this disaster-ravaged country. Did you feel out of place?
Biewald: It's a funny thing to say, but I felt privileged to be able to see it. I had never been to a developing country before. It sort of made me uncomfortable--something like disaster tourism. But it was also great to meet the people who do a lot of the work now and see some of the problems they have. At one point during the training the electricity went out. So they all pulled out their cell phones and read the manuals that way.

What would you say was the most profound lesson--if that's not too strong a word--that you learned while in Haiti?
Biewald: I think a lot of people figure this out faster than me, but to viscerally see how important jobs are and how much impact it's possible to have by giving people jobs has really stuck with me. It took several hours to get the Internet working and workers just waited patiently the whole time. I was really moved by how hard they worked and I hope we can send them lots of work in the future.

We're almost out of time, so I want to move on. So, tell me what your favorite new tech tool from the last few months is.
Biewald: Interesting question. Let me think for a second, I'm kind of a Luddite.

It always makes me laugh when people in important positions in the tech industry confess to being Luddites.
Biewald: I always tell people I'm the opposite of a lifehacker. Maybe I shouldn't admit that. So, I've been playing with Foursquare a lot since they came out on BlackBerry and it's been really fun. I guess they're older than a few months.

What do you like about it?
Biewald: I love the term "mayor." It's so perfect. It's the best title.

Are you the mayor of anything?
Biewald: I've been trying to become the mayor of Crowdflower. I put a sign up on our door saying "the mayor drinks for free." And a homeless person walked in and claimed to be the mayor. I love [San Francisco's] Mission district.

Well, we're almost out of time, but I have to ask, because I asked my first "45 minutes on IM" guest, Gary Vaynerchuk: IM is great. It gives you a terrific transcript and allows both people to be thoughtful and articulate. But it also lets you multitask a little bit. So, what else have you been doing while we've been chatting?
Biewald: Hahaha. Mostly, other employees have been IMing me, and also a guy I told who I was doing an interview walked into the room. And I have a feeling he thinks I just lied to him to blow him off. So I will have to explain what this was.

Well, you can show him this interview when it publishes as proof.
Biewald: I'll do that.

Well, that's time. Forty-five minutes goes fast. Thanks so much for doing this. I appreciate it.