Ep. 65: Debating the nature of robot mindsIn the wake of Watson completely obliterating its human opponents on Jeopardy, we sit down with two experts on human/computer competition to discuss the similarities and perennial differences between the two types of minds.
Hi everyone welcome to reporters' roundtable -- -- needle in San Francisco and big news in artificial intelligence this week. The IBM Watson project defeated jeopardy champions Ken Jennings and Brad -- in a three night primetime demo match. Before we get sort -- this just take a quick look at some highlights here. Watson what is agricultural you are right. What are tools you're right but what is staggering genius that's -- what is sanskrit sanskrit is correct that what is flowers for -- them. Yes you. -- -- -- -- -- that was. IBM Watson -- thing that -- the human competition there and the question we're gonna be talking about on the show is what does this mean this win mean for computing and more importantly. For us for humanity. That's the topic of this week and we have two great guests to discuss -- And both have current books on the topic of computer. Verses human competition first up we got Stephen Baker who's the author of final jeopardy man vs machine and the quest to know everything. Steve reported on the development of Watson from inside IBM headquarters to write this book and he -- businessweek's senior technology writer or that thanks for joining the -- Also branching out a little bit from the whole Watson thing we have Brian Christian -- He's the author of the most human human what talking with computers teaches us about what it means to be alive this book will be out on March 1. He's also the author of the recent Atlantic cover story called mine verses machine which is a great great -- on this whole topic. Both of these works -- the story of Bryant's participation in the annual -- their prize which humans. Face off against turing test computers to convince judges that they are in -- human and the computers are not. Brian. I should note. Won the competition for the most human one of all the people in the competition and machines congratulations Brian I hear -- and thank you thank you for joining. That's what the now. I wanna show 01 other click here before we get into the discussion this is two things here first this is Watson doing what he does best. This -- that one and then I wanna watch him flopped completely. -- not so modestly titled his memoir a heartbreaking work of this. Watson what is staggering genius that's right. Good job -- category now while it's -- answer. I'll wager 2127. Dollars weird bad. The New Yorkers and they were always -- CNET review of this said in its brevity and clarity. It is unlike most such manuals a book as well as a tool. Let's try what is Dorothy Parker. Now that was a very interest thing -- -- That was a very interesting answer because obviously the the answer called for in the title the book and he got completely wrong if you thing he does not me every time I do that. So let's get into this discussion here first about this jeopardy problems the jeopardy tournament. Steve what problems had to be solved in order to make Watson a viable contender. Like safest the key its simplest -- Watson and he could see it if this is listed in that. Elements style question was figuring out. What that question is asking for -- looking for a person of a type of a book in it in that case -- -- -- tool as well as a man so Watson's busy looking at tools and manuals -- -- -- just statistical calculations. And it comes up utterly befuddled and configured and ends up with Arctic are on the 14% confidence basically. -- arms and now. So what human skill then it or skills and how many different skills as Watson trying to do with little computer brain in order to compete with the humans. -- first it's not a little computer. It's nice -- -- nine. Industrial sized refrigerators and physical fitness fiction they just look at about our -- -- -- see it in a struggle so much to match us -- with all that computing and all that electricity but anyway. It it is sending out. A hundred different algorithms the same time each at different specialty. And after it parts that's that these subjects -- for what it's looking for -- each algorithm brings back its own. Candidate answers and then Watson in May be half a second has to decide which 82000 potential answers he has confidence. And and -- it -- enough confidence to -- Now. Brian in earlier we were talking about an email on you said that some of these questions just seem to be control left questions can you elaborate on that. Yeah well that those -- that struck me especially about for example The Beatles lyrics which I thought was really just to give away category. So I mean one of the things that's really exciting about Watson. Is that it represents I think -- step -- of things like Google where you have to be extremely explicit and precise about you know what you're searching for. On such questions like hate the -- don't make it and -- -- -- and make it better to me -- We're disciplining as a human because I knew the machine's gonna -- in Latin but it also seemed like. From IBM's perspective they weren't showing Watson to its full advantage. I'm whereas a question I thought was extremely fascinating. Was the question about the Baltimore character from Harry Potter after -- there are gloriously and mention mobile. Latin name so it's it really defeats that control -- Approach in this pretty fascinating. But house at that we can answer that and the computer camp because of that was an easy question for anybody who had even a passing familiarity with Harry Potter series a wise and that's that question was so easy for us. Well part of what's so amazing about human language is our ability to sit reference points really early on in a conversation. And -- to continue to refer back to you so computational linguists have struggled. Immensely with simple things like Providence. So I say. You know I took the pizza out of the oven and put it on the table it's obvious that it is the pizza. But if I say I took the pizza -- of the open and closed it. It's obvious that it is the haven't done any sorts of things you don't think about it all. But treatment involved. A huge amount of real world knowledge on in the same thing goes for -- as -- referred people so. You know it's not uncommon you you can imagine that seen in high school where someone comes at the -- -- -- heady talk to hurt it. And they know exactly -- that refers to but to any outsider it's you know. Gibberish you know -- it's meaningless word in this this. Becomes relevant when you were trying to convince people you were human and computer which will talk about later attorneys -- frame right now it's Steve. Do you think that the Watson programming team overlook certain aspects of human dialogue or or intellect and creating lots of computer based on what -- -- on the jeopardy game. Well I watched many many practice rounds before this I sort of new Watson's -- -- and I have to say. There are the control F things that that -- was talking about. And Watson early in its development could handle because it is easy. Beauty you know it's just that felt like and it's not too difficult to figure out what its pocket -- the question is asking for. But in many questions it was really really hard to two or three years ago. Watson would not have done nearly -- what -- lost an embarrassing. Lott lost because it couldn't wouldn't have figured out many of the ones that were -- that and The Beatles lyrics but it's not it's perhaps not as part of the Baltimore want. And so they got to the point where they can get. 80% at the clues and and diet and getting better get about ninety -- Now say they got clothes and they were dealing with it what is -- that sentence what does this referred. No one uninteresting things about this this contest was that it strikes me that the jeopardy game is created for human contestants. Now I've never really thought about the -- if the game obviously players -- human. But in this particular case in this game one of the players is not human and -- different strengths in different. It has a different strengths and different weaknesses. For example is very -- yeah yeah for example reaction time -- pressing the -- I mean Watson had a solenoid over the buzzer. Which was kind of a cute little thing we could just part -- soldering wires there are so he had to like -- electrical impulse from -- went down. That was -- only human in -- written. Human like aspect there. Other and that he didn't read the questions. Using his eyes its eyes it didn't listen to the host to read the questions out loud. If we took the -- -- challenge out of the equation. And made speech recognition they are OCR optical character recognition part of the -- how -- we think Watson would have done. -- Watson is sub human in so many ways it's it's too long -- a list right. -- what they did was -- they started with this very dumb machine. Four years ago and that took two hours to answer a single question and did it very poorly. And buy a working at at and make -- Sort of endless improvement the -- the Japanese do -- auto plants. They increased its -- increases performance -- where it could win a game against humans. This does not mean that it can perform human level and any other in the other way except that there is limited world of jeopardy. Now -- were you involved in I know you. The Steve you. Were reporting from within IBM on the development of Watson where you also able to get inside of jeopardy if they noted they craft the questions given. Knowledge of who the contestants were verses how they normally do it. No and this was a matter of huge debate between IBM and jeopardy in jeopardy at one point. Was so upset about this that they -- they were picked up pulling the plug hole competition with back ethnic and because. IBM said we have built a machine that is going to win a game of jeopardy and the jeopardy people say we want good entertainment and we want a fair match. Now so they start saying. Is it fair Watson has its advantage in both speed and so it at one at that point -- that this had to build a -- for -- at odds with email. Right okay and the -- -- -- -- like shattered all basic picture you're trying to. Impose human constraints on our machine our machine has other enormous handicaps languages and higher before end -- this machine. It doesn't know what this isn't a question about all important it's easy for humans so we have an events and -- evidence of this so that the jeopardy of identities were also saying. Listen we are really worried when -- -- human judges are writing the clues for this game. That subconsciously. There's gonna be this bias against our machine. And that. They are going to basically change -- -- a game of jeopardy into a turing test and guiding all kinds of clues that have a new windows and difficult wording. To trip up our machine and -- -- was. Really concerned about this and they -- -- heated arguments of the -- with the at jeopardy -- about it. And finally. Edit -- -- jeopardy for a franchise because he was I am saying. You guys have biased the idea. Bias in the game show -- ways and yeah text and out of the 1950s they had became shows scandals and -- -- they could use in this match is. Is any -- and it. Implication of them not being entirely fair. So they finally went back and -- you know what we're gonna use under games we've already written. And we're gonna take thirty of those -- and blindly. And have you noticed outside firm picked them and they they went to great lengths to make hazards -- games were not. Written for or against -- machine the want to have to say -- if the machine lost its timing came back for rematch. It would be facing a turing test because that's human -- could not help. But defend their franchise by pushing pushing jeopardy more towards what what makes humans special. Well we haven't gotten the turing test question here but -- but. Brian any any comments on the idea of writing questions for a human -- -- computer. Yeah well it's you know I didn't -- people asking me -- where you rooting for Jennings and writer or Watson. And I said -- you know really this is a three party contest. It's the human contestants Watson and the writers so. At some level I feel like I'm actually rooting for the writers. More than -- -- for Jennings and -- because you know they're they're out there doing their best. On in some sense though I thought that. The the verdict would fall to the types of questions so that really it seemed to me that the competition was in the hands. Of this show's writers. That and yes but this but the thing is it it would. You know idea did this -- an engineering job and so they had a they had 200000. Jeopardy clues through history that they trained machine. And so the machine with used to -- and was built for those types of question that if they changed to a turing test. There's no cup this humans can always beat the machine and if if it comes underwriters thinking about it. Now one other thing I wanna talk about it here briefly before we get on to talking about turing tests and and the low enterprise and map which is fascinating. Is Watson. This. For all its bulk and and computational horsepower and storage -- -- that was disconnected from the Internet as are we. Humans most the time so fair's fair right he couldn't Google anything it couldn't Google anything -- -- it's. Our it couldn't do anything that why this artificial constraint and and it was any discussion about should Watson be tapped into that to the global intelligence are not well. Say they knew that that would. They had. Strictly fair competition each contestant had to rely on his or her -- or its own. We're -- resources and other people Bratton and couldn't use the oxygen. And you know one thing if -- just think about this towards the there's all this. There's it could have been any reports of state has had a bytes of head of -- -- Watson's. Data cache when it's 75 gigabytes -- it and is that right yeah. Because it couldn't make it because it has search and come back and in a second and have it goes Bluetooth. On endless -- that transit optimize so why. If he'd got simplify its it it it if it has 75 gigabytes of storage in -- -- -- wide -- that need. Nine refrigerators. -- in order to access a single -- and a half inch hard drive. Because it goes through the city but gigabytes that. All these different algorithms each one looking what -- looking for words that Ryan and -- looking for certain types of puzzles. And and it's -- in Nantes. Sort of an industrial. Each each clue is an immense industrial operation. What evolved so although it basically distributes all -- -- to 2005 and. One of things I read recently with the last two weeks or so I think an -- I think this is my associate of mine working here. Is that somebody did a study of all the computing horsepower there is on the entire planet all the CPU's all the -- connections all the embedded processors and all that stuff they figured out that. The combined planetary. Computing horsepower with about equivalent to one human brain. They should go through that setting any comments -- and it's it's. Fascinating statistic. -- about a -- -- very vocal crowd. -- -- -- in there that there is some great statistic I read and -- you probably know what you -- know what the power consumption of Watson was. -- but justice to think about this roomful of machines. That takes two air conditioners that's using you know -- megawatts. On and Jennings and -- -- these three pound processors that are running on you know it tuna fish sandwich -- it couldn't cola. I think that's actually sort of an exciting I think about. What it what is it about the mind the brain that it that is those -- -- For. And if -- Darwin or hours or god. It shouldn't question. On the well part of what I think it's so interesting is that. It's -- -- that's really giving us a fresh look at at that very question so -- if you go back over the history philosophy. There's this obsession all the way back to Aristotle. With just what is it that makes human beings unique -- and especially -- antiquity they're really it says with well why do humans have souls but -- Do not capsules. On in the -- -- answered that they keep coming back to. Is well we're really -- -- dealing with abstract concepts like numbers we're really good at making logical deductions and all of these sorts of things. We're really good at contemplation incident philosophers -- kind of patting themselves on the back. -- What to me is really fascinating about the development of AI is that it's precisely those things which we once thought we're so special. The ability to do mathematics to make logical deductions and so on. -- It's precisely what the computer has been able to do far better than we -- so I think it forces us to kind of go back. On this ancient question and really start revising and asking yourself well okay what is it that says special about us well let's -- Extend them that by talking about the competition that you were involved in which is the -- enterprise. Which is a turing test competition give us a little. In a nutshell what is the low enterprise and -- what with the competition you're involvement. -- -- The turing test is basically a conversation. Test so instead of asking and answering trivia questions you're just happening -- -- Pretty standard informal chat conversations over on a text protocol. So what's interesting about it is that. The computers are trying to beat that humans so much as they're trying to pass themselves off as humans. So you basically assemble a panel of judges together that are having these short chat conversations. Both with a bunch of human strangers and then with a number of these chat -- who are pretending to be human strangers. On and so the -- really falls and the judges. Over a span of about five minutes to. How can they steer that conversation in a direction that's going to expose. The ai programs from being the frogs. They are and what's going to enable them to identify who the real humans actually our number and you participated on of these -- -- 2009 wasn't. Yeah that's right I'm I'd seen in the newspaper. -- it is very famous prediction that by the year 2000. Computers would be fooling -- about 30% of the time and as a result we -- Sort of commonly talking about them as though they were intelligent and can think it's. On this. Particular prediction didn't come to pass. But in 2008. The top permit that the enterprise came -- just one vote shy. Meaning that 30% threshold it managed to fool I think 25% of the judging panel. On so there was this kind of scare I think that went through. At least I I felt a little nervous or anxious about that result. And to some extent wanted to see if there's something I can personally do to kind of stepping in and defend that defend my fellow -- sapiens and you did. I did -- and. Yeah and so you've got into the 2009 competition. Two to give -- the results here used and managed to convince the judges that you were more human than any computer. Yet congratulations well thank you -- how. How did you that what secret sauce is there to did you have to study your opponent computers before you did this or what is it that made you. Different from the box. All. I did. One of the things that's been really interesting about the contest is that. Organizers generally till the human participants to -- to consider its. On you don't have to really prepare you are given so just get out there and be yourself you know there's nothing to worry about. My reaction was well I don't know what I think there is something to worry about. In a sense that on. If for one of these programs have been developed over the course of many years. By people who are and it singularly bent on figuring out how they can crack the turing test. I'm so just to show up and do whatever came naturally seem like to me it'd been uneven playing field. But beyond that there was kind of this deeper psychological and philosophical question which is. The goal of the true test is to have the good conversation. On which isn't always -- -- excellence completely naturally to -- you know we have. Public speaking coaches and you know conversation coaches and we -- -- friends for dating advice you would but -- you -- an effort to date. -- So it does seem to me that. DDR if you -- of conversation isn't necessarily something that comes completely naturally to humans. On and -- be part of our strategy was just sort of trying to figure out what makes the good conversation. Now let's talk a little bit about adaptability. Both Bryant and Steve chime in on this one. It's widely believed by the general population that computers are formidable there are such great. Competitors to us because they're so adaptable something isn't working -- betting strategy in a in a wagering game isn't working they adapt. And the humans get stuck in conceptual intellectual rights you can't teach an old old matrix. But people who the chess playing computer was dismantled after beating -- off. And in 2009 you managed to beat the computers at their own gain in the local enterprise what what is going on here are we in fact. Not the old dogs and our computers more stuck in their -- than we are. I'd certainly like to think so on I mean what it thinks it is really fascinating for me when I was going back in doing research on the deep blue incomes are matched. Was that in a long chess series like the you know the world championship series. On you've got -- real kind of you mind game where you're preparing certain opening news. And they're trying to guess what you're preparing and prepare something against stating you're trying to -- -- -- and song. And -- actually turns out to be a huge part of chess strategy just as it would be that something like poker. -- and I thought was very telling. That it was human grand masters hired by IBM. That were having those discussions every night in basically re setting machines strategy for the following day. So even though during that game machines on its -- These kind of strategic adaptations. We're being done by real people so to me I mean. To me that speaks to the sense in which. One of the real defining human characteristics to my mind. Is that ability is that real memory ability that ability to always sort of figure out what's going on in do you one better and yeah that's that's what Stephen as saying that. If we were to have agreements. I love that jeopardy contest. It would be hard to imagine that the writers. You know are already scheming up new more difficult -- questions button on and to me that's really part of the defining thing about the human. Mind and also the human spirit. 01 Watson one of the things that. Distinguishes -- and I think is its ability to look at the gray instead of black and white. It weighs evidence and this is in this it's unlike traditional computers a traditional computer. You know Watson was made to look foolish on it on a clue in which it it seemed answered that the rocket with US city right. As traditional computer with have a list this have a list of Toronto's Canadian city holistic -- -- US cities and would not even considered -- as possible. As a possible answer that question. But that same computer would not consider that Alice Cooper could possibly be a man. Or that eight -- walk could -- in Manhattan because it it has these rules and Watson doesn't have rules Watson has evidence. And so it looks at it it says -- I've got to have -- I might not understand this. I have to have -- open mind if there's a lot of evidence comes in that case -- the chip itself but if there's a lot of evidence that comes in. It says Alice -- is the man that I've got to. Basically. Disregard. The evidence that Alex -- the girl's name and and and perhaps even -- on this woman's name being a man. And an -- watts it's flexible and understands great and has an open mind. And one thing that humans -- humans tend to know things and once they know things oftentimes we don't question. And that makes -- inflexible and -- Interesting so where do we think we're going to see computers competing with the human mind next first both of about. You know games -- for example. I read a story poker is an unsolvable problem when it comes to bedding in and reading -- it it is that the next great grand challenge for computing. If she could keep the good poker face that's -- -- they think that's that's that's but wouldn't wouldn't humans be able to recognize its patterns pretty quickly. -- I don't know I mean that yeah let's. That's much more of the deep blue -- computed -- in the Watson tied I have to think it's Watson. Really just goes through a lot of words and knowledge but that's all I'm not as much I don't know much about this game play Italy backtracked a little bit here program curious about people who we didn't talk about that a chess playing computer -- -- -- -- or culture created by IBM. What it's what did deep blue peach Watson. Deep blue taught Watson that if you take on a great challenge to get terrific PR here come and that. That's is that jeopardy game what's really at IBM commercial more than anything -- -- and and and idea needs those sorts of things that to convince people and to show people that they do exciting things computing they're not selling products I talk. Like the summit idea and -- outside these these things but. I -- it. It. We talked many analysts like -- on the computer would never do that. Ever talking about the next great challenge poker what and what people who taught -- -- -- lake club lots of its. Apparently distributed parallel computing massive parallel. And -- -- Computer architecture less than anything else -- rest of it it's very very. Okay. What about and other pursuits now. When we talk about them and the battle of the wits the battle the mine's rather between. A technological mind and organic -- one thing that comes from my mind. Is a battle of war -- you've got -- Strategies and tactics and information overload and and the father of -- In this contest between two mines and their resources is this an area of of computational challenge that you guys are aware of. And I personally I can't speak to knowing exactly. What the cutting edge in need in -- Modern Warfare is. But it strikes me that warfare what -- one of the really critical. Components of fighting war is. Being aware to not only what the situation looks like objectively. But what information. What the situation looks like to -- apartments in what the opponent thinks the situation looks like to you and what you think they think you think and you get this -- -- spiraling. Out of control. -- version of points of view and. I think is going to be one of the next challenges and -- is how to deal not only -- situations where there's a certain. Objective quality -- You know Baltimore is the enemy Harry Potter. The energy is the person in this particular Beatles song. But in -- question is. -- maintaining. It and comparing it to different subject duties. That means that we haven't really seen yet and. I really intrigued I mean there is that there's quotation from the youth and computer scientists think his -- this markets. Met Marcus the -- that. On what you're saying really the ultimate game for -- computer is something like survivor. We're in order to -- survivor you've gotta be aware that -- like both brick thinks that Kelly thinks I'm voting fertility but Kelley knows that I know that Rick you know. -- You start getting into those situations and -- I haven't seen any kind of AIX and -- That's their proliferation of subjectivity is rather than -- the really -- awareness that this objectivity. That's that's a really long -- -- apartment for computers trying to figure out what. People other people might be thinking about what it might have heard all those things now one of the thing go head start it's just it's -- workfare goes. You -- there's been at the end of there's justice growing. Massive -- warfare is creating ads and soldiers in battlefields all the rest. Ours are kind of carrying all these sensors and each one of sending back Alderson from H and so. And so that's the kind of a guy said it goes into. Bat like this. Analyzing -- battle might have more to do with the sort of intelligence expert on it and advertise. The satellite -- the information now. What's next for Watson and for that for that -- and we've seen reports that that they're gonna take the Watson technology put in the health care is that. What's next that -- again. -- I don't think one thing is that I had better I said that this whole thing with a -- right I -- It's not a human being but if that's the place -- the human being to get on national television and create awareness of the school's technology. So once they get on television and get all the attention that they say all you know we were just kidding we're really not trying to -- humans and we really have Al. That's -- fallible machine makes all kinds of mistakes but. It is really good at digging through masses -- mass of unstructured data using its its language abilities. To come up with hypothesis it's. And so then you don't say oh it really screwed up one Toronto and Baltimore it's that it thinks that's okay if you come back if you go -- much of medical data and come up with. Ten diagnoses. And two of them are good. Then that's -- get their eight make the ridiculous that it's up to the humans to make at this instant and so Watson. Could have a really great career that type of technology considering them you know if they can make it cost effective. At digging through documents in and coming up with answers that call centers. You know legal offices -- papers. Legal -- that that's -- player's -- Well -- immediately for precedents -- -- legal precedent -- watts and fifty. Of good you know we could look to put Washington in charge of the patent office. That's exactly right yes. Now I think that's actually a perfect example yes now. How do we make computer mines compatibility compatible our own in -- cut thrust of daily life and one of the things that people talk about a lot is robot cars which is all fine. If all the cars on the road our robot cars then they all kind of have an understanding of this for Persian it doesn't exist to think I know the program in that car -- it's the same as the program this car or can look it up -- -- a different one. And they know what the -- gonna do but they can't plan for what humans are gonna do so how do we solve for this blended environment we've got. Intelligence is that are incredibly fast but rigid verses. Our own which are just well it's a different kind of intelligence how -- you -- the two worlds together. In the real world. That's a great question. -- I really -- and anybody doing work on that. Well we we all have to do it right here I mean because -- that it -- these machines we have in our pockets and they're gonna get smarter and smarter we all have to have come up with a strategy that what we put our own heads. And how we use how we use machines that are being used by its patent -- -- and. I think that's right I mean it's really interesting. If I can remember when -- -- people's phone numbers. And now I know people that don't even know their own phone number. Found that S. Alone as we sort of get these prostheses. You know on it sort of changes. Us you know -- the extender capability so it's become slightly dependent. And I I think one of the most important things is just to sort of. Stay aware that. You are. Interacting with technology you know that. When you use Google you're not just tapping into pure information but there is a particular company in a particular AI algorithm. That is kind of. -- mediating. And it's got certainly. You know limitations in certain capabilities. -- so -- think part of it is is to not allow the technology become so transparent. That we aren't aware that our experience is being mediated by typical. Interest and welcome to to begin to wrap this up what's next -- we do watching for and we we've seen computers BS that -- at jeopardy. We know they can drive. The turing test they are getting. Well first better than works but as the next you'll be better again. What are the next big challenges and that's the next big milestones in computers -- human mind contest that we should be watching out for -- 11 better be -- uninterested in is. The DARPA is funding. Is looking -- vision programs. And trying to teach computers to -- things and reach conclusions about and understand that not just announced that herbs. Like if two people go into a building one -- goes into building carrying a package and yet they got and then comes out about the package that -- and that. Program should assume the way -- to -- that left. The package there. Button and so that's it sounds and that if something you'd have a three roll and understand that stand. And compass conclusions but the real challenge is teaching computers but three year -- can do. You know that has -- three girls are remarkably. -- remarkably intelligent -- Miraculously talent to and that's that's -- hard thing to teach computers it's that basic this stuff that's. -- Brian Brian what are you watching -- well. I mean it is interesting I think we are probably a number of years away from seeing a fully fledged in -- turing test beating piece of software but what's really interesting to me. Are -- ways. That that type of software is kind of creeping into our lives so. We've all probably had that experience of getting an email from someone where they're really excited about an iPod discounts and we think -- I'm not so sure that this friend of mine is actually. Sending me this. I its X you know we're basically dealing -- -- tests I think. It's true test is really creeping into 21 century life. On and so one of the things that really fascinates me. Is that if this this concept of what does it mean to. Communicate in this really original and idiosyncratic. Way you know to use novel metaphors to express that you know a sense of style and personality. Questions like that which. Maybe it when it twentieth century were more esthetic questions. On now -- questions even -- things like computer security. Mom if I talk -- -- really it really idiosyncratic way you'll be more likely to know that I'm not standing yet. -- so this weird I think merger. Things like. What does poetry acting with computer science. I don't know the technology is forcing us to ask a lot of really interesting questions I think for the first time. -- you do it so you're also poet. And yes I'm waiting a little biased and being excited -- that are and are you worried. Are you worried. The policy let me know when somebody you know when there's a computer -- that is worth anything. And yes story -- that it gentlemen thank you very much your time we had with -- them. Personally thank Stephen Baker is the author final jeopardy man -- machine in the quest to know everything. Steve thanks very much for making the time to join us. Thank you. And also Brian Christian who's the author of the upcoming book the most human human what talking with computers teaches us about what it means to be alive with. Which will be out on March 1 and is also he's on the cover of the Atlantic. And recovered for the Atlantic issue which is -- right now. Frank thank you -- for -- the time. Thanks it's -- pleasure. Thank you for producing thank you everyone for watching this edition of reporters' roundtable will be back next week with another great show I don't know on what yet if you want to offend the some topic idea send them to roundtable at cnet.com. If you -- find the conversation. Notes for this and some links to discussions that we were having here it's reporters' roundtable dot cnet.com. And until next week thank you for watching.