Q&A: The formula behind FiveThirtyEight

Political commentary on the Internet is nothing new these days. So how did Nate Silver go from blogging under the guise of a chili pepper to hosting Election Night coverage with Dan Rather in a matter of months? By focusing on one number: 538.

While a number of sites and other media outlets offer aggregated polling information that can give a snapshot of the state of the presidential race, Silver's site takes things up a notch.

FiveThirtyEight.com--named after the number of votes in the electoral college--uses a predictive algorithm to determine the most likely electoral outcome based on polling data and other variables, such as pollster reliability and demographics.

Silver's methods are based on Pecota (short for Player Empirical Comparison and Optimization Test Algorithm), an algorithm he wrote to predict the performance of baseball teams. As an analyst at the sports media company Baseball Prospectus, he made a name for himself as a numbers guy with startlingly accurate results.

FiveThirtyEight is simple in design but crammed with numbers, graphs, and maps that deconstruct the data. The site also features a blog where Silver and writer Sean Quinn offer insightful commentary on campaign events.

Even though Silver launched the site as recently as March, its straightforward approach, daring predictions, and short but impressive track record has put it on the map of political sites to follow. The Washington Post featured Silver in its 14th annual election prediction contest this year, and he'll be reporting on Tuesday night's results with Dan Rather on HDNet.

As of Monday morning, Silver's projections had Democrat Barack Obama winning the presidency, with about 340 votes, and gave Republican John McCain less than a 4 percent chance of becoming president. Silver is open on the site about his support for Obama, but he claims that it shouldn't interfere with his methodology.

So far, Silver's methodology has held up well. His site became popular after he beat out most of the pollsters and pundits in predicting how the Indiana and North Carolina Democratic primaries would turn out.

Of course, the real test for FiveThirtyEight comes Tuesday. Silver discussed with CNET News what makes his site unique and what he plans to do with it in the future.

Why did you start FiveThirtyEight?
Silver: I'd been writing on Daily Kos for some time, and by March, the outcome of the primaries was pretty clear. Obama had a lead it'd be hard for Clinton to overcome, so it was to see which candidate would match up against John McCain.

I started really to lend some sanity to the media narrative about polls. There is so much hyperventilation when a poll moves in one way or another. People tend to look at polls that are outliers.

Why did you decide to start blogging under the pseudonym Poblano?
Silver: It wasn't really thought through. I just started writing on Daily Kos under that, and I kept that up for a while. Eventually, I started getting media requests. You can't name yourself after a chili pepper and do much media stuff, so I had to out myself at some time.

I don't like people who think they are experts at everything. The New York Times interviewed some astrophysicists on the election, and I'm sure they're great astrophysicists, but I didn't like their election methodology. I do work in other areas, so I wanted to build out my own brand (in politics) before revealing myself.

You developed an algorithm called Pecota. Can you explain how it works and how you've applied it to your site?
Silver: There are a couple fundamental things we're trying to do. One is take this big soup of polling and make sense of it. We rate the polls on how accurate they've been in the past and their sample size. We also look at whether polls tend to lean Republican or Democratic. Sometimes people think a state's moving a certain way when really, they're looking at results from a McCain-friendly pollster or an Obama-friendly pollster--but it may not be biased.

We translate things in terms of the electoral college. It kind of boils down to that one number. That's kind of the end product. People make the mistake of thinking the states where the race is closest are most important, but it's only when the popular vote's within a couple of points that this stuff matters.

Are you taking into consideration any exit polling or demographic trends from early voting?
Silver: Not directly--I wish we had some way to do that. Polls have different ways of accounting for the early vote. Some of these pollsters don't even ask if you voted early. There's some evidence there are factors to consider.

In Georgia, 35 percent of early voters have been African-American. That's a really high number--in 2004, that number was 25 percent. I do think that's another kind of margin for uncertainty to consider, especially since a third of the country might vote early.

Can you tell me about your text-messaging service with Credo Mobile?
Silver: We've been sending out a text message every time the polling thread is updated. Sometimes, it happens at 1 in the afternoon, sometimes it happens at 10 at night, so we send texts to people so they don't have to be nervously checking at their desks all the time. It's been a big success. We have 10,000 people signed up just for that service.

Do you plan to implement any other uses of technology into your site?
Silver: In some ways, the site's not all that sophisticated. I use Microsoft Excel to design graphs and charts, and upload them with my Flickr account. I'm not an expert programmer, but we try to focus on making it a reasonably attractive site.

You also have different layers where users can access the site. If you want to read the FAQ, it's 1,000 words, but we also present just simple red-and-blue maps. We want people who might hear about the site on the news or something to not feel intimidated by it.

How much traffic do you generally see on a daily basis, and when did the site really start taking off?
Silver: Right now, about a million unique visits per day. We'll see hopefully maybe 3 or 4 million on Tuesday. It's been a pretty steady progression upward. On the night of the Pennsylvania primary, we got 7,000 hits. After the conventions is when people really started to tune in--it's also when polls become more important. We started getting numbers in the six figures, and it kept going.

I think Sarah Palin helped a lot--people were obsessed with that. I think it's why young voters are more engaged--because there are more compelling people running, not just white guys in suits.

You'll be covering election night with Dan Rather, is that right?
Silver: Yeah, with HDNet. I'll be in a war room, and have a laptop and a bunch of Red Bull, and I'll say probably every hour how I think the night is going. We should know a lot fairly early. If the networks are able to call Virginia pretty early, McCain is in for a rough night.

Mostly, I'll play it by ear and be honest with people about where I think things are headed. Most of my users are Democrats, but if things look bad for Obama, I'm not going to sugar-coat things. I hope to steer away from things like race, and explain who won the election and why.

We want to see hard numbers; we don't want to see exit polls. In the primaries, Indiana was called for Clinton, but Obama came within a half point, so I think some networks were getting nervous. The exit polls have been horrible.

What are your plans for the site, once the election's over?
Silver: I think there are a couple different directions we could take it, and one will be to look at Congress. Whoever gets elected, you're going to see less gridlock--especially if Obama gets elected.

We could use the site to understand which congressmen vote for which bills, and why. Based on this person's district and history, we can say they should've voted for this bill and why. We can look at who are the swing votes in a piece of health care legislation.

We might also branch out. Any time you have (a set of predictive models), there are different ways to use it, like predicting movies that are going to be successful. We're really proud of the brand, and it'll continue on in some form.

How would you respond to criticisms that it's impossible to build a good poll-based predictive model for elections?
Silver: If the polls are wrong, then we're going to be wrong too. We can massage the data, but if there is some kind of a Bradley effect, for example, then our numbers are going to be wrong.

But we do emphasize that the actual errors in presidential polls are a lot larger than the reported (margin of error). In the primaries, the average miss was six or seven points. When the polls miss, they all miss in the same direction, so to that extent, averaging them can only do you so much good.

How did you team up with Sean Quinn?
Silver: He wrote to me, and I knew his writing from Daily Kos. He's worked for campaigns and knocked on doors, so that's been his point of view. I think that's missing from a lot of political news coverage. They'll talk about where the candidates are, and they'll have buses and planes following them, but I think the more important part is what's happening when the candidate isn't there at the field offices.

What do you think have been the biggest surprises to come out of this election season and out of your work on FiveThirtyEight?
Silver: I'm surprised how many different kinds of people have gotten in touch with me. First the pollsters, and then others. The founder of Craigslist e-mailed me and said he likes what I'm doing. I had a meeting with the French ambassador in Chicago because he said he'd enjoy sitting down and talking about politics for 20 minutes. You become sort of an authority on this--in some ways, there isn't much competition. Hopefully, the model will acquit itself on election night.

There is so much information and disinformation on the Internet--do you think the explosion of political Web sites ultimately is making us more informed, or is it just easier to read what you want to read?
Silver: I think that people definitely filter certain information. It also happens in cable news. You can watch a Fox News or MSNBC report, and get all your news from there. I think that is a danger. I try and read right-leaning blogs and Web sites (as well as) left-leaning ones, sometimes in a critical way, sometimes not so. There are certainly reasonable opinions on both sides.

It's always easy to tell yourself the story you want--there are national polls with Obama at +2 and polls with Obama at +13 right now. We are trying to create a better-informed electorate by giving people the big picture. It's only when you take the polls into context that they tell you a story, and that's what we're trying to do.