Last.fm interview: Behind the music

People who listen to Last.fm have scrobbled an incredible 275,000 years of audio around the world. In the time it took you to read that last sentence, they scrobbled a whopping 4,000 songs. In its first week on Xbox Live, 120 million minutes of music were streamed. The company is also in love with open-source software and Intel.

We know this because we hung Matthew Ogle, the company's head of Web development, over a barrel of piranhas until he sung like a canary.

When we got bored with the piranhas, we tickled his feet until he told us how Last.fm pays labels and artists, the kinds of hardware and software it uses to ensure smooth streaming, and how the London-based company plans to bring about the destruction of Spotify. (Disclaimer: Last.fm is owned by the same company as CNET UK, CBS Interactive.)

Yeah, we're pretty good to you. The full, undiluted interview is below. Grab it while it's hot.

Tell us about the hardware and software used to stream music from Last.fm.

"We stream all music directly off our servers in London. We have a cluster of streaming nodes including a bunch of powerful machines with solid-state hard drives. We have a process that runs daily which finds the hottest music and pushes those tracks on to the SSDs streamers that sit in front of our regular platter-based streaming machines. That way, if someone is listening to one of our more popular stations, the chances are really good that these songs are coming off our high-speed SSD machines. They're fast because every song is sitting in memory instead of being on a slow, spinning platter.

"This is something that worked really well for us during the launch of Last.fm for Xbox and it's one of the reasons we were able to handle the launch without hiccups. The SSDs are made by Intel. We actually contacted a few guys and said, 'Give us your craziest, most cutting-edge SSDs,' and the Intel ones came out on top. Beyond that, our streamers are all running Linux and using MogileFS -- which is an open-source distributed file system, which is a little bit like a software RAID system."

How much data passes through Last.fm?

"One number that's pretty cool is relevant to our recent Xbox launch. In our first week of use, 120 million minutes of music were streamed.

"One thing that's even more popular than our radio-streaming service is scrobbling -- the process of sending the name of the track you're listening to to Last.fm's servers. You can scrobble from over 200 different online music services and desktop clients, such as iTunes, Winamp, Hype Machine, etc.

"During peak hours, we get more than 800 scrobbles per second which translates to about 43 million scrobbles per day. Since 2003, which is when we invented scrobbling, we've broken 35 billion scrobbles. That translates to about 275,000 years of  scrobbled music."

How big is the Last.fm music library?

"Thanks to scrobbling, again, we have pretty much the largest library online. Even if we don't have a piece of audio available for streaming, by virtue of scrobbling, we still know a lot about every track. Even if it's from your friend's garage band, scrobbling automatically creates a page for that artist and that track. Other users can then come along and fill out the wiki, upload images, and as more people listen, that track will get automatically connected to other tracks and become part of our recommendation system."

How do artists and labels make money from Last.fm?

"It's complicated, but we essentially pay per stream. Depending on which country you're in and what catalogue is available there, we'll stream tracks and produce regular reports to all the content providers that we work with and pay them royalties based on that. It's then up to the labels to distribute the wealth to their artists. We give the labels a breakdown of which artists should be accruing what royalties, so they have fairly good information on what they should be paying who.

"A lot of our content also comes from indie artists that aren't signed to a label. They can sign up to our music manager, upload their tracks to us, and get stats on how many times they're being played. For those guys, we have an artist royalty programme. There are more than 100,000 artists using that. You upload your music, tick a few boxes, and get a similar deal to the majors -- every time a track is streamed, you earn royalties and if you reach a certain threshold, you can request we cut you a cheque.

"The amount per stream depends on territory and on the advertising revenue that Last.fm makes in the given quarter, so it's not a fixed amount."

What are your plans to bring about the destruction of Spotify?

"We all admire Spotify and think they do a great job of being an online jukebox in the sky. One of the first features that Spotify rolled out in response to user demand was scrobbling. That's because, to a lot of passionate online music consumers, no matter where they're consuming music, if they're not scrobbling, then the music is being wasted.

"Even before Spotify reached out to us and we talked, they had engineers working on implementing scrobbling and it's been really successful. That's something we want to encourage more of in the future.

"We think we're part of a much larger online music ecosystem that our users are already aware of. They're already using a variety of sources to consume music and our role is to be the place that feeds back into and essentially that way we can act as the home base for your music life online. We don't think any other service, Spotify included, is really positioned to do that right now."

Click continue below to read about Last.fm's plans for radio and what the most popular artist of all time is...

Last.fm is a powerful tool that lets you see into the mind of the music-buying public. It could be very valuable to the record industry at large. What's your working relationship with the record labels?

"We have pretty strong relationships with most of the majors and they've definitely been interested at one time or another in pulling specific data to, for example, judge what an artist's next single should be. In aggregate, a lot of our data is very useful to them and we're obviously very careful about that. We are hardcore geeks -- died in the wool Unix nerds if you go back into our history -- so as a result our data is under lock and key. It is stored in ways where you can get useful information out in aggregate, but we feel very strongly about ensuring individual users' private data is not identifiable.

"Our users trust us with something that's very important to them and close to their hearts. Scrobbling essentially gives us permission to spy on our listeners' music habits and the reason they let us do that is because they get so much more out of it at the other end, including full data export. We're constantly working on new products that make more use of our data and we have a team of music scientists that spend all day figuring out better algorithms.

"We've got a few things coming soon including a 'Best of 2009' feature, which is based entirely on scrobble. With this, you can overlay your personal listening history for the year on top of the larger trends.

Any plans to sell music digitally?

"We've debated this for years. When I first joined in 2005 that was one of the things on the whiteboard. Ultimately, now, in a way, buying music online is almost a solved problem.

"There are tonnes of shops like iTunes, lots of speciality online shops for different labels and genres, so our role is to make sure whenever you find music on Last.fm you can purchase it from the best shop in the best country in the format you want. We've been working hard on improving our system for linking to ecommerce, because we think that's very important."

Who's the most popular artist in the Last.fm's history?

"This may be quite underwhelming, but just in the last month The Beatles have ascended to the throne of 'most scrobbled artist ever'. Their recent re-releases have given them the extra push they needed. For a long time Radiohead and Coldplay were the chart toppers.

"This presents huge problems to our recommendation algorithm guys, because if you enter either of those bands, it recommends every other popular artist. We've had to do a lot of work to de-Radiohead-ify our algorithms!

"Generally though, having The Beatles on top represents the fact our audience spans all age groups."

How many people access Last.fm on a fixed PC versus those on the move?

"At the minute, about 5 per cent of all our streams are mobile streams and that's been growing steadily since we launched our first official iPhone app, which is now about a year and a half old. There are plenty of other apps built by third parties, which is something we're encouraging.

"Anywhere you're listening to music is where we want to be, so our mobile-streaming strategy is just as important as our mobile-scrobbling strategy. We want Last.fm to be your definitive music profile and the URL that you can send someone when they ask, 'Hey, what are you listening to?' We want to be everywhere and mobile's really exciting for that.

"On a related note, there are a couple of different apps in development using our events API and creating gig finders in your city. They take the list of concerts we have and overlaying those on a map to show you what's happening near you on any given night."

CBS is launching over-the-air HD radio stations in the US, where playlists are generated by Last.fm's weekly charts. Why is a Web 2.0 company embracing old-school radio?

"It's tempting to think of it as counterintuitive, but it's really complementary. There's never been a radio station that's actually influenced by what real people are listening to around the world with rich data updated weekly. It's also great for getting the Last.fm name and brand and what we're about in front of a wider audience who are used to traditional radio. It's a gentle way of introducing the concept and they might be intrigued about the workings behind it and go and check out the Web site."

Are you planning to launch such a service in the UK?

"We'll see how that goes. We're paying close attention to it and if it takes off then who knows, but there are no plans right now."