Spotify vs Sky Songs: Sound quality blind test

Sky has launched a subscription-music service called Sky Songs, but its streaming-audio bit-rate is much lower than Spotify's free service. The question is, can anyone actually tell the difference? We put it to the test.

We dragged 16 people from around the CBS Interactive offices kicking and screaming into a quiet room. We gave them a pair of £500 reference-grade headphones and a high-end audio processor, and played them the same section of Michael Jackson's track Billie Jean twice -- once from Spotify, once from Sky Songs.

On paper, Sky's streaming audio quality -- at 48Kbps AAC+ -- is just a third as good as Spotify's free service -- at 160Kbps OGG Vorbis -- and just one sixth as good as Spotify's paid-for service, at 320Kbps OGG Vorbis. So the results, theoretically at least, should be cut and dried: Spotify sounds better.

There's more to sound quality than just a bit rate, of course (the type of codec used has enormous influence). We're simply testing product against product for perceived sound quality -- Spotify vs Sky Songs.

With our subjects tethered to a chair, we hit play. They weren't told which version was which, or even what to listen out for. They were simply told to choose version A or version B as sounding better.

The results

Of the 16 people tested, six people -- over a third -- thought Sky Songs ('version B') was the higher-quality audio. Conversely, ten people identified Spotify ('version A') as being the higher-quality track.

When asked, the majority of the participants choosing Spotify pointed out that instruments sounded "cleaner" at the higher bit rate. Some also noted that cymbals, hi hats and vocals in particular sounded better.

But what was unusual was the reasoning a couple of people gave for picking Sky Songs. They thought Sky's version produced better bass, and therefore was encoded at a higher bit rate. Both versions of the song were taken from the same CD, and therefore was not affected by better bass via remastering.

Things to bear in mind

Although conducted fairly, our test was still very subjective. Everybody hears 'quality' differently -- to some it's that bass sounds deeper or louder, to some it's that higher-frequency sounds are "crisper" or better defined.

Also, AAC+ and OGG Vorbis compression methods offer different advantages in terms of compression efficiency, largely because the psycho-acoustic algorithms they use are different.

But we were also testing on very high-end equipment. Many people will simply use iPod earbuds to listen to music, or bog-standard PC speakers. And through this lower-quality gear, the differences between Sky Songs and Spotify's free service could be harder to identify.

Conclusion

Whatever the reasoning, Spotify's bit rate is higher, and the majority of our participants correctly identified this. If you choose to pay for Spotify's premium service, the quality is higher still. We can only conclude, therefore, that if you're purely looking for quality of streaming audio, Spotify is still your best choice.

Update: We have added a paragraph to clarify that this is simply a casual, anecdotal comparison of two products, and not a definitive study of the benefits of AAC and OGG Vorbis compression formats. We are well aware that AAC, OGG Vorbis, MP3 or WMA files of identical bit rates will not sound the same. If this was a serious study of codec performance, we would have used 16,000 people, not 16.