Adventures in music analysis
Founded by two MIT Media Lab alums, The Echo Nest is creating tools for software developers that could open up a whole new world of interesting music applications.
Founded to two MIT Media Lab alums, The Echo Nest is focused on what it calls "music intelligence." The company is developing software technology that can analyze the sounds within music files, text within online articles and blog postings about music, and other online data (such as songs being downloaded in a particular week). It will then license this technology to developers--commercial and non-commercial--to help them create a whole new class of music software and Web applications.
It's possible to imagine hundreds of possibilities. A music company could build an application to identify current trends in order to allocate marketing dollars to tracks that are most likely to succeed. A search engine might use the technology to tweak results based on what users are most likely to be interested in. Somebody might create a music analysis program that could help record companies identify potential hits from unsigned musicians, creating competition for pioneers HSS. (HSS's biggest claim to fame: its software apparently predicted that Norah Jones' first album would be a hit.)
It's a big and fascinating vision, but so far the company has released only one developer tool--the Analyze API, which takes an MP3 file and analyzes it globally for features such as tempo, dominant keys, time signatures, and so on. The API also breaks the file into discrete musical sections and analyzes each one for pitch, timbre (or tone), and loudness. (It works as a Web-based service today--developers upload the MP3 and get an XML file with the song data. The documentation explains what's in that XML file and how to use it.)
But my personal favorite is the visualization app developed by MIT graduate student Anita Lillie--it's like on your computer screen. Essentially, she created 12 different bands, one for each possible note, then graphed the song within those bands, using colors for timbre and brightness for loudness. Her Web site has screenshots and a video showing how various songs "look," including Beethoven's "Fur Elise," a couple of Daft Punk tracks, and Nirvana's "Come As You Are," which features shapes that look a lot like smiley faces. (The video's also available as a downloadable .mov file).