Can sound quality be measured?

I've met a lot of audio designers in my time, and all of the best ones have one thing in common, they have great "ears." They know what good sound sounds like. The opposite camp is populated with engineers that rely exclusively on measurements to "prove" their designs are better. To my way of thinking, the second group rarely makes great sounding products. Audio is too complex to be analyzed with just numbers alone.

Nowadays I'm meeting more digital audio engineers specializing in designing room and speaker correction software. They are usually very nice people, and their graphs and tests always look impressive on their laptops, but the presentations fall apart when I listen to their sound. Results vary from not bad to truly horrendous, but great sound is the least likely end product of their hard work.

Apparently, they were so focused on measuring sound they forgot to listen, or hire someone who actually knows what good sound sounds like. If the goal was to achieve better measurements I'd congratulate them for their accomplishments. But it's not, and discovering exactly what types of measurements indicate improved sound quality is an art. An art few of these engineers have mastered.

There are a couple of reasons why measurements fail to correlate with subjective sound quality assessments. First, electronics and speaker measurements have little to do with the sound of music. Test tones are too simple and predictable; music is far more complex and random. Reproducing the sound of a violin or a drum kit are exceedingly difficult tasks, and since the real goal of any hi-fi is to play music and not test tones, the designer's first priority should be making products that sound "good" for the intended market. For example, if you're designing DJ headphones, you aren't trying to deliver the most accurate bass. Far from it, you want to pump up the bass. Amplifier designers shouldn't waste their time trying to create an amp to drive simple test loads, they need to make an amp that handles the complexities associated with real speakers playing music. And amplifier designers don't know which speakers are going to be used with their amps. Every speaker presents a different type of "load" to the amp.

In the 1970s there was a big push to lower the "total harmonic distortion" specifications of amplifiers to ever lower levels, based on the belief that if they measured better they would sound better. They were measuring the wrong things. The narrow focus of designing for measurements rarely produces bona-fide sonic improvements. That's not to say that in the hands of truly talented engineers measurements aren't useful, they absolutely are, but measurements are no substitute for listening.

I've heard all of the latest auto setup and calibration systems featured in Denon, Onkyo, Pioneer, Sony, and Yamaha receivers, and the results are hit or miss. Granted, these systems can improve the sound, but more often than not, the processing merely changes the sound. In those cases, I can achieve better results by listening and making adjustments with the manual speaker setup. The processed sound might measure better, but again, that doesn't necessarily correlate to sound quality enhancement.

I recently discussed the measurement quandary with my friend, writer Brent Butterworth, who believes measurements are useful tools, but we never came to a meeting of minds on this matter. I'm paraphrasing here, but he said that measurements that reveal flaws in the sound of a speaker might go unnoticed by the ear, and that some speakers that don't measure well, can still sound subjectively good. So there you have it.

So if trained engineers struggle to derive useful information from measurements, I can't imagine how consumers looking at a wiggly line on a chart helps them decide which amp or speaker to buy. I'm not referring to specifications or numbers like watts per channel or driver sizes; I'm talking about charts and graphs that plot "spectrum of 1 kHz sinewave, D.C. to 1 kHz," or "anechoic response of tweeter on axis." If you have ever picked up useful information from peering at charts in audio reviews, and you're not an engineer, please share your insights in the Comments section.