How we test: Washing machines
Reviewing a washing machine isn't quite as simple as running some dirty laundry through a cycle -- here's the how and why of our test methodology.
Production crews make laundry seem so magical in commercials. In these ads, disgruntled parents lament over the stubborn grass and grape juice stains that their current washing machine just can't handle and suddenly, voila! -- a new, better model appears that somehow manages to wipe away any and all evidence of past wear.
While I've fallen victim to the hypnotic allure of commercials like these (on more than one occasion), they don't exactly reflect reality.
Our job is to determine how well each washer actually cleans. But we can't just toss dirty clothes into a machine, run a quick cycle and eyeball the results. Things like ambient temperature and humidity as well as water pressure and even the size and type of each load can have a major impact on the overall score. We also need something a little more reliable than my eyesight to judge performance.
So, here's a rundown of how the CNET Appliances team is tackling washing machine testing. It isn't the sexiest topic ever, but we do want to be transparent about this process and that includes our overall commitment to consistent, comparable quantitative results.
The lay of the land
We recently transformed a 15-by-13-foot section (that's 195 square feet) of our appliance test facility in Louisville, Kentucky, into a climate control chamber for reviewing washing machines. (We also built separate climate rooms for refrigerators, dishwashers and dryers for a grand total of four test chambers. See my coworker Steve Conaway's progress-tracking GIF below to get a sense of the scale of this project.)
All of these rooms control for temperature and have humidity monitors -- the washer room where we store both the machines and the testing material is set to a balmy, consistent 72 degrees.
The room has two review stations so we can test a couple of washing machines simultaneously. Both stations are equipped with a hot and a cold flow meter to tell you how much water the machine is using at a glance, as well as a water temperature sensor, a pressure sensor, a water hardness monitor and a voltage reader. All of that is there to ensure that we review washing machines in a consistent environment from day-to-day and from unit-to-unit.
The washing machine test stations feel vaguely submarine-y.
There are also a lot of things happening outside the climate room to make all of this work properly, including a mega-capacity hot water heater, a water chiller and various pressure regulators.
All of this is designed with the express purpose of standardizing the test procedure so that each washer gets an equally fair shake. But that's just the start of all of the tech magic we have happening behind the scenes.
Welcome to AHAM world
The Association of Home Appliance Manufacturers, or AHAM, is a Washington, D.C.-based trade group that creates test guidelines for a variety of consumer products, including washing machines.
The preface of its "Performance Evaluation Procedures for Household Clothes Washers" document, for example, states that, "This standard [the AHAM standard] contains test procedures which provide a practical means of measuring specific performance characteristics of any brand or model of automatic household electric clothes washers."
AHAM is especially well known among appliance manufacturers who sell products in the US, because its standard is accredited by the American National Standards Institute, or ANSI -- a not-for-profit organization that oversees the creation of country-wide standards across many different categories.
All of that climate control and test station business I mentioned in the previous section? Yeah, that's all due to AHAM, and we largely follow the test conditions prescribed by their standard.
AHAM also provides detailed information on the type of materials you need to use per load, as well as the size of each load and even the amount of AHAM-formulated powder laundry detergent you should use.
Many manufacturers use AHAM tests during the product development stage to ensure that new models meet certain criteria.
While there are a lot of potential tests available through AHAM, we decided to focus on two specific criteria -- how well a washing machine removes stains and how rough (or gentle) it is on fabric. Of course, we'll also factor general design, features and value into each review, but the abridged AHAM standard we're following ensures consistent performance results -- and that's a critical component of washing machine testing.
How-to
With all of this climate control and test station tech in place, we can move on to the specific procedures involved in testing a washing machine, AHAM-style.
Some manufacturers vary the size of each load of laundry per washer -- or even per washer cycle. Others simply test using 8 pounds of laundry, which AHAM says is the average size of a load of laundry in the United States. We also test using this 8-pound standard, but will consider adding additional tests for compact or large-capacity machines, as needed.
An 8-pound load of AHAM-approved laundry consists of assorted "filler materials" -- hand towels, pillowcases and fitted sheets. Some of these materials are "new," or have never been washed before; the rest have been preaged either 21, 42 or 63 cycles. This ensures a fair balance of new-to-used items, just like a typical at-home laundry cycle.
While these filler materials help round out each laundry cycle so it more closely mimics washing at home, they don't directly contribute to a washer's overall performance score.
Instead, AHAM suggests stain strips soiled with sebum (the oil on your skin -- kind of gross, I know), carbon black/mineral oil, blood (pig's blood), cocoa (chocolate and milk) and red wine (aged). There's also a sixth section that's unstained and acts as a control.
As far as AHAM's rationale behind those specific stains, a representative told me that they are among "the more challenging types of soils that you would find in a laundry load. They also are repeatable and reproducible from lab to lab in order to assure an apple to apples comparison between machines and manufacturers."
AHAM stipulates that four stain strips be used for every 8-pound load of laundry and also that one length of the strip be sewed onto any of the hand towels from the filler materials.
How to load a front-load washing machine using the AHAM standard.
To judge general wear and tear, AHAM suggests mechanical action strips. These odd-looking square sheets of fabric have five holes arranged exactly like a 5-dice. Like the stain strips, we add four mechanical action strips to the standard 8-pound filler load.
But, we can't just toss the filler load, the four stain strips and the four mechanical action strips into the machine. There's a very specific loading process and it differs between front- and top-load machines. Check out a screenshot of the front-load washer procedure to the right (borrowed straight from the pages of AHAM). It may look bizarre, but it's really just another way to standardize procedures between machines.
After that, we're ready to actually wash the laundry. Specific cycle selection is completely up to the tester, but we decided to stick to the default settings for now. For the most part, that will mean a normal cycle, with normal soil, hot water and a high spin. We also use 59 grams of AHAM detergent per cycle -- that's the exact amount AHAM suggests for an 8-pound load.
We also run three identical cycles per machine to rule out any anomalies. Both the stain strips and the mechanical action strips are standardized by lot numbers to minimize variation between test runs. Still, atypical results can occur.
In that case, we will refer back to a "reference" machine. AHAM suggests a very pricey Electrolux machine called the "Wascator" because it's known for consistency. We got a commercial Huebsch washer and dryer pair instead, but they function the same way -- they act as constants to compare against the results from a test machine. So, if we get an unexpected result during testing, we can run another cycle with the same type of load in the reference machine to determine if there's an actual performance problem or if there's an issue with the filler load/stain strips/mechanical action strips/cycle/detergent we're using. (Maybe we accidentally used 53 grams of detergent during one test instead of 59, for instance.)
We also run two "normalize" wash cycles in the reference machine for every three wash cycles in a test machine -- and use the reference dryer to dry the load between test cycles. Normalize cycles help rinse off residual detergent from past cycles, which can artificially inflate test scores.
A stain strip before and after a cleaning cycle.
Scoring
We use a gadget called a reflectance colorimeter to test how well each machine removes stains. It measures the percent of reflectance so we can compare the difference before and after each cleaning cycle -- and also compare its stain-removal result to other machines.
Each of the five stains gets four readings -- one for each "quadrant" and the info is recorded. The higher the number, the better it removed the stains.
To determine general wear and tear on each mechanical action strip, we use a ruler to measure and then count all of the attached strings that are 2 millimeters or longer after a cycle. The higher the tally, the tougher the machine was on your clothes.
This is a delicate balance for manufacturers because a higher wear and tear score on the mechanical action strips generally translates to a better stain removal score on the stain strips.
Measuring strings on the mechanical action strip.
After compiling all of the scores, we carefully look over all of the performance data and begin weighing it against other important review criteria -- from a washer's design and ease-of-use to its features and overall value. And now that we've officially kicked off our washing machine testing, you can expect to see a lot more laundry reviews from us in short order.
In the market for a dryer, too? Check this space.