If you're looking closely at the Blu-ray player reviews on CNET, there's a frustrating trend that complicates buying decisions--Samsung Blu-ray players get solid editorial reviews from CNET, but user opinions are consistently poor. While there are always differences between CNET reviews and user opinions, the differences with Samsung Blu-ray players stand out as being consistent and large.
Ratings have been standardized to a 100-point scale to make comparisons easier.
User opinions from Amazon and Newegg are also included to give some perspective. Their ratings are consistently more positive than CNET user opinions, although we're not sure why.
There are some factors inherent in CNET's review process that can explain why editorial opinion and user opinions are different, and they're worth pointing out.
CNET relies on review samples from manufacturers. CNET gets review samples directly from manufacturers rather than buying them off the shelf like a regular buyer. It's completely possible that manufacturers handpick review samples for us; if there's a problem with a certain "batch," a company can make sure we get the good batch. As much as CNET editors might like to purchase all our review samples, we don't have the budget for it.
CNET's review periods are relatively short. To review as many products as we do, we obviously can't test every product as long as a standard buyer would. If a product has quality control issues that cause it to break down after a couple of months, that's not something we're going to catch. That's why user opinions are so important.
Scanning through the user opinions for Samsung Blu-ray players, this is a possible culprit. There are many reports of units completely failing, either right away or after a few weeks.
Dissatisfied users are more likely to post a review. When you really get burned by a product, you're going to be motivated to go through the effort of posting a user review. If everything just works, you're probably too busy enjoying it to bother posting a review. However, this is equally true for other products, so it doesn't do a good job of explaining why Samsung does particularly poorly with CNET users.
CNET editors' tastes are significantly different than our readers. Samsung's Blu-ray players tend to include a lot of cutting-edge features, but are a little less user-friendly than some competing models. (The PC streaming feature on recent models is very difficult to get working.) We'd say Samsung models appeal more to tech-savvy customers, and if our readers skew more mainstream, that could explain some of the difference in ratings.
Products that use home networks are more likely to have problems. In addition to the aforementioned PC streaming feature, any product that utilizes a home network is more likely to elicit negative user opinions. Even in 2009, home networking can be a pain and sometimes products are the scapegoats for other issues, like a faulty router or slow streaming speeds from an ISP. However, it's worth pointing out that LG BD390 has built-in Wi-Fi has significantly better user opinions, so it's unlikely that this totally explains the difference in the ratings.
User opinions are an unverified, open forum. For all the reasons listed above, user opinions are an important part of CNET and offer our readers more information to make their buying decisions. That being said, CNET and other sites like Amazon and Newegg don't have any way of verifying user opinions, including the writers' level of tech experience or whether they even own the product. There's nothing even stopping a less scrupulous manufacturer from posting fake bad reviews. I'm not aware of any evidence of this, but readers should always be aware of those caveats when reading user opinions.
In addition to the ratings discrepancy, Samsung has had a sketchy history when it comes to firmware updates. Recently, Engadget HD reported (relying on a thread in CNET's forums) that Samsung has twice issued problematic firmware updates for the Samsung HT-BD1250 home theater system with a built-in Blu-ray player, making the units unresponsive. Back in January 2008, we reported on the slew of expected firmware fixes for Samsung Blu-ray players, one of which came late, and some would argue that they all should have been issued earlier.
Ratings discrepancies and firmware problems on a single product wouldn't raise a red flag, but taken together it's a troubling picture. Editorially, we're stuck in a tough position. We stand behind the reviews we published; going back and changing those reviews or ratings wouldn't reflect what we observed during testing.
On the other hand, consistently low user opinions and firmware issues certainly make us reluctant to wholeheartedly recommend Samsung's Blu-ray players, and readers should be informed about what appears to be a pattern of inconsistent performance.
For now, our solution is to leave the reviews and ratings untouched, but add an editors' note linking back to this blog so readers are aware of some of the issues.
We contacted Samsung about our findings and the company issued a response, which is included in full below.
SAMSUNG RESPONSE
Samsung is a leader in the Blu-ray category, and has been since its inception. We continually work to innovate and push the envelope with our products and will continue to make a concerted effort to make the user experience more seamless and hassle-free.
This past year was the first time that our products, or any competitive products, crossed over from the traditional CE space into having Internet connectivity. We have been compiling feedback from the call center and have been improving the experience since the first products shipped, but are making specific improvements for 2010 to make the setup process easier and more automated, requiring less manual input from the consumer.
Along these lines, we were the first to deliver a PC Streaming feature for consumers that streams files from a PC to a Blu-ray player without installing new software that other systems may require for full functionality. Samsung's system is based on Windows XP's file sharing protocols, making it familiar and fully functional for the majority of PC users. Setting up the system is no different than setting up file sharing from one XP-based PC to another.
Security parameters (either set by software installed on the PC or by the network's security settings or both) that do not allow for automatic detection of devices for file sharing can create additional complexity, however. We recognize that it was inconvenient to manually input the PC's network address due to these security parameters. At the end of June, we issued a firmware update that saves the PC information to the device; thus eliminating this inconvenience. Also, our BD player supports XP-based security level and Vista's security level is higher than XP's. So Vista doesn't respond to our BD player's request for auto searching, but we do support manual search for Vista. If the user intentionally sets a lower security level in Vista, it makes auto search possible.
Customer service is another specific area we continually work to improve. As soon as we heard of some cases where the firmware upgrade process may be causing a problem outside of the 90-day warranty period, we decided to waive charges on players that required service or resetting as a result of this process. We would note that the firmware updates themselves are safe, and are not an issue. However, if the player is turned off, or loses the Internet connection during the process (ie drops a wi-fi connection), it can cause problems. We have taken several steps to make this process more reliable in current products, and will do even more in future products. If CNET continues to hear of any such issues, please let us know immediately so we can remedy the situation.
From real-time feedback from the Call Centers to reader feedback on sites like CNET, we are always looking at solutions that will make connectivity to a variety of content a much easier experience for the consumer as well as developing technological advancements that push the capabilities of our players.