Counting blurry lines: Should CNET test for motion resolution on HDTVs?

In the course of testing for HDTV reviews here at CNET, I always compare displays directly against one another side-by-side using both normal program material--typically Blu-ray movies, HDTV, and standard-definition material--and test patterns from special discs. I'm always on the lookout for new test patterns, so earlier this summer when I spoke with another TV reviewer, Gary Merson of hdguru.com, about his tests for motion resolution, he was kind enough to pass along a Blu-ray Disc called "FPD Benchmark Software for Professional." It contains a variety of test patterns, most of which I've seen and used before, with one notable exception. A suite of patterns and program material is devoted to testing and demonstrating motion resolution, and I'm considering incorporating it into CNET's regular HDTV tests.

Evaluating an HDTV's resolution with still patterns is easy on fixed-pixel displays like flat-panel plasmas and LCDs and rear-projection microdislays. Just put up a resolution pattern, which consists of alternating black and white lines that correspond to the display's native resolution, which is 1920x1080-pixels for a 1080p HDTV. If every line is clearly resolved, the display passes the full resolution of the pattern. With motion, however, the evaluation gets a lot more tricky and subjective.

The main pattern used to test motion resolution from the FPD Benchmark disc is called the monoscope, which scrolls across the screen at a set, medium-speed rate. A series of vertical wedges consisting of four parallel lines, each with numbers corresponding to a vertical resolution between 100 and 1,200, is set in the middle of the pattern. While the pattern scrolls across the screen the lines blend together and blur, to a lesser or greater extent depending on the display in question. To arrive at a number that corresponds to the display's vertical resolution, the viewer (er, reviewer) must judge when the lines become blurry and blend into one another. The number closest to the place on the wedge just before the blending occurs is the motion resolution.

Unfortunately, that judgment can vary. I rounded up a couple of CNET staffers, Matthew Moskovciak (a reviewer highly familiar with test patterns) and Matthew Panton (our summer Intern), explained the criteria for the test, and asked them to write down the results they got from looking at seven HDTVs' motion resolution, without telling me or one another. The calibrated TVs were placed side-by-side and set in the picture modes that gave the best motion resolution results. The table below summarizes our findings:

As you can see, we agreed completely on only two of the televisions, and while we were within 100 lines of one another in most other cases, that variance is still significant. Aside from the variance, I have a few other reservations about including motion resolution among the regular tests I perform for HDTV reviews.

First off and most-important, to my eye and those of Matthews at least, differences in motion resolution are very difficult to discern with actual program material. For one example, we watched a high-definition football game with the best display in the test (the Pioneer PDP-5020FD plasma) placed right next to the worst (the Samsung LN46A550--the only LCD in the test without a 120Hz mode) and we found it hard to detect more blurring in the Samsung LCD, even when paying close attention to potentially blurrier areas, such as the hash marks when the camera follows a kickoff downfield or the churning legs beneath the return man. For two displays with such disparate results according to the test pattern, I think most people would expect to see more of a difference. "Difference" is the key word here; we did perceive a good deal of blurring in many fast moving shots on both displays, but either that blur was inherent in the source (usually a consequence of quick camera movement) or was a consequence of the brain's inability to process fast-moving details. In any case, most of the blur we saw couldn't be faulted on the displays. This issue raises one of the most-important and tricky question TV reviewer's face: If a difference revealed by a test pattern doesn't translate to any significant difference that experienced reviewers can detect with real program material, how worthwhile is it?

Second, to get the full blur-busting capability out of the 120Hz LCDs, namely the two Sonys, the Vizio, and the Samsung A650, I had to engage their "smooth" antijudder processing. With smooth modes disabled on these sets, they scored as low as the 60Hz Samsung A550. The problem is that smooth processing, to my eye at least, is inappropriate for film-based sources like DVD and Blu-ray movies; it makes them look way too much like video. Ideally, one could disable the smooth modes and still get the benefit of the higher motion resolution (engaging Smooth gained 100-200 lines of motion resolution).

Third, the test disc itself was produced by a Japanese consortium of plasma manufacturers, so it can hardly be called impartial. I try to use test material that's technology-agnostic in my reviews, although in this case (and that of the HQV Benchmark discs I use to test video processing) I think the test material can reveal important differences in displays, regardless of who produced it or what agenda they may have.

Compared with other the Geek Box tests, which are (by design) objective, any inclusion of motion resolution testing would have to be taken with a grain or ten of salt. But I hear lots of readers complaining about motion blur on their LCDs, or citing blurring as one reason why they chose plasma over LCD, which leads me to believe some sort of motion resolution test, no matter how flawed it may be, would be welcome. What do you think? Would you like to see motion resolution tests included in the Geek Box or at least mentioned in CNET reviews, regardless of the issues mentioned above? Let me know right here, and I'll take your comments into consideration. And if you have any other tests you'd like to suggest, feel free.