Vista's PC-rating tool gets a revamp

In May, the software maker promised to make changes to the Windows System Performance Rating tool, which aims to assess how capable a machine is of harnessing the upcoming operating system's new features. Critics were unhappy with the way it presented scores and how it came up with its ratings.

The tool is designed to help consumers make sense of Vista's fairly complicated needs when it comes to memory, graphics performance and other internal components. It looks at five benchmarks and presents an individual score for each, as well as an overall rating for the system.

The newly renamed Windows Experience Index includes tweaks both in the built-in software and in the way that the assessment is described. Despite these, some partners still believe that the score generated by the tool is not a balanced reflection of a computer's abilities.

Officially, Intel said, "We continue to work closely with Microsoft to shape and influence (the rating tool), but we have no further comment at this time."

However, a source close to the chip giant said that it remains concerned that the tool places too much emphasis on the graphics and memory power needed to take full advantage of Vista's Aero user interface and advanced media features. "It's very heavily focused on graphics performance," the source said.

In contrast, Intel believes that the tool doesn't adequately account for important characteristics, such as whether a processor has more than one core, or the battery life offered by a notebook, the source said.

The source said the chipmaker applauds the notion of offering consumers a simpler system to understand PC performance, but argues that Microsoft's software does them a disservice by not reaching that goal.

"If this thing is promoted to consumers, they should understand what its strengths and weaknesses are," the source said.

Microsoft has made a number of changes to the software, though many of the tweaks won't be publicly visible until it releases the first near-final "release candidate" version of the operating system later this quarter. The oft-delayed Vista is scheduled to be in consumers' hands by January.

For starters, Microsoft redubbed the tool the Windows Experience Index, arguing the moniker better represents what it measures.

It has also given the new name "base score" to the overall rating generated by the tool. It's an effort to clarify that the main rating is the lowest score given to an individual component, rather than an average of each of the five subratings. The tool rates a system on its processor, memory, hard drive, graphics card and gaming graphics.

Chipmaker Advanced Micro Devices, which says it is generally supportive of Microsoft's effort, said that Microsoft's changes should make the tool better. The new name makes it clearer that it is not a raw measure of PC performance, but rather a Windows-specific assessment, AMD program manager Clarice Simmons said. She also praised Microsoft's move to rename the score generated by the tool.

"I know that several partners gave them some feedback that the overall rating was a little bit difficult to understand," Simmons said. She added that there is a natural tendency to assume the overall score is the average of the various components.

In another change, Microsoft is allowing the base score to be more varied. The earlier version offered decimal point ratings, such as "4.1", for assessments of the individual features, but relied on whole numbers for the overall score. The revamped index now uses decimals for the base score as well.

Good for graphics
The changes, though important to the partners that largely support the rating tool, don't appear to address the concerns of Intel and others.

Graphics chipmakers, meanwhile, are understandably pleased with the prominent attention given to their products, noting that Windows Vista relies heavily on graphics chip horsepower to generate its Aero user interface.

"It should be very clear to everyone how important graphics are," said Andrew Dodd, a software product manager at graphics specialist ATI Technologies.

"Windows Experience Index is designed to help average consumers easily understand their Windows Vista PCs overall performance, and to simplify the process of determining whether certain software applications will run smoothly based on their system components," it said.

It added that computer makers and retailers can use Windows Experience Index to "help customers understand the general performance capabilities of a Windows Vista PC and the types of experiences it can support, relative to other Windows Vista PCs they offer, thereby enabling easy-to-understand differentiation."

Software makers will also benefit, as they can use the scores as to communicate the minimum and recommended system performance for their programs, Microsoft said.

Scores on the doors
The initial version of the tool will offer a 1-to-5 ranking, but additional numbers will be added over time, Microsoft said. That means a PC that gets a "3" today will continue to get a "3", but eventually there will be computers that rank a "6" or a "7".

"We will introduce new scores (6.0-6.9, 7.0-7.9, and so forth) periodically as new hardware component performance capabilities increase," Microsoft said in a statement. "The existing numbers will simply remain defined 'as-is' (i.e. the performance metrics for a 3.2 will always be the same regardless of how many new numbers are added). We created this scoring system with upward scalability and hardware technology advances in mind."

The company also offered some assessment of what the ratings mean, though it did not provide a complete breakdown of each score and what it signifies.

"Using this base score, a customer can then determine if their machine will support their desired experiences (i.e., if a person uses their Windows Vista PC for office productivity and Web browsing, a base score of 1-2 would suit their purposes; or if another person uses their Windows Vista PC for HDTV recording and 3D gaming, a base score of 4-5 would suit their purposes)," Microsoft said.

Dodd said ATI's main concern is that the score will represent an accurate reflection of what the graphics chipmaker's components can do in real-world use.

"I know they are still making changes to what determines whether you get a '4' or a '5'," he said. "As long as Microsoft makes it clear what each rating means and why they are getting that rating, it is a very good thing for end-users."

AMD's Simmons said that even if the tool underestimates the importance of a PC's processor, it is still a benefit if it leads to fewer returned computers and a boost in customer satisfaction. Although it will ultimately be up to PC makers and retailers how to use the scores generated by the Windows Experience Index, Simmons said she envisions the ratings appearing on the kinds of fact sheets that accompany a PC on store shelves.

"Even if it is not necessarily good for the CPU itself, it is good for the industry," Simmons said.