Sun admits Java testing error
A top executive says the company erred in not admitting that it matched code from a Java benchmark tool for one of its own Java compilers.
The company erred in not admitting that it matched code from a Java benchmark tool for one of its own Java compilers, Sun Software president Janpieter Scheerder said today. A benchmark is a battery of tests that measures the speed and performance of software running in various configurations.
Kicking off the "Inside Sun Software Day," Scheerder began his remarks with a mea culpa for Sun's actions, revealed last week in a report by CNET's NEWS.COM. At that time, Pendragon Software, makers of the CaffeineMark Java benchmark test, accused Sun of taking code from the CaffeineMark software and adding it to a beta version of the Solaris 2.6 Java just-in-time compiler. CaffeineMark is one of several developers that have created Java benchmarks.
Last week, Brian Croll--director of product marketing for Solaris, Sun's flavor of the Unix operating system--denied that Sun lifted the code. Today, however, Scheerder made it clear that Sun had made a big mistake.
"Nobody was trying to do anything malicious," Scheerder said. "We just optimized [the Solaris Java compiler] too much."
A Sun public relations manager called the episode a "big-time organizational breakdown" in which an engineering prototype that was never meant to go public was posted on the Web with all attendant documentation, along with a press release that touted the software's performance. Sun has also posted an explanation on its Web site.
"Sun committed an unintentional error when we published Java performance numbers for an engineering prototype that included code that specifically looked for a piece of code in the Caffeinemark 3.0 benchmark," according to a company statement.
In a release dated October 20, Sun bragged that, according to the CaffeineMark 3.0 test, Solaris 2.6 ran Java applications 50 percent faster than Windows NT. But it neglected to say that it had set the compiler to look specifically for a chunk of code from CaffeineMark. Reusing such a large chunk of specific code risks diverting too much of the compiler's resources, resulting in lower performance once the compiler is deployed in the real world, said Ivan Phillips, president of Pendragon.
After taking issue with Sun's test results, Phillips said he asked Sun to retract its claims and remove the compiler from its Web site. As of last week, Sun had not retracted its claims, so Phillips went public with his accusations.
Scheerder stressed today that the compiler, which was part of the Solaris 2.6 Java Development Kit 1.1.4 beta, was not shipping product. The company pulled it from its Web site soon after Phillips contacted them last month.
The news comes four days before the International Organization for Standardization (ISO) decides if Sun is qualified to be the official submitter of Java technology if and when Java becomes an international standard.
The official submitter has the responsibility to gather industry consensus and present it to the ISO's technical committee for consideration. There is some concern that Sun, which owns Java, might not be a neutral submitter. So far, 11 countries have voted yes on Sun's bid and one country--the United States--has voted no. A total of 27 countries are scheduled to vote by Friday.