W3C seeks help, patience with HTML5 tests
It's much too early to judge browsers based on 232 HTML5 tests, the Web standard group says. Legit conclusions will have to wait for tens of thousands of more tests.
- Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
The World Wide Web Consortium is griping about this week's premature judgments regarding which browser best complies with the HTML5 standard and is asking for help to improve its tests.
Philippe Le Hegaret, who oversees HTML5 and other standards at the W3C, today chastised those who concluded from tests done so far that Internet Explorer 9 is leading the race to support the new Web page technology. The W3C has added 135 new HTML5 compliance checks in the last month, bringing the total to 232, but that's nowhere near enough, he said.
"It seems that people are trying to draw conclusions from the tests or from the results, including whether one browser or another is better," Le Hegaret said in a blog post.
Apparently they shouldn't be doing that yet. "An increase of 135 tests isn't meaningful. It's way far from making the results significant in fact. We'll need several dozens of thousands of tests to make those results indicative," Le Hegaret said.
Le Hegaret's remarks appear aimed at media reports that jumped on the idea that IE was leading the standards race after Microsoft pointed to the "Official HTML5 Test Suite Conformance Results" in an IEBlog post last week. It's a juicy idea, given how many years IE spurned Web standards, but browser standards experts bristled at conclusions being drawn from the limited set of tests.
What Le Hegaret would rather see is more assistance in fleshing out the test suite.
"We need all the help we can get to make the test suite relevant and informative," Le Hegaret said. "Unless the community starts helping W3C, we won't be able to properly test HTML5."
Microsoft, eager to move to something more sophisticated than the Acid3 test that's been cited widely in recent years, has been helping with the testing effort. It's submitted 2,853 tests to the W3C so far, though many of those are for other Web standards besides HTML.
Microsoft's active participation has indeed forced Web standards insiders to update their views on the company. The changes wrought by the new era were on display this week when Microsoft evidently ran out of IE Acid T-shirts to hand out at the W3C's Technical Plenary Advisory Committee meeting (TPAC) in France.
"Never imagined when [I] took this job that people at #tpac would ask for #IE shirts. Times change," Sylvain Galineau, a Microsoft program manager dealing with IE Web standards, tweeted today.
Updated at 6:13 a.m. PDT to remove a Galineau tweet that wasn't in fact related to the W3C test situation.