Today's Safari flops on Apple's new browser speed tests

Apple's current Safari browser doesn't fare very well on a pair of new speed tests for Web browsers -- but the company's next version likely will make significant gains in the highly competitive market.

The tests, called JetStream and Speedometer, measure two important attributes of browser performance, respectively: the JavaScript programs and the Document Object Model (DOM) that lets browsers change elements of a Web page. Both are important for Web pages and Web apps that are interactive, not just static documents -- that is, the browser tests are a good indication of whether you'll enjoy or be frustrated by the time you spend on Facebook, Gmail, Flickr, and Cut the Rope.

"Scores on JetStream are a good indicator of the performance users would see in advanced Web applications like games," Apple programmer Filip Pizlo said in a blog post about JetStream. And his colleague Ryosuke Niwa said about Speedometer, "We decided to write a new benchmark for the end-to-end performance of a complete Web application."

Apple announced the tests in conjunction with its Worldwide Developer Conference (WWDC), where the company annually reveals new features programmers can use to write better software for Macs, iPhones, and iPads.

In my tests, Safari fares relatively poorly on the new benchmarks, lagging Google's Chrome, Mozilla's Firefox, and Opera Software's Opera. But there are signs that the next version of Safari, which Apple showed off at WWDC as it demonstrated OS X 10.10 Mavericks, will jump ahead.

That's an important step if Apple wants to improve its weak fourth-place ranking on PC browsing and keep its top spot in mobile browsing as Chrome rises through the ranks. Faster browsers make Web sites and Web apps more responsive for people using the browser, and that performance translates directly into more engaged users who read more online, perform more searches, and make more e-commerce purchases.

One high-ranking Apple developer took a jab at Chrome after the benchmarks emerged, indicating that the next Safari gets faster and that Apple is sensitive to Google's performance claims.

"Chrome loses to both Safari and Firefox on both JetStream and Speedometer. Surprising for 'A faster way to browse the web'(tm)," Apple's Maciej Stachowiak tweeted.

Although watching your browser run the benchmarks is about as fun as watching clothes spin in the washing machine, those tests are important: they show the priority areas where browser makers need to make improvements. Mozilla developed its Kraken benchmark and Google its Octane benchmark precisely so they could build a browser to do the right thing.

The Apple tests reveal an interesting priority: Mozilla's asm.js technology.

Apple likes asm.js

Google for years has been advocating a technology called Native Client that lets programmers run lightly modified versions of software written in C or C++ in the browser -- at nearly the full speed as it would run outside the browser as native software. That has some appeal for programmers like game developers who often reuse packages like the physics engines that control object behavior in those games.

Mozilla, though, doesn't care for Native Client -- indeed, no other browser maker has expressed any support. It prefers to put its effort behind improving JavaScript, the universal language of Web programming, and has come up with a Native Client alternative called asm.js.

With asm.js, the browser is optimized to run a subset of basic JavaScript very fast. Separate software -- Mozilla's Emscripten, for example -- lets programmers convert their C or C++ software into that subset. The approach has attracted some gaming companies' interest, and cross-platform programming toolmaker Unity 3D has dropped Native Client in favor of asm.js.

Apple is paying attention. Criticizing Google's Octane test as imbalanced, Pizlo said it "downplays novel JavaScript technologies like asm.js; only one of Octane's 15 benchmarks was an asm.js test." He also said Apple wanted to measure "new JavaScript-based technologies that have captured our imaginations."Apple evidently thinks the asm.js area is one such important area.

"Finding good asm.js benchmarks is difficult," Pizlo said. "So we built our own."

Software written with asm.js will run on a browser that hasn't been optimized for it -- it'll just run more slowly.

Speed test results

I ran Apple's JetStream and Speedometer tests on a 2012 MacBook Pro Retina laptop with OS X 10.9 Mavericks and a 2009 Dell Studio XPS 16 with Windows 8.1. I also ran Mozilla's Kraken and Google's Octane. In each case, the machine had been freshly rebooted, only the browser doing the test was running, I used the most recently released stable version of each browser, and plugins and extensions had been disabled.

It's not fair to compare the test results from the different machines, since the Apple is three years newer than the Dell, but the results for each machine are interesting.

Firefox generally came out ahead on both machines. Chrome outpaced it in Google's Octane test, but on all the others Firefox 29 was victorious except for losing by a hair to Opera on Mozilla's Kraken test on the Dell.

Internet Explorer 11 fared relatively poorly on my machine, arriving in last place except on the Speedometer test of DOM performance, where it beat Opera and Chrome. Opera and Chrome generally posted similar results, which isn't surprising given that Opera last year dropped its own Presto engine at the heart of its browser and adopted Google's Blink engine instead.

Why can't we all just get along?

Microsoft and Google didn't comment for this story. Mozilla, though, would like to see something beyond JetStream, Octane, and its own Kraken.

"JetStream is built almost entirely on top of previous benchmark suites," Engineering Director Vladimir Vukicevic said. "The focus on asm.js is interesting, but otherwise, JetStream adds little that is new, and thus suffers from the flaws of the suites it is built upon."

He characterized JetStream as a sort of Frankenstein's monster -- a hodgepodge of speed tests cobbled together from years of programming work across the industry. The browser makers can do better, Vukicevic suggested.

"All of these benchmark suites, as well as Mozilla's own Kraken suite, come from an era where each browser vendor would release their own benchmark suites," he said. "We look forward to discussing with other browser vendors how to create a genuinely new JavaScript benchmark suite that represents modern Web workloads."