With its new Safari 8 browser built into its, Apple delivered on its promise for a significant boost in browser performance.
On two new speed tests Apple introduced this year called JetStream and Speedometer, Safari vaulted over Google's Chrome, Mozilla's Firefox, and Opera Software's Opera. On a third test, called JSBench -- the one Apple chose to highlight last week as it announced the release of Yosemite -- Safari far outpaced those rivals. But on two other tests, Google's Octane and Mozilla's Kraken, Safari lagged.
Mixed results on benchmarks are nothing new: it's impossible to create a synthetic test that predicts how well a system will do with the vast variety of real-world computing challenges.
But browser makers have picked these tests to guide their development, and high scores generally mean browsers will be faster when you're posting to Facebook, searching at Bing, buying at Amazon or typing at Google Docs.
Faster browsers are important for another reason, too: They let Web developers make more sophisticated sites and apps, giving new power to tools like Google Docs and games like Cut the Rope. So it's no surprise browser makers have been locked in a speed race for years.
I performed my tests with the latest stable versions of the browsers on a 2012 MacBook Pro Retina-display model. For another perspective, I also tested on a 2009 Dell Studio XPS running Windows 8.1, where I could throw Microsoft's Internet Explorer into the mix but where Safari isn't available. The tests were running with extra extensions disabled, no other software running, no other browser tabs open, and with a fresh restart of the browser beforehand.
To show relative progress, I also compared the newest browser versions to the stable versions available in June 2014 for all the tests except JSBench.
New benchmarks for new browsers
Three of these benchmarks -- Google's Octane, Mozilla's Kraken, and Apple's JetStream -- are conceptually similar. They run a mixture of tasks such as encryption algorithms, GameBoy emulation, PDF rendering, and data sorting.
Not coincidentally, each browser maker's browser scores highest on its own benchmark test. You don't need to be cynical about this result, though. Browser makers steer their browser development with the tests, paying close attention when changes give one browser or another a competitive advantage. For one example of the process in use, check Mozilla's Are We Fast Yet site.
"Safari substantially lags behind Firefox and Chrome on existing benchmarks such as Octane, Kraken and Sunspider. Firefox is in fact the fastest browser on these benchmarks," Chief Technology Officer Andreas Gal said. Apple's browser engineers created Sunspider years ago, but have since moved away because of shortcomings.
Mozilla will add new Apple tests into the mix if they prove their merit, Gal added.
"Apple appears to have picked a new benchmark that other engines don't yet optimize for," Gal said. "We are constantly evaluating what benchmark best represents Web workloads, and if benchmarks like JetStream or Speedometer become relevant we expect other browsers to improve on them."
Indeed, just to pick two examples, I found Google Docs very snappy in the new Safari. But Safari 8 fared much worse at Atari's Web-based "Centipede" game than Chrome or Firefox.
The results are in
On the JetStream test, Safari 7 was behind Firefox, Chrome and Opera in June. But now Safari 8 handily outdoes the updated versions of those browsers. Safari 8's score of 162 was better than the 145 for Firefox 33, the 133 for Chrome 38, and the 132 for Opera 25.
Because browsers work on multiple operating systems, they can be run on multiple machines. My Windows computer, with an older processor and other components, doesn't fare as well as my Mac, but it's still useful for looking at the relative performance of the browsers on the machine. There, Firefox 33 was in first place with a score of 106, followed by Opera 25 at 101, Chrome 38 at 97 and Internet Explorer 11 at 80.
Octane results were broadly similar, but not as dramatic. There, Safari 8's 35 percent improvement over Safari 7 wasn't enough to give it a better score over No. 1 Opera, No. 2 Chrome or No. 3 Firefox. On Windows, IE again was in last place.
In recent months, Chrome and Opera made significant gains on the Speedometer test, and Firefox improved a little. Safari had been in second place in June but passed Firefox for the lead in October. On Windows, leader Chrome and second-place Opera edged ahead, while third-place Firefox and last-place Internet Explorer posted lower scores.
JSBench gave a very different view of the competitive landscape, trouncing rivals. On this test, where a low number is better, Safari 8's score of 16.9 was vastly better than Chrome at 98.8, Firefox at 99.1, or Opera at 106.8. On Windows, Internet Explorer was the top scorer, though not by a wide margin.
Victory on on browser benchmarks can be fleeting, but it's a top priority in the tech world.
And for Apple, Safari is important not only for Macs but also for iOS -- in particular because many programmers build their apps using Apple's Web-rendering technology behind the scenes.
So, although Apple made major browser performance gains in October, don't expect them to be the final word.