Performance showdown: Windows 7 vs. Snow Leopard

As someone who uses both platforms for work and personal entertainment, I've been wanting to do a performance comparison between Windows 7 and Mac OS X since I first got my hands on the Windows 7 RTM (the final build of the OS) more than two months ago, but decided to wait until I could compare the two apples to apples. (No pun intended.)

The right time seems to be now, as Snow Leopard has been out for a while and has even been updated to 10.6.1, and Windows 7 has been at the OEM (original equipment manufacturer) vendors for almost three months and has also had a few updates. Furthermore, Boot Camp 3.0 seemed to make Windows run better than ever on a Mac.

Just to clarify, Boot Camp is not a virtual environment but simply a bundle of native Windows drivers--software that makes the OS work properly with hardware components. These drivers include chipset, video, networking, and so on. As a matter of fact, you can get most of these drivers from the components' manufacturers (or via Windows update). However, Boot Camp also contains drivers for Apple's proprietary hardware including the iSight Webcam, keyboard backlight, and multitouch mouse pad, and therefore it's best to get this bundle instead of looking for drivers individually.

For the sake of transparency (I know a lot of you feel passionately about one operating system or the other), I will disclose how I conducted my testing so you can duplicate it if you want. There's no rocket science involved here; all you need is a good stopwatch, a MacBook Pro, and a lot of time.

It's important, however, to note two things. First, the testing described in this article is somewhat anecdotal as it was performed on only one computer and, to some extent, was conducted differently from how we generally test computers for CNET reviews. (Read CNET's official reviews of Windows 7 and Snow Leopard.) Second, by talking about all this in such detail, I will seem much nerdier than I actually am. (Editors' note: This jury is still out on this one.)

First off, the test machine is a 15-inch unibody MacBook Pro with a 2.5GHz Intel Core 2 Duo, 4GB of RAM, and a 512MB Nvidia GeForce 9600M GT video card. This is the 2008 model of the computer that comes with a removable battery and doesn't have the SD card slot. (This is not the latest 2009 model that comes with a nonremovable battery, which packs a lot more juice.)

Mac OS X Snow Leopard is installed on the stock 320GB hard drive (a Hitachi model HTS543232L9SA0). Windows 7 64-bit is installed on a 320GB Western Digital Scorpio Blue (model WD3200BEVT). I chose Windows 7 64-bit as Apple claims Snow Leopard is now a pure 64-bit OS with most of its built-in applications being constructed with 64-bit code.

These two hard drives have virtually the same specs, supporting a SATA 3Gbps interface, having 8MB of cache memory, and spinning at 5,400rpm. I got a new hard drive so each operating system would have a hard drive of its own, with only one partition. The computer can be transformed from a Mac to a PC and the other way around just by swapping out the hard drives. Alternately, in real life, you can have both operating systems on one hard drive by running Boot Camp Assistant from within Snow Leopard to create a new partition for Windows.

(By the way, thanks to the laptop's nice design, it was very easy to swap out the hard drives with the help of a small Phillips-head screwdriver and a tiny torx wrench. The installation of Windows 7 64-bit was then done just like with any PC: I booted the computer with the installer DVD and followed the onscreen installation instructions. I was able to get the Windows OS up and running with Boot Camp 3.0 installed after less than an hour without any hiccups. Boot Camp 3.0 provided all the latest drivers for Windows 7 and no driver update was necessary.)

For each OS, during the tests, the computer had the following software installed: iTunes 9, QuickTime, Call of Duty 4 Modern Warfare, and Cinebench R10. All are 64-bit except Call of Duty and QuickTime. Windows 7 was tested with QuickTime 7 (version 7.6.4), which is 32-bit, and Snow Leopard was tested with QuickTime X, which comes with the OS. The reason is that QuickTime X is not currently available for Windows and you can't install QuickTime 7 on Snow Leopard.

Both computers were set up for high performance in power management. No other settings were changed. Windows 7's Visual Effects was left at "Let Windows choose what's best for my computer," which, in this case, equaled all items being checked except "Save taskbar thumbnail previews." Snow Leopard's graphic setting was set to "Hi-performance mode."

Except for Cinebench and Call of Duty 4, which have a scoring system of their own, all other tests are time-based. I personally performed all the time-based tests and the Cinebench test, while my colleague Joseph Kaminiski, who has tested hundreds of computers for CNET reviews, took care of benchmarking the operating systems using Call of Duty 4. Nonetheless, we cross-checked our results.

The project took a few days. Originally, I wanted to also do the same testing on an iMac, but as it turned out, Boot Camp 3.0 doesn't provide support for Windows 64-bit running on iMacs. We did each test multiple times and checked the consistency of the results to make sure they were not affected by any aberration, such as me pressing the stopwatch button too fast or too slowly.

Here come the scores

In time-based tests, Snow Leopard consistently outdid Windows 7. It took only 36.4 seconds to boot up, while Windows took 42.7 seconds. In a shutdown test, Snow Leopard took only 6.6 seconds, while Windows needed twice the amount of time: 12.6 seconds. Both computers, however, took just about 1 second to return from sleeping. For this reason, I didn't actually test the wake-up time as it was too short in both operating systems to produce meaningful numbers or even allow me to measure the difference.

In an iTunes conversion test, where I timed how long it took iTunes to convert 17 songs from the MP3 format to the AAC format, Snow Leopard took 149.9 seconds to get the job done. Windows needed 12 seconds more for the same job.

The last time-based test was the multimedia multitasking test, where I measured how long it took QuickTime to convert a movie file from the MP4 format into the iPod format, while having iTunes converting songs in the background. This is sort of an unfair test as I had to use QuickTime 7 for Windows 7 and QuickTime X (which Apple claims to have much improved performance over the previous version) for Snow Leopard. The result: Snow Leopard beat Windows big time, taking just 444.3 seconds to do the job while Windows 7 dragged with 723 seconds.

So Snow Leopard took the lead in booting up, shutting down, and running Apple's software. It was a different story, however, with other third-party benchmarking software.

Cinebench R10 showed that Windows 7 was noticeably better than Snow Leopard in 3D image rendering--with a score of 5,777 versus 5,437 for the OS X (higher is better). In gaming, Windows 7 also offered higher frame rates. In our Call of Duty 4 test, Windows 7 scored 26.3 frames per second (fps) while Snow Leopard got only 21.2fps. Joseph tested the game with a few different maps and we picked one that registered the highest scores for both operating systems to report. Consistently, Snow Leopard was always 5fps to 7fps slower than Windows 7.

The last test--which took the most time and probably will prove the most controversial--measured battery life. In a blog a while ago, I said that Windows 7 offered about the same battery life on the MacBook Pro as Snow Leopard. Well, I was wrong. While it was indeed better compared with what it was with Boot Camp 2.1, Windows 7 on the MacBook Pro still has a significantly shorter battery life than Snow Leopard.

As I needed to fully charge the battery before each test to make the tests go faster, I decided to test the battery life with the same settings as the performance tests, which drain the battery much more quickly than in normal usage. These settings include the computer's screen, as well as the keyboard illumination, being set at their brightest; the speakers being turned all the way up; and the Wi-Fi connection being turned on. After that, I made the computer play a high-def movie clip on loop and in full-screen mode until the computer died.

The results? Windows 7 lasted 78 minutes, while Snow Leopard managed to stay on for 111 minutes. These numbers are, of course, the worst case scenario--in real life, you'll get much longer battery life for each OS with regular usage. Personally, I could easily get about 3 hours with Windows 7 when running the MacBook Pro using the operating system's recommended "Balanced" power management scheme. Nonetheless, it's obvious that Windows 7's battery life is just about two-thirds of Snow Leopard's on the MacBook Pro.

By now, more than anything, I believe drivers are the culprit for this discrepancy, as with Boot Camp 2.1, I was able to get just around an hour and a half with Windows 7 with general usage on the same machine. I've also seen many PC laptops where Windows 7 also offers much longer battery life.

The conclusion? First all of all, you'll get much better battery life running OS X on Mac laptops than running Windows. Secondly, performance-wise, Windows 7 is probably a better choice if you are a gamer (there are more games developed for Windows, anyway), even on Mac hardware.

Third, if you can get by with just software designed by Apple and if money is not a big issue, you will be happy with a Mac. Examples of these software choices are iTunes, iLife, QuickTime, Safari, iChat, and so on (and you probably won't need much more than those for daily entertainment and communication needs). Finally, if money is not an issue--and it definitely is for most of us--you should get a Mac anyway. It's the only platform, for now, that can run both Windows and OS X.

Note that this article touched the two operating systems only from the performance point of view. (Mac is also really pretty and Windows offers a lot more options and compatibility.)

It's also worth keeping in mind that both operating systems were tested in their "clean" state (fresh and with a minimum number of apps installed) and using Mac hardware, which is naturally optimized for Snow Leopard. As you use them, the performance will change, most likely for worse because of software clutters gathered over time. It's hard to measure which one gets more affected by this than the other. However, when Apple allows installing OS X on PC hardware, I'll for sure run the same tests again.