3Dmark for Android separates performance champs from chumps

With its latest version of 3DMark, Futuremark attempts to create a benchmark that allows for cross-platform comparisons among all tablets, smartphones, and PCs.

Futuremark's 3DMark benchmark uses an impressive graphics engine to stress your tablet's or smartphone's GPU to its limits. Screenshot by Eric Franklin/CNET

As a lifelong gamer, 3D performance (and by "3D" I mean polygonal, not stereoscopic) in tablets is something of particular interest to me. Over the last couple of years, I've watched mobile devices increase in performance, getting closer and closer to the capabilities of PCs and consoles. Judging by their performance histories, tablet and smartphone 3D performance is likely to exponentially increase over the next few years and we may soon be carrying around devices that are as powerful as an Xbox 360 (or more powerful) right in our pockets.

Futuremark's latest version of its long-running benchmark series, 3DMark is now available for Windows 7, Windows 8, and Android (version 3.1 minimum). The company is planning to release versions for iOS and Windows RT as well. The Android version was released to the press last week and I've had a little time to run it on a few of Android's fastest devices.

What is it?
3DMark's mobile platform test is called Ice Storm and depicts a sci-fi spaceship battle over an ice planet. The 3DMark score is derived from the results of three different tests: two graphics tests made to stress the system's GPU and one physics test focusing on CPU performance. The tests can be run at both 1,280x720-pixel resolution with normal textture and post-processing (partical effects, etc.) or at 1,920x1,080 with higher quality textures and post-processing effects. The images are then scaled to the device's native resolution.

Plenty of effects accompany hundreds of thousands of polygons. Screenshot by Eric Franklin/CNET

Graphics Tests 1 is an overall triangle throughput test that throws an average of 530,000 vertices into the average frame, equally about 180,000 triangles and 4.7 million pixels rendered per frame. Graphics Test 2 focuses on pixel processing, and includes many more post-processing effects like particles, blurring, and high dynamic range. The test renders 12.6 million pixels per frame, but only 75,000 vertices are displayed per frame.

During Physics Test 1, GPU load is kept to a minimum as the CPU is focused on making physics calculations. While the results from the graphics tests provide a good indication as to how well 3D games will perform, the physics test is better indicator of overall system performance.

Just 10 years ago, this level of graphics sophistication would have tricky to pull off on then current home consoles. Screenshot by Eric Franklin/CNET

The 3DMark score is not a unit of measure, but is instead derived using a weighted harmonic mean from both the graphics physics scores combined, with a higher score indicating faster performance and higher frame rates.

Device CPU GPU RAM Android OS tested
Samsung Galaxy Tab 8.9 1GHz Dual-core Nvidia Tegra 2 ULP GeForce (8-core) 1GB 4.0.4
Samsung Galaxy Note 10.1 1.4GHz Quad-core Exynos 4 Quad (4412) Mali T400MP4 2GB 4.1.2
Nexus 7 1.2GHz Quad-core Nvidia Tegra 3 ULP GeFOrce (12-core) 1GB 4.2.2
Asus Transformer Infinity TF700 1.6GHz Quad-core Nvidia Tegra 3 ULP GeFOrce (12-core) 1GB 4.1.1
Nexus 10 1.7GHz Dual-core Samsung Exynos 5 Dual (5512) Mali-T604 (quad-core) 2GB 4.2.2
LG Optimus G Pro 1.7GHz Quad-core Qualcomm Snapdragon 600 (Krait 300) Adreno 320 (single-core) 2GB 4.1.1
Alienware M-17x 1.73GHz Intel Core i7 Q740 ATI Mobility HD 5800 4GB Windows 7
Josh Long/CNET
Josh Long/CNET
Josh Long/CNET
The top mobile performers were really no surprise. The LG Optimus G Pro nudged out the Nexus 10 for the top spot. The Optimus houses a Snapdragon 600 with a 1.7GHz quad-core Krait 300 CPU and an Adreno 320 GPU. This beat out the Nexus 10's Exynos 5250/Mali T604 CPU/GPU combo by a few frames per second, but overall their graphics performance is comparable.

I also threw in a gaming laptop just to give you an idea of how far mobile GPUs have to go before they get anywhere near matching what's currently available on higher end platforms. Currently, there's no Extreme version of the test on PC.

As you can see from the devices' relative performance, Nvidia's Tegra 3 is getting a bit long in the tooth. Not a bad showing, but one that will continue to be overshadowed by newer processors from Qualcomm and Samsung, until Tegra 4 with its ridiculously bountiful 72-core GPU makes its debut.

Also of note (heh) is the Samsung Exynos 4 Quad (4412) in the Note 10.1. While the Note 10.1's Mali T400MP4 can't keep up with the higher end GPU's, the Exynos 4 Quad's CPU performance ranks as one of the fastest on the list. Its impressive performance indicates that for non-graphically intensive tasks (for example, quickly and accurately interpreting signals from the S Pen sylus), it's more than up to the challenge.

3Mark will allow for a 1-to-1 graphics comparisons between Android, iOS, and Windows RT devices in addition to PCs. Screenshot by Eric Franklin/CNET

While I'd love to start using 3DMark as an official benchmark for tablets and smartphones, more testing will first need to take place. Certain smartphones will throttle their processor speeds if things get too hot internally, lowering overall performance. And there's nothing like a 3D game to get a processor revved up. This unfortunately leads to inconsistent scores. There are ways around this (refrigeration and surrounding the device with frozen fruit packages have been suggested), but until I can figure out a method I'm comfortable with, I won't yet promise you'll see scores for every tablet or smartphone review.

If I do decide that 3DMark is a useful enough test for reviews, expect a much larger and comprehensive post in the future. One detailing the graphics and physics test results of every tablet and smartphone that both make sense to test and that I of course still have access to.

3DMark is now available for Android 3.1 (or above), Windows 7, and Windows 8. Look for a Window RT and iOS version soon.