Early Prime Day Deals Roe v. Wade Overturned Surface Laptop Go 2 Review 4th of July Sales M2 MacBook Pro Deals Healthy Meal Delivery Best TVs for Every Budget Noise-Canceling Earbuds Dip to $100

What Gatekeeper in OS X 10.8 means for Mac security

Apple's next iteration of its Macintosh operating system has a new security feature, but how will this help secure your Mac?

In recent years Apple's OS X platform has seen an increase in malware, which, while minimal in comparison to increases for other platforms, has proven that no platform is invulnerable to attacks.

With the malware scene making some inroads into OS X, Apple has taken some steps to protect Mac users from it, primarily with the introduction of the XProtect feature in Snow Leopard, which detects the presence of known Mac malware signatures in newly downloaded files.

In the current release of OS X (version 10.7 Lion), Apple did not advance its malware security effort much, and Mac users must rely on their ability to recognize threats or use malware scanners to identify malware. But in the upcoming OS X 10.8 Mountain Lion, Apple has included a new technology called Gatekeeper, which approaches the malware problem from a different angle.

The conventional method of managing malware is to install an active scanning program that will check downloaded files and periodically scan the system for threats. If found, the threats are quarantined or deleted. However, this process invites an endless cat-and-mouse game between the security software and malware developers.

With Gatekeeper Apple has taken a different approach that centralizes security around numerous security technologies that are already present in the OS.

In the past decade, Apple has been steadily developing separate technologies in OS X that together build a framework that helps to combat malware:

  1. Sandboxing
    OS X has gained ways for developers to sandbox applications and prevent those applications from accessing system resources that they don't need. This option has been voluntary for developers, but Apple is mandating that as of March 1 all programs distributed through the Mac App Store must have sandboxing enabled.

  2. Digital signatures
    Apple's Xcode enables developers with approved Developer IDs to digitally sign applications, which, like a checksum for an individual file, acts as a notification flag to the system if the program has been altered. This makes it so the system can preferentially trust signed applications over unsigned ones, and block apps without a valid signature. For instance, if you modify the latest version of iTunes to add colors back to the sidebar, you will break the signature of the program and the OS X firewall will no longer trust it and will ask you to manually allow or deny network connections to iTunes every time it is launched.

  3. Mac App Store
    In more recent developments, Apple has provided a one-stop shop for Mac applications with its Mac App Store, where it approves all applications that are available through the store. This approval process has been seen as restrictive by some developers, and many like Microsoft and Adobe have not yet jumped on board with their highly popular programs. Nevertheless, the App Store provides a gateway to approved programs.

With these three features, Apple can implement its solution to malware essentially by centralizing security under one authority: its own. When you turn on Gatekeeper in OS X 10.8, you basically tell the system to only allow applications to run that have been signed and approved by Apple.

Gatekeeper settings
Gatekeeper's settings have two security levels, and an option to disable the feature. Apple

There are two security settings for Gatekeeper: to allow only programs downloaded from the Mac App Store to run on the system, or to extend this trust to signed applications created by developers in Apple's developer program.

In a way this approach is very similar to Apple's KidSafe parental controls feature in its iTools services (Apple's predecessor to .Mac, MobileMe, and iCloud) back in 2000, in which the company maintained a library of approved Web sites that were allowed to be viewed on a system. Unfortunately the vastness and complexity of the Web crushed the feasibility of this venture, but Apple's tightly controlled developer program and App Store distribution service make this type of approach more practical when it comes to programs. Apple can take advantage of the Mac App Store's popularity to block out any harmful material, should you wish to subscribe to it.

Even with this new technology and Apple's efforts to get developers to join its developer program, there are many useful and legitimate programs for OS X that do not subscribe to Apple's developer program and are distributed on developer Web sites instead of through the Mac App Store. Luckily, in Mountain Lion you can allow these programs to run by right-clicking them and installing them despite your chosen level of Gatekeeper security, or by disabling Gatekeeper altogether; however, you will need to be a little more aware of the programs you install on your system.

A final point about Gatekeeper is it works by using the quarantine extended attribute that the system tags to files when they are downloaded. This means that it will only work on programs that are downloaded to the system, and will not protect you from Web-based threats such as Javascript-based trojans that execute within your Web browser. However, despite this the advancements that Gatekeeper offers do greatly set back the development of successful malware on OS X.

Now playing: Watch this: Apple's next iteration of Mac OS X: Mountain Lion

Questions? Comments? Have a fix? Post them below or e-mail us!
Be sure to check us out on Twitter and the CNET Mac forums.