How to reinstall OS X after malware infection

While instructions are available to remove the latest malware threats for OS X, some may choose to simply reinstall OS X and start from a clean slate.

Topher Kessler MacFixIt Editor
Topher, an avid Mac user for the past 15 years, has been a contributing author to MacFixIt since the spring of 2008. One of his passions is troubleshooting Mac problems and making the best use of Macs and Apple hardware at home and in the workplace.
Topher Kessler
8 min read

The recent Flashback malware for OS X has caused a bit of a stir in the Mac community, and while it has only affected a fraction of the OS X install base, it still has had people who have indeed found the malware on their systems writing in to CNET and on the Apple Discussion boards.

For the most part, people have been finding the malware on their systems by having an antivirus scanner or reverse firewall such as Little Snitch installed, and have either been given an alert that the malware was either found or a program file with a short name beginning with a period attempting to contact remote servers via bizarre-sounding domain names such as cuojshtbohnt.com, and gangstaparadise.rr.nu.

These clear attempts have spurred investigation into the malware and have shown that this activity is the first part of the malware attack, where the malware has broken the Java sandbox and the program is trying to download the payload that will subsequently piggyback on local applications by altering launch environment variables either within the program or in the user's account.

So far the malware has been fairly well described, and is not viral in nature, so for any particular variant it installs to a single location and runs from there to affect the system. As a result, when a variant has been characterized, you should be able to remove it from your system by following detailed instructions. However, malware can change rapidly (as Flashback has demonstrated) and because new variants may appear that will change the attempted modes of attack, there may be those who cannot determine which variant they may have encountered and doubt their abilities to manually clear the malware from their systems.

In these situations, there are two approaches you can take. The first is to get a reputable malware scanner such as VirusBarrier, Sophos, or ClamXav, install and update it, and then have it scan the system for known variants of the malware. By doing this you can at least quarantine any malware files found.

This is a recommended approach; however, it does rely on malware definitions having been defined for the malware, which may lag behind initial findings of malware.

The second approach is to forgo attempting to manage the malware and perform an OS reinstallation. While this will ensure that you start from a clean slate, it will be a bit of a burden for some people to do, especially since you may not be able to trust Time Machine backups or system clones to be free from the malware and therefore may not be able to simply restore your system from a backup.

If you can remember an exact instance of when your system was affected by the malware, such as when you installed a recent update to Flash that might have been the malware, or when you first saw any other warning signs pertaining to the malware, then you might be able to reinstall using backup from before the problem occurred; however, in many cases you might not be able to reliably identify such instances.

If you have decided that it would be best for you to play it safe and wipe your system and start over, by following this procedure you should be able to do so while preserving your data.

  1. Sync and back up
    First ensure that your system is properly synced to your Cloud-based services (iCloud, Google, Yahoo, etc.) to ensure items like contacts and calendars are saved. You can also go to Address Book, iCal, and other programs that you regularly use, and export the calendars, contacts, and other data to save to a flash drive or other separate storage medium. Such actions will ensure you will be abel to restore some of these items without relying on sync services to manage them for you.

    In addition to syncing, be sure your system is backed up. Use Time Machine or a cloning tool to back up your files, or at the very least manually copy all the folders from your home directory to an external hard drive, and do this for every active account on the system by logging into each and performing these actions.

    When you are done backing up, unmount and detach the external hard drive you used for the backup.

  2. Deauthorize or unregister applications Some common applications like iTunes have authorization and registration features for viewing and managing content, so be sure to reauthorize these features before continuing as you might run into problems when configuring the programs again. For instance, iTunes only allows 5 computers to be authorized to a specific iTunes Store account, so you can deauthorize the computer by choosing the option to do so in the "Store" menu to prevent the store from assuming you have authorized more systems than you own.

  3. Format the drive
    Reboot the system to the OS X installation DVD for OS X 10.6 or earlier (hold the C key at startup with the DVD in the optical drive), or reboot with the Command-R keys held for OS X 10.7. When the OS X installer loads, select your language and then open Disk Utility (available in the Utilities menu if it's not presented in a Tools window).

    In Disk Utility, select your boot volume and then use the Erase tab to format it to "Mac OS X Extended (journaled)." This process should be fairly quick, and when done should leave you with a blank hard disk.

  4. Reinstall OS X
    Quit Disk Utility and then open the OS X installer. Do not choose any option to restore from backup. Follow the onscreen instructions to select your newly formatted hard drive and reinstall OS X, and then wait for the installation to complete.

  5. Create a new account
    When OS X is freshly installed it will ask you whether you would like to migrate data from a backup or from another computer. Avoid doing this, and instead create a fresh user account for yourself (you can use the same account name and other information).

  6. Update the system
    When you first log into your account, go to Software Update (in the Apple menu) and update the system to the very latest version. Run Software Update several times until no more updates are available.

  7. Deactivate Java
    The latest Flashback malware threats target systems with Java vulnerabilities. While Apple stopped shipping Java with OS X Lion, prior versions of OS X do have it installed by default. Often Java is not needed for running applications in OS X, so unless you have specific need for it, then turn it off. Even if you suspect you might need Java, you might consider starting with it disabled and then only activating it based on demand.

    There are two general ways to manage Java in OS X. The first is through application-specific settings such as the preferences for Safari, Firefox, and other Web browsers, where you can locate settings to disable the Java plug-in and Java management (do not disable JavaScript). These settings will ensure speific programs do not use Java, and for the most part will be enough to prevent Java from being taken advantage of on the system; however, if you reset Safari or install a new Web browser then you may inadvertently use Java.

    To prevent inadvertent uses of Java by programs, you can open the Java Preferences utility in the /Applications/Utilities/ folder and uncheck the listed Java runtimes to disable them systemwide. If upon opening the Java preferences you get a warning about needing to install Java, then your system does not have it installed and you do not need to do anything else.

    If you do need Java installed and active on your system, then be sure to apply the latest Java software update, and consider disabling it in Web browsers.

  8. Restore your data from backup
    The next step is to copy your data back to your system from your backups. Do not use Apple's Migration Assistant tool to do this since it will restore folders and applications that may have been altered by the malware, so instead copy the files from your Documents, Movies, Music, and other home directory folders to their respective locations within your user account.

    The current Flashback malware has affected contents of the user library, particularly the Launch Agents folder, and while you can restore the contents of the folder to your new user Library to preserve some settings and configurations, for the sake of the extra care being taken in this approach, it is best to leave that folder alone and only restore individual items out of it only as needed.

    At this point you can set up iCloud or other sync services in the system preferences, and then launch Address Book, Mail, iCal, and other programs you use to configure those programs and the accounts you use with them. If your contacts and calendars are missing, then you can re-import them from the manual backups you previously created.

    Perform steps 6 and 7 for any additional user accounts on the system by first creating the account, deactivating Java, and then restoring the account data from the backup.

  9. Reinstall applications
    The next step after restoring your accounts is to reinstall the applications you use. While your previous set of applications were backed up before you started this procedure, avoid restoring them or opening them because in one mode of infection the Flashback malware does directly alter some of these programs. Instead, use the backup as a reference for which applications you previously had and reinstall them from their installation discs, the Mac App Store, or other means by which you originally obtained them.

    When you have installed your applications, be sure to fully update them and then open and configure them according to your preferences.

    At this point your system should be back up to a usable state, and you should be able to continue your workflow as it was before reinstalling. If you find you are missing some required fonts, sounds, or other files that your applications need, then you can access them from the global /Library folder from the backup or in the /Library folder from your user account.

The final step in this process is to protect yourself from further infection. While disabling Java as mentioned above is one step, you can take additional ones to help secure your system. Install a reverse firewall such as Little Snitch to help detect and block programs from phoning home to remote servers, and consider installing an antivirus utility.

While you do not have to configure the antivirus tool to diligently scan all files on demand, you can set it up to scan common downloads folders only (such as the Desktop or the Downloads folder within your user account) and then once a week or perhaps once a month have it scan the whole system. For now, despite the latest malware news, this should be enough to ward off malware and provide you with ample protection.

UPDATED: 4/8/2012, 12:30pm -- Added information about deauthorizing applications before formatting (thanks to MacFixIt reader Michael N.)

Questions? Comments? Have a fix? Post them below or e-mail us!
Be sure to check us out on Twitter and the CNET Mac forums.