Workarounds for quarantine bug in OS X Lion
Changes to the sandboxing rules in OS X Lion appear to be flagging files as quarantined when they've been edited by sandboxed applications, with the result that scripts and other executable files no longer work.
Quarantining files has its benefits, but a current bug is causing some trouble.
When you first download a file in OS X the system considers it to be foreign and tags it with a quarantine flag. This flag prevents it from running if it contains executable code, at least until it is scanned and the user is notified that the file contains code and could potentially change files on the computer. If the file is an application, then when you open it the launch services will present a warning that the file was downloaded from the Internet. Once you confirm that you wish to continue, the quarantine flag is removed.
However, there appears to be a bug in the latest version of OS X Lion in which the quarantine flag is attached to existing files on the system, preventing some code such as user scripts from being executed. If you create shell scripts, for instance, then opening them in TextEdit and making changes to them will result in them no longer being executable and will result in an "Operation not permitted" error in the Terminal.
Some users have found that this problem happens even if you make edits but do not save the file, suggesting the problem could lie with the system's Autosave features. But if this were the case then it would happen to all programs that support Autosave such as Pages or other iWork programs, and that hasn't been reported happening, which suggests the problem is elsewhere.
After investigating the problem, I found it appears to only happen with Apple's sandboxed applications, such as TextEdit and Preview, suggesting the issue has to do with alterations to the sandbox rule configuration in the OS.
Sandboxing is a method of voluntarily restricting a program's ability to access system resources, and this is done by subscribing the program to a predefined rule set, called "entitlements," when it is compiled. The entitlements, which are defined in Apple's frameworks (central libraries), can be altered at a later date to refine how sandboxed applications behave, and it appears Apple may have adjusted some of these in such a way that they now result in any newly edited documents from sandboxed applications being quarantined.
To see this in action, open TextEdit and create a new file in your home directory called "test" and put some text in it. Then open the Terminal and enter "xattr test.rtf" to print out the extended attributes associated with the file. You should see both "com.apple.FinderInfo" and "com.apple.quarantine" appear in the Terminal, which indicates the file is flagged as being quarantined.
Now remove the quarantine flag by running "xattr -d com.apple.quarantine test.rtf" in the Terminal, and then repeat the xattr test.rtf command to confirm that the flag has been removed. If you now edit the document, and then run the xattr test.rtf command to list its attributes again, you will see the quarantine flag has been reapplied. The same thing happens to documents opened and edited with Preview.
In most cases this should not be a problem since most documents are not executable, but since TextEdit is the editor that Apple ships with OS X, a number of people use it to make quick edits to scripts and other executable files on the system, and so because of this bug, in the latest version of Lion their scripts will no longer run.
At this point there are a couple of options:
- Remove quarantine attributes.
If you rarely use TextEdit to modify script files, then you can reverse the quarantine by removing the attribute from the file, similar to the demonstration above. To do this, locate the file in the Finder and then open the Terminal application. In the Terminal type "xattr -d com.apple.quarantine" followed by a single space. Then drag the file to the Terminal window, which should append its file path to the command, and press Enter to run the command.
Often people put edited multiple scripts in a central "scripts" folder they have created, in which case you can easily remove the quarantine flag from all the files by targeting the parent scripts folder recursively instead of individual files. To do this, type "xattr -d -r com.apple.quarantine" in the Terminal followed by a single space, and then drag the parent folder containing the scripts to the Terminal window, and press Enter to execute the command. - Use another text editor.
The problem here has to do with sandboxed applications that interface with the system's launch services, so you can use a text editor that is not sandboxed such as TextWrangler, or if you are familiar with the Terminal then any command-line text editor should work just fine (most people who script should have some familiarity with vi, pico/nano, or emacs). - Disable quarantining.
The system's quarantine feature is a good security measure so I would not recommend this to everyone, but if you are confident in the files you download and run on your system then one option is to disable quarantining altogether. To do this, open the Terminal and run the following command, then reboot the system:
defaults write com.apple.LaunchServices LSQuarantine -bool NO
This command will use the defaults system (the preferences editor) to target the Launch Services preferences and change its setting for "LSQuarantine" to a false or "off" value, so the service will no longer flag new files with the quarantine attribute.
Questions? Comments? Have a fix? Post them below or e-mail us!
Be sure to check us out on Twitter and the CNET Mac forums.