Sandbox errors may temporarily break specific OS X application functions

To fend off attacks and faulty programming, Apple has implemented sandboxing routines to manage what system resources applications have access to. While the sandboxing process should allow the applications to run as intended by the developer, sometimes an odd bug or configuration error may result in random breaks in functionality.

For example, TextEdit is one of the sandboxed programs that Apple includes in OS X. Sandbox entitlements allow it to save to the filesystem and accept keyboard inputs, among other features, so you can use the program to create documents and edit them. However, if a sandboxing error involving one of these entitlements occurs, then it may prevent the program from accessing the specific resource such as accepting character input from the keyboard. So for example, you might be typing normally at one point and then experience a freeze in which no new characters are added to the document.

While such behavior might cause you to suspect a system hang, before forcing items to quit or forcing the system to restart it will help to first better characterize the problem.

As with many errors in OS X, one of the first steps to take when characterizing it is to open the OS X console and see if any relevant errors or warnings are being listed. For more information, you can click the "Clear Display" toolbar button in the console and then try the performing the functions in your faulty program with the Console open in the background. When an error occurs the console will often show rapid groups of repeated error messages as it attempts a routine several times before failing, or runs into a problem with an error repeating for buffered data or queued tasks.

If warnings immediately show up in the console from your actions in the program, then you can see if they are from the program itself, or from an associated process such as the background sandboxing routines or the kernel. If you look at the screenshot in this post, you will see the repeated sandbox-related errors associated with inputs, suggesting that an error is now preventing TextEdit from requesting access to a system resource from its sandbox.

Since in this case TextEdit is not accepting character input due to a sandboxing fault, you can see if the problem is specifically with characters or with all inputs to the program by using hot keys such as Command-S for saving or Command-O to open a new document. If these hot keys work then it shows that the keyboard is working fine, that the application is able to use them, and that the problem is specifically with characters not being passed to the program.

In addition to checking alternate key inputs, you can see if this is a global problem for all applications or one that only affects the current program, by switching to or launching another program and trying the input there. Sandboxing is set up on a per-application basis, but the rules used are configured in central entitlement files, so testing with another application might better characterize the problem as a bug or a fluke.

To clear these problems, in most cases all that's needed is a simple relaunch of the application, which may involve quitting and opening the program again; however, if the problem happens in multiple applications (indicative of an error in a background task), then you may need to either log out and back in, or restart the system. Luckily for Lion users Apple's Resume feature will autosave work flows and restore them when programs are next launched, so quitting and relaunching the program or restarting the system to clear problems such as this should be relatively painless to do.

Questions? Comments? Have a fix? Post them below or e-mail us!
Be sure to check us out on Twitter and the CNET Mac forums.