Tutorial: Troubleshooting Spotlight

CNET staff

Sept. 3, 2009 9:45 a.m. PT

8 min read

Spotlight was touted as one of the most generally useful new features in Mac OS X 10.4 (Tiger). Using a home-brewed metadata indexing system, Apple delivered a search mechanism that claims superior speed, accuracy and manipulation of results relative to engines offered by previous iterations of Mac OS X and various Windows flavors.

Unfortunately, many users found Spotlight to represent a step backward in some respects when compared with the simple find functionality offered in Mac OS X 10.3 (Panther) and its predecessors. On slower systems, or when subject to index corruption and other problems, Spotlight itself can prove sluggish and cause other applications slowness as well. It can also have problems returning accurate results under some circumstances.

Let's take a look at some of the most pressing Spotlight troubleshooting issues and how to work around them. As a last resort, we'll examine turning Spotlight off completely and seeking a replacement.

Spotlight's limitations First, there are a couple limitations you should know about before deciding whether or not to use Spotlight as your primary local search engine.

Invisible files Spotlight, in its current form, refuses to find invisible files. Although the user can set a criterion to search for both visible and invisible files, or invisible files only (through the Finder-based method of accessing Spotlight), no files are found.

Files containing single asterisks Spotlight cannot find files and folders containing a single asterisk (*). It can find a single asterisk in a file name as long as it is not at the end of the name or a word in the name without an asterisk at the beginning of the name.

For instance, out of the following filenames:

*XXXXX

XXX * XX

XXX*XX

XXXXX *

XXXXX*

XXXXX**

Spotlight will find 1, 2, 4, and 6, but miss 3 and 5.

Places it will not search There are some limitations to where Spotlight will search. For instance, Spotlight will not search in bundles such as frameworks and application packages.

Performance boosters (for Spotlight and the system) As aforementioned, Spotlight can be downright slothful when afflicted with index problems, when other system processes are hogging resources, or in other situations. Fortunately, there are a few procedures you can use to boost the speed with which search results are returned.

In addition (and more importantly for some users) there are workarounds you can use to minimize Spotlight's impact on the rest of the system.

Paste search strings One of the biggest gripes with Spotlight is a sometimes dramatic pause that occurs after typing one or two characters, as Spotlight begins searching immediately for all data containing those letters. There is a simple tip for avoiding this slow-down: Type the search string in an open text field then copy and paste the text into the Spotlight search field.

Disabling indexing for certain folders For some users, simply turning off Spotlight indexing for particular problem folders will resolve performance issues -- decreasing the time interval between search term entry and results display, as well as taking less of a toll on the system.

For instance, the mailbox folders used by some e-mail applications will be re-indexed every time a small change is made, resulting in spikes in processor usage.

Folders can be excluded in the "Spotlight" pane of System Preferences in the "Privacy" tab. Simply click the "+" button in the lower left corner and select the folder to be excluded, or drag the desired folder to the list.

Temporarily disabling Spotlight and re-launching Another workaround that has proved successful for some users is simply turning off the Spotlight indexing process for installed volumes then turning it back on. Here are instructions for doing so in the Terminal:

1) Using the mdutil command-line utility in Terminal, turn off indexing for each of your drives. Example:

sudo mdutil -i off /Volumes/your_hard_drive_name_1

sudo mdutil -i off /Volumes/your_hard_drive_name_2

2) Then use mdutil to remove the indexes from each drive

sudo mdutil -E /Volumes/your_hard_drive_name_1

sudo mdutil -E /Volumes/your_hard_drive_name_2

3) Physically remove the .Spotlight directories from the root of each drive.

cd /Volumes/your_hard_drive_name_1

sudo rm -fr .Spotlight-V100

(do the same for your second or third drive)

Make sure to carefully type the "rm" command -- a typo could result in deletion of critical files.

4) Use mdutil again to turn indexing back on for each drive

sudo mdutil -i on /Volumes/your_hard_drive_name_1

sudo mdutil -i on /Volumes/your_hard_drive_name_2

5) Spotlight will now re-index all drives and should behave in a normal fashion. (No longer use 60%-80% of your CPU)
Killing the mdimport process Klling the mdimport process via the Terminal commands:
ps -auxwww | grep mdimport
followed by
kill -9 theprocess#returned

reduces processor monopolization by Spotlight.

Giving Spotlight processes less priority via "renice" For some users, giving Spotlight processes less system priority via the "renice" command results in better overall system performance, though performance from Spotlight itself may suffer.

The easiest way to perform this process is to enter the following command in the Terminal:

renice -n (new renice setting number) -p (process ID number)

The higher the renice setting number (up to 20), the lower the process priority. The process ID number for various Spotlight operations can be found using Activity Monitor (located in Applications/Utilities). The most implicated process is mds -- launch Activity Monitor, and note the process number next to mds.

renice -n 19 -p NNN

(where NNN is the process number received from Activity Monitor. You can also get the process ID for "mds" with the Terminal command "ps -acx | grep mds")

This can also be accomplished with the freeware utility BeNicer. After downloading an launching this application, enter "root" as the user and select the "mds" process, then change its renice setting to a higher number (18 or 19).

Disable Spotlight for backup target volumes Users should disable Spotlight for backup target volumes (by using the "Privacy" tab of the Spotlight pane in System Preferences) before beginning the backup process to prevent live indexing as the files are copied.

Failing to do so can result in failed backups, significant performance issues, and other problems.
Note that this procedure must be repeated each time a backup drive is unmounted, then re-connected and made to receive data.
Do not attempt to use Spotlight before other menu items load Scott Rose notes that if you attempt to perform a Spotlight search from the Mac OS X menu bar before other, third-party menu bar items have loaded, it will not function properly.

Kill the SystemUIServer process Launch the Activity Monitor application (located in Applications/Utilities). Look for the Process Name SystemUIServer. Select it, then click "Quit Process" (the red button in the upper right that looks like a stop sign) and confirm the quit. The menubar should refresh after quitting this process. Wait a minute or two then re-attempt usage of Spotlight.

Improving results accuracy

Reviving unsearchable folders In some cases, folders can seemingly spontaneously become unsearchable by Spotlight, with the search function refusing to find the folder itself or any items contained therein. This problem can sometimes happen after an Archive and Install process. For instance, if items in the "Previous System" folder are moved back to an active user folder, Spotlight may not be able to find anything within that user home folder. This problem can usually be remedied via the following process:
Open the Spotlight pane of System Preferences

Click on the Privacy tab

Drag the problematic (unsearchable) folder into the field, adding it as a place Spotlight does not search

Quit System Preferences

Re-open System Preferences and again click on the Spotlight pane

Remove the folder from the Privacy field

Quit System Preferences again

Files that can choke Spotlight Certain types of files are not handled well by Spotlight, include the following:

Future creation/modification dates causing problems, workaround Certain iterations of Spotlight have some serious issues dealing with files that have creation dates in the future.

As such, if you've not yet installed Mac OS X 10.4 (Tiger) on a system, you might want to run run Norton Disk Doctor 8.02 (which has a feature that corrects these files dates) to fix such wrong dates before installing Tiger. Norton Disk Doctor is not compatible with Tiger, so, again, it should be run pre-update.

Finder.DAT files Spotlight may have problems indexing and searching for Finder.DAT files -- data files usually containing text created by an older Classic Mac OS application like ClarisWorks or spawned by another third-party application. Try eliminating these files if you are experiencing stalls or other problems with Spotlight.

Entourage data Spotlight may not properly search Entourage data under some circumstances. There are a few fixes for this issue:

Open Entourage's preferences and press "rebuild" under Spotlight Preferences

Restart and wait a few minutes while Spotlight indexes the new data.

Move the files "Microsoft Entourage.mdimporter" and Microsoft "Office.mdimporter" from /Library/Spotlight to ~/Library/Spotlight and then use the following command in the Terminal (the Terminal is located in Applications/Utilities): mdimport ~/Library/Caches/Metadata/Microsoft/Entourage/2004

Disabling Spotlight, turning it off completely; seeking alternatives

You can temporarily disable Spotlight easily with the shareware tool Spotless. This tool also allows you to delete existing Spotlight indexes, and can easily recreate the ".Spotlight-V100" directory, which can become damaged for various reasons and cause indexing problems.

If you absolutely cannot live with Spotlight in any dose, you can turn it off completely by editing Mac OS X's hostconfig file, which determines the system components that are loaded at startup. This can be accomplished via the following process:

First, enter the Terminal and use the command:

sudo pico /etc/hostconfig

to open the hostconfig file in the "pico" text editor. (For more information about using pico, type the command "man pico" in the Terminal).

Scroll down the hostconfig file and file the line that says "SPOTLIGHT=-YES-" and change it to: "SPOTLIGHT=-NO-" (without quotes).

Exit pico, saving the file, and restart Mac OS X. Spotlight will no longer launch at startup.

To go one step further and remove Spotlight from the menu bar, remove the file "Search.bundle" from the folder /System/Library/CoreServices (move it to the Desktop or another external location) and again restart.

Once you've removed Spotlight, it's time to find a replacement. The most popular alternative search utility for Mac OS X is EasyFind. This program does not rely on indexing to do its thing, and is freeware. It will not deliver intra-document results with the accuracy achieved by Spotlight, but is considerably less resource-intensive and does not suffer from some of Spotlights file-type limitations. It can, for instance, search for invisible files and files inside packages. Other alternatives include:

NotLight

LiveSearch

NameFind

FoxTrot Personal Search 1.1