In several situations when running OS X you may encounter question marks. Of course there should definitely be one somewhere on your keyboard (likely next to the right Shift key), but there are other instances where question marks may appear in the system where there shouldn't be one. Usually these indicate an instance in which the system cannot locate a specified file, location, or other resource and in its confusion shows a question mark.
When OS X boots, it will by default use the boot volume that is set in the PRAM, but if that setting is not valid or nonexistent, then the system will poll each drive for a suitable boot environment and attempt to boot from the first one found. When the system is in the process of looking for a boot volume, it will display a blinking question mark to indicate that a boot volume has not yet been found.
If there are no boot volumes found, the system will continue to show the question mark; however, sometimes people may see the question mark only briefly before the system boots normally. This is because even though there is a valid boot partition for the system to use, it is not set as the default boot drive, so the system has to poll each drive until it finds the boot volume. To prevent this, go to the Startup Disk system preferences and select the default boot drive.
Another source of question marks in OS X is the toolbar at the top of each Finder window. While you can customize the toolbar to rearrange items and add some optional buttons, you can also add custom links to documents, folders, and even applications. While the Dock is the recommended location for storing links to common applications, some applications (such as those that will open the current Finder folder in the Terminal) can be useful to have in the Finder toolbar and can be put there by simply dragging the icon to the toolbar.
If the linked file in the toolbar is then removed, the system will not be able to locate it and will show a question mark in the toolbar in place of the file's icon. When this happens, you can either restore the moved or deleted file, or remove the link from the toolbar by holding the Command key while dragging it off the toolbar.
Similar to the Finder toolbar, the Dock will also show a question mark if a file, folder, or application that is linked in it is no longer accessible. This may happen if you have deleted the file, if the file is stored on a networked or external volume that is not mounted, or sometimes if you run an applicaiton uninstaller that removes the file but does not remove it from the Dock. If this happens you can drag the question mark off the Dock and it will disappear with a cloud poof.
Have you found other locations where OS X shows similar question marks? If so then let us know below in the comments.