The first thing I'd try is Disk Warrior. The desktop directory might be corrupt, which is what is keeping FileMaker from opening the document. If that doesn't work, then I'd give Norton Utilities a try. It's still pretty good at file recovery and restoration.

Failing that, you could try TechTools but I've seen few situations where it did anything helpful that Disk Warrior didn't do.