Recheck all your work, you may need to disassemble to do that. While you may not need washers on the posts, verify that those posts are located on solder pads, metal-to-metal contacts. If not, then a fiber/paper washer should be used at weaker non-metallic mounting points.
The noise is probably from some capacitor its about to blow or close to it as it tries to maintain its voltage levels. -OR- its some fan that vibrates or seems to as its loose the same for any other loose mounting or device, they need be snug but not over tighten.
As for the power supply/PSU its has caps as wells, so be sure all those connectors are properly used. No, not all connectors are used as some are required for different uses and yours may not need them all. Just be sure what those are and verify the 115/230V selector the PSU is at 115V(in USA?).
I have just assembled my first PC (Foxconn micro-ATX motherboard, Rosewill 450W PSU, GeForce 7600GT, 2x512MB RAM, 120GB HDD). I connected everything to everything else, then installed the power supply. I made all the necessary connections except for the ATX 12V connection on the motherboard to the PSU. It was an 8-hole connection, but the PSU didn't have anything like that. It had a 6- and a 4-pin connector, both of which fit, but I ignored it. I also did NOT place washers or anything between the motherboard and the case.
When I plugged in and turned on the PSU, there was a buzzing noise around where the motherboard connected to the PSU at the 24-pin connection. When I turned off the PSU, the buzzing persisted for a second, then went up in pitch and faded out. I tried this a couple of times, achieving the same results, then disconnected the 24-pin connection. I turned the PSU on, and nothing happened. Then, I reconnected the motherboard to the PSU and turned on the PSU again. Nothing happened. No buzzing. Now I can't get it to do anything at all.