Computer keeps freezing

For the past 3 months or so my PC has been freezing. It started mainly on start up but lately it could freeze 3 minutes after start up or 2 hours after. The pc was built by my brother in December with all new parts. The usb ports at the front have always given trouble, and the 3.0 port doesn't work at all, but I believe that's due to the case.
I tried changing the graphics card at first and that fixed the problem for a day, I then changed back to my original graphics card and changed the update to a previous one. That solved the problem for a month before the problem started back again.
I then changed back to the other graphics card but this time the freezing continued. I put the CPU under a stress test and it only froze after I turned off the test. I tried using a different ram stick and tried putting it in different slots. I disabled the front USBs and changed the netweork cable, which fixed the problem for a few days before it started freezing again. The fact the computer doesn't freeze in safe mode leads me to believe it's either a software issue or a motherboard issue. All fans are working and the PC temperature is often normal when it freezes. I also just recently updated the bios after ASUS suggested I do so. It baffles me that the pc can somehow be fine for a few days to a month but then start freezing all of a sudden. However, it has been freezing non-stop the last three days which is unusual. Yesterday is was freezing in the morning but I left it in safe mode for a few hours. When I restarted it again it, I managed to play a few hours of a game and also got to use skype, it only froze after closing the game.

Any ideas? it's not alot to go by but I've started emailing ASUS about the motherboard in hopes they'll have a look at it or replace it but the emails are slow going back and forth.

I have an ASUS m5a97 R 2.0 mother board and a 500w corsair power supply which I was under the impression is enough but, some say I might need a higher wattage?

Thanks.