The +12V rail was split and barely 20A per rail. The GPU will demand a lot more so that explains why it works without the GPU.
-> Use single RAIL PSUs is the advice since it avoids you having to do deep research.
(Sorry if my English isn't that good, not my first language)
I have a problem with my two years old PC. Sometimes, when I try to start it, it will boot normally for about 5 seconds, and then it will shut down abrutly, as if there was a power outage. There are times where it will function normally for hours with no problems, and times where it will shut down after a random action (connecting my keyboard or my ethernet cable) or for no reason whatsoever. And sometimes it will shut down at launch everytime I try to start it, until I try to move the cables and the fans inside a bit, and then it will start normally.
There really doesn't seem to be any kind of rhyme or reason for this problem and it's driving me nuts.
My friends seem to think the power supply is at fault here, and someone lended my a more powerful one (I have had a Be Quiet! Pure Power L8 CF 500w for two years, and the one he lended me is a 600w) but the problem stayed.
I tried disconnecting some component of the PC, and it seems like it's running fine when I disconnect my graphics card. I have a GeForce GTX 960 4Go.
My processor is an Intel Core i5-4460 (3.2 GHz), I have one 8 Go 1600 MHz ram bar, a 1 To hard drive, and a 240 Go SATA III SSD drive.
All of the components have been bought separately two years years ago.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic