It should latch when you press the start button. I'm thinking the green light indicates the presence of the control voltage that, when grounded through the start switch, is supposed to latch the power supply to the on position. You might try and open it and reseat connections and devices as you can. Clean out dust build up, etc. If it has overheated, however, things can get damaged.
I have an HP Slimline s7320n that wont start. When I press the power button it does nothing but when I hold the power button down the HDD spins but nothing else and stops when I let go. There is a green light on the back of the power supply that stays on when the computer is plugged in. The thing that I do not understand tho is that I can sometimes I can get the computer to turn on but it will freeze after a few hours of running and wont turn back on after. I believe it is something with the motherboard but I cannot find a new motherboard anywhere and it is out of warranty. Does anyone have any suggestions? Thanks a lot!