if you want to dig a little. Find the two wires coming from the switch. Trace them to the motherboard connectors. The switch is the "momentary" type which means the contacts close the connection on the wires only while the button is pressed. You can do the same thing with a small screwdriver between the two connectors on the motherboard. If it powers up every time, the switch is bad, a not uncommon occurrence.

You might find a replacement switch here, input the PC make and model number.

http://www.impactcomputers.com/dellparts--parts-for-dell-dimension.html