Just to be be clear: resetting your router is different from restarting your router. A router reset will allow you to set it from scratch where it will work at optimal levels. Do this when you feel you're having issues with your network or after you've upgraded your internet speed and don't seem to get the speed that you're supposed to get. It's also the only way you can re-gain access to a router's web interface once you've forgotten the password (the password will then be restored to the default password.)
It's important to note that when you reset your home router you will lose all of your current network settings, like the name of the Wi-Fi network, its password, etc. (So don't do this just for fun!) If you want to retain the current network settings, you should first log into the router's web interface and save the current settings to a file (most routers will have this setting in the web interface). While in the interface, you can choose to reset the router via its web interface using a function called Reset or Restore, which restores the router to its factory default condition.
The most universal way to reset a router, however, is via the reset button, usually found on the router's back or underside. This button is almost always recessed but you should be able to access it with a straightened-out paper clip.
To reset, make sure the router is turned on, press and hold the reset button for 10 seconds, then release it. The router will then restart in the factory default setting. Now you can proceed with setting it up like you did when you first got it or, if you saved the settings, restore them from a file.
By the way, don't reset your modem, that's the job of your internet service provider. On most router/modem combos, the reset button will only reset the router portion of the device.