Can you explain what you mean with ""off"? My monitor here has a power button to turn if off and on, and on the mouse and keyboard there's only a LED. Some motherboards keep power on the USB even when turned off, so on one of my PC's the mouse LED stays on when my PC is turned off.

And can you explain how they are "on" if you turn off the computer.