In Vista and 7, I don't believe you can remove it, only disable it. In XP, it should be listed if you installed it. Since you neglected to give an OS, you'll just have to sort the rest out yourself.
I want to uninstall windows defender and it's not listed in my programs when I go through control panel. How do I unistall Windows defender?