I am sure if you are not using Disk Degragment utility then it doesn't take any memory or disk space to make your system slow.
I prefer not to remove the defragment utility. It can be helpful any time.
Is there any reason why I shouldn't uninstall or delete Windows Defrag program? I have installed another defrag program and would like to remove any programs no longer used, such as mentioned above.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic