Like I said, it's the best IE out there, but that's not really saying much. It lags behind other browsers to such a significant degree in almost every area, if it didn't come bundled with Windows no one would ever use it. Generally speaking, no one ever did until Microsoft started bundling IE4 with Win98. They actually started earlier with some little known releases of Win95 and IE3, but Win98 + IE4 is what kicked off the big anti-trust lawsuit (of which Microsoft was found guilty), and finally propelled IE into its dominate position. Until then, it barely made a dent in Netscape's market share.
And your suggestion, which was already mentioned, doesn't actually remove IE. That just removes the front end program most people associate as being IE, but it really only represents a control program. It's maybe 10% of the overall program. You cannot remove IE and leave Windows in a usable state, Microsoft has seen to that. They've been steadily working it further and further into the core of the OS for over the last decade. It started with the sunset years of Win95, then Win98, it became a prerequisite to NT4 service packs which caught some in a vicious cycle of needing IE to install a service pack and needing a service pack to install IE. WinMe and 2000 both continued the trend, as did XP, Vista, and 7. Each successive release has seen IE worked ever deeper into the OS. By today, Windows Explorer and Internet Explorer are largely one and the same.