I've found that even though Firefox is set as the default browser, sometimes IE will still stick it's ungly, insecure program right in your screen. 
To deal with the file associations, go start->control panel->folder options (in Appearance and Themes), select the "file types" tab, and scroll down until, you reach "HTM" and "HTML." Select one, click "change," find Firefox (if it's not on the list, browse "c:\program files" for "firefox.exe"), make sure the box next to "Always use the selected program to open this kind of file" is checked, click "OK," and repeat for "HTML."
Another way of doing this is to find a shortcut (stored on your computer) to a webpage, right-click it, select "properties," click "change" next to "opens with," select Firefox, check "always use...," and OK your way out. However, it will only handle one of the two (usually HTML, not HTM). Thus, the first way is best.
As far as removing Internet Explorer completely, you're right to have second thoughts. IE is required for downloading updates from Windows Update, as well as for accessing a select number of websites which use ActiveX controls (not supported by Firefox) or are otherwise incompatible with Firefox. Thus, it's recommended to leave IE installed, but only use it when you have to.
Hope this helps,
John