Firefox will leave registry entries for your favorites, etc. This from the Firefox page:

Note: Do not unzip Firefox to a directory that already contains a previous version of Firefox. Either delete the existing directory, move it out of the way or create a new directory. If you install Firefox on top of an existing Firefox or Mozilla directory you will have problems.

http://texturizer.net/firefox/download.html#install

But I'd back up everything just in case.