Here's the skinny.
When you download a file (program) it is usually compressed in some way as well as being formatted to ensure that nothing happens to it along the way. The compression is usually designated by the suffix .sit.
Once the file has been decompressed, you are left with the .DMG file which is a disk image file. It will mount on the desktop as the "white lozenge shaped icon" If you have ever put memory card into a card reader while it is attached to the Mac, you will see the same thing. it is a DISK.
Double click it and you will see all the goodies inside it. Let's take Firefox as a case in point. Once you have downloaded and decompressed the .sit file and the .dmg file is mounted on the desktop, double click it. Inside you will find the Firefox application. There used to be a text file that explained how to install firefox. It said, words to this effect, drag the contents of this folder into the applications folder on your hard disk. Do not run Firefox from this location." Unfortunately, that text file is missing from the latest download of Firefox.
So, before you do that, go to the Applications folder and create a new folder and name it Firefox. Now drag the application from the mounted disk image on the desktop into the newly created folder. Once it has moved over there, you can trash the "lozenge" It is no longer required.
Back in the application folder, inside the Firefox folder, drag the icon down into the dock. Remove any other Firefox icon from the dock.
All is now well. Basically what you were actually doing was running the program from the disk image. Like running a program from the CD.
I think you will find everything is ok now.
Here endeth downloading and installing, 101
P