It depends on the model, since some only have two memory banks, others have four. Sounds like yours only has two, so in that case, yes, you would need to remove the old RAM to install the new RAM.
Again depending on the model, it shouldn't be overly difficult. You just unscrew the little cover on the bottom of the system, then if it's a white plastic model there will be little plastic levers you pull to one side that releases the RAM. All newer models have a little plastic flap that you pull down on, and you'll hear it make a kind of thump sound when it releases. The RAM SHOULDN'T come out all the way, but sometimes it will anyway. It should come out only part way, but enough that you can get it with your fingertips and pull it out.
Now, a quick primer on how to handle RAM modules to avoid ESD damage.
1: Always hold them from the edges
2: Never touch the black chips on the module
3: Never touch the gold "teeth" on the module
4: Never touch any other part of the module that looks like anything other than bare plastic
I tend to use the tips of my index fingers to hold it on the very edges where there are a semi-circular notches, then slide them into the slot, where I can then use my thumbs to push them in the rest of the way.
My IMAC is running slow after upgrading to snow leopard. Apple told me I should add more RAM and the rep told me where to buy it. I followed the directions off the Apple website but the only way I can figure out how to insert the new RAM (2 of 1GB) is by removing the old RAM (2 of 512 MB). Is this what I was supposed to do? I didn't see anywhere to add it without removing the old. It didn't mention it in the directions and I am not computer savvy in any way. Thanks for the help.