...which ones have you tried? Clicking then delete, right-click & delete, CUT without paste, drag to the recycle bin are 4 that come to mind. If none of these work, you can get hold of a live CD like Ubuntu, boot from it and do the delete.
`
Good luck.
I have three icons for files I downloaded but later deleted in my Download folder (Windows 7). When I try to delete them, I get a "This is no longer located (in the folder) error message and I'm not allowed to delete these icons. They appear each time I reboot my system. The actual files are gone. It's the icons I want to get rid of.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic