They are referred to as HSF [Heat sink& fan]. Is it operating at full speed???
Do you have any case fans to bring room ambient air into the case to help have cooler air available to the HSF to help it keep the CPU cool.
By cache, do you mean the amount of RAM memory that you have in the system? You can't increase the cache without replacing the CPU.
You could use more memory, but remember that the memory will add additional heat to your system, so maybe solve the temp system first. Maybe by providing a little more clarity about the system and the fans [all of them].
Leaving the PC on doesn't lead to a hotter unit. The system will reach thermal stability in relatively short order, it does not continually get hotter. The temp of the CPU will vary at various times depending on the amount of work it is doing at any instant. The temp can also change,somewhat, as your the temperature in your room changes.
Loud,sounds funny because, in general, the way one gets better cooling is to have fans that run at higher rpm which also means they make more noise. That is unless your fan has a bad bearing or bushing and is wobbling and making noise.
Okay, I'm using a fan and grill to stop my CPU from becoming a an egg fryer!.... my CPU is around 60/80c I'm not too sure....... It's a Celeron 2293MHz at 128K Cashe (I know, I should upgrade!) my queston is.... because I always leave my computer on, my CPU might heat up as my PC is an OEM (eMachines) so it's not designed to idle..... also it's LOUD!..... I was thinking of maybe a CPU cooler (socket 47![]()
http://www.overclockers.co.uk/acatalog/copy_of_Gigabyte.html
..... Prehaps it'll be more stable?

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic