then it wouldn't get better.
One thing I found about ME. If you install with less memory, like 64mb. you get a better install. then put the rest of the memory back in when your finished.
ME need a lot of ram to run because it has a memory dump issue but, you don't need it all to install it.
You could have a harware issue but, why does it get better after 10 mins.is the question? doesn't make since either. Have you cleaned the inside of your case lately? Maybe the cpu fan is clogged and it takes awhile to get up to speed. That would make some since. Give us more details to go on.
I have a Gateway Select 1000 running ME and it locks up continuosly the first 5-10 minutes after startup. If I let it run for 10 minute and then re-boot or leave it on it works flawlessly. I've done a complete re-install of Windows ME to make sure it isn't a software problem. I feel this is hardware problem effected by heat. Is there a way to check the possibilties without bringing it in?

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic