This is an MRU list, or "Most Recently Used", and there are, or should be, options within WMP and within Windows itself to manage such lists. But finding these options is not always easy.
So I would use a file-cleaning utility that also deletes such lists. There are many, but the one we recommend more than others is CCleaner, from http://www.piriform.com/ccleaner
It's a free utility, but if you choose this, just watch the install process carefully. Sometimes CCleaner will offer some 'extra' like a browser toolbar or an anti-virus scan, that you don't need. Make sure you remove any such offer during the install process.
With CCleaner you can safely remove unwanted temp files and temporary internet files, and other things like mru lists. The image below shows the "Applications" part of the opening page which has Windows Media Player selected by default. Sorry, the image isn't that large;
See if that helps.
How do you clear streamed videos on windows media player? I've got the history folder cleared but when I play a video you can click on previous list and it goes back to some old videos that shouldn't be there......