The short answer is yes, you will be able to do all that you have in mind with the equipment you have now. In fact, it sounds like you have everything hooked up already to do this. All you need to do now is setup the network on each machine. You did not mention what operating system you are running (Windows XP Home/Pro, Win 2K, etc.) And that will make the details change a little bit.
Once it is all setup you will just have to create a share with the folders that contain your music files and you will be able to access them from the other machines an visa versa. It is always a good idea to have a back up of these things though, even though I understand your reluctance to having to copy them all over to your new machine.
As far as to the "how to", there are a lot of sites out there that will walk you through this. If at least one of your machines is running Windows XP it has a built in wizard to setup a home network, and it works pretty well usually. It even contains a function to help you use your XP disk to setup other computer that are not running XP themselves.
Be sure to disable, or understand how to configure your firewalls if you are running any. This is a common problem people run into when they try to setup a home network.
Hey guys,
I'm new to posting to the forums but have used them several times for the great advice that is on them. I would first like to say thanks to all the people that post replies to the questions here.
I have some questions regarding setting up a wired broadband internet connection. Please bare with me as I am new to networking so my questions may seem trivial to some.
I was recently given a Pentium 4 with a DI-704P 4 port router (http://www.dlink.com/products/?pid=63). The new P4 is a significant upgrade from my older P3 and Celeron laptop, however, most of my music collection is on my P3 and my wife's collection is on the laptop. Here are my questions;
1.) Is the router that I was given all that I need to set up a home network so I can share the files amongst all the computers?
2.) I have plugged in the computers to the router and the router to my cable modem and all is well, all the computers can successully access the internet at the same time. However, how do I go about configuring them to share files amongst one another as well? (Anyone have a decent link to a basic step-by-step process on the web so you don't have to waste your time typing a detailed description?)
3.) Once I succesfully network all these computers together, will I be able to play the mp3's on my old P3 from my newer P4? Or will I have to still copy all the files over from my P3 in order to listen to them on the newer P4?
Thank you all very much for your time and advice on this, it is really appreciated.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic