Buying a new computer has *MUCH* better hardware. However, the downside is that you are stuck paying for Windows (typically) with all the additional expensive software needed to maintain it. Windows has been known to need a lot of RAM for optimal efficiency and is the main target for hackers/crackers/malware/spyware.
Getting a used computer - XP will no longer be supported and 'abolished' per se by 2014.
You may consider getting a used computer and putting Linux on it. Usually free, does not require huge amounts of RAM, but DOES require a reasonably fast CPU for it to work well. Faster the better. It will work on any older CPU though.
Puppy Linux is among the fastest versions of Linux out there as it runs entirely from RAM (after assuming you load a saved file from a hard drive). RAM is generally faster than most hard drives.
If you need full featured Linux OS then you can get PCLinuxOS, Linux Mint, Ubuntu (bloated one sigh) and those are 100% free. Linux Suse is good too and so if Fedora. Linux is immune to ALL Windows problems.
For extra speed, you could also get an SSD Hard Drive. Small - yes. Expensive - YES. But quite fast, uses 1/10th the power of a regular hard drive, and is lightweight. It also does not put out any noise or much heat at all.
A lot depends on what you plan on using the computer for.
Gaming - either Linux with a virtual drive running windows or Windows.
Anything internet - Linux all the way! Far less chance of getting infected with problems than with Windows.
Business - Linux has a few options. Open Office is the main business software (free too).
Buying a used computer - if you can get one under or around $100 and use Linux, you shouldn't need to upgrade anything.
A boxed computer costs more. Building a screaming box costs a fortune.
No matter what your choice is, get a quality external USB/eSATA hard drive. Western Digital is about the most unreliable hard drives I have ever used.
Seagate among the best. But NOT Maxtor. Samsung is a work horse, but a bit slow due to the small drive cache.
SSD HD's - you need to research those. Too new to determine reliability.
I use a Crucial and a Samsung SSD HD's. I put XP on one and Linux on the other. Easy to choose which I need simply by restarting the computer and selecting the drive I want. I have far too much money invested in Windows software just to toss it - this is why I keep XP around for those 'special' times...
I use a real hard drive (internal) for temp files and data storage with an external 2TB for long term data storage.
Both Linux and Windows can use the regular hard drive for temp files & storage. If using Windows, be sure to put the page file on the real hard drive and also move My Documents to that too. Don't forget to move your temporary internet files there as well for both OSes.
My system is an old AMD64 X2 4200+ with 4gb of DDR Ram. It suits my needs pretty well.