#1: Find an efficient power supply. Find a power supply that is highly efficient (+75% as rated on the box. Look for 80+) which runs cooler, and has a large fan that spins slower, thus making less noise. Not coincidentally, more powerful power supplies have bigger fans and are often more efficient. If you don't play games you don't NEED more than 350 watts, but a 450-600 watt PSU will be quieter because it isn't straining to get the power to your components.
#2: The CPU fan that came with your Intel or AMD chip sucks. Why? because the fans are tiny and have to run faster to cool the processor. Get an aftermarket heat sink with a big (80mm or larger) sleeveless fan. Once again, the larger the fan, the slower it can spin to do the same amount of cooling.
#3: Big case fans. Small case fans are louder because, once again, they must spin faster to move the minimum amount of air to keep the inside of your PC cool. Unfortunately, case designs and fan sizes go together...you can't replace small fans with larger ones because the fans must match the holes in the case. Thus, you want a case with large fan openings and large slow fans in the brackets. Many manufacturers have standardized on 120mm fans for quiet and efficiency; some fan mounts are made for 140mm fans, that's almost 6" across! Conversely, while you want maximum cooling, a case that's all perforations, mesh and vents is just going to let noise out. Get a case that has a solid front (except for maybe one fan opening) and as few side and top vents as possible, to keep noise inside where it belongs.
#4: Your video card. This is the noise source that you have the least control over since most people generally don't replace the fans on a graphics card (though I did, with a noisy ATI 9800 Pro, see below). Each manufacturer is different and you just have to read about the different brands once you have an idea of which card to buy. The more advanced the card is, the hotter it'll run, and the fans will be correspondingly louder. But even between models (let's take an ATI 4850, a midrange gaming card, for example). A Diamond Multimedia model may be quieter than a Sapphire model solely because of the maker's choice of fans. In general, the brands that are less well known spend less thought on noise, but they often cost less as well. Fact is, when you're not a gamer, video cards are much less noisy, so all the better for you. Get an inexpensive card and it may be quiet enough.
If you follow all these steps, you can still probably build a midrange PC for around $600-700, especially if you steal the drives and such from your old PC. But what if you DON'T want to build your own PC?
If you really intended for your question to be, "well I don't want to build a PC by myself; whose brand is exceptionally quiet?" then I have a weird suggestion for you. Wait for it...
Buy a Mac.
Yep, a Mac. Apple computers, for all their faults and priciness, are among the quietest computers you can buy. Each Macintosh model is designed from the ground up to be noiseless, and unless you get a loud aftermarket video card (the Apple-branded ATI 9800 Pros were really well-known for this), are virtually silent in day-to-day operation, gaming included.
If you don't want to be a switcher, the "new PC" rule to follow is, the cheaper the box, the lousier the noise abatement. Look at HPs, Dells, whichever, but if it's a low-end PC, it'll have low-end parts in it and less time spent on engineering. There's a reason people who buy better PCs are happier with them: it's because the designers went a little further to do good industrial design. And that benefits you, the user.