Records, aka LPs, have been around since the 1950s, so there are lots of them out there. I've bought great records for a buck or two at thrift shops and yard sales, and found them on the street for free, but records aren't yesterday's news; lots of young bands are releasing LPs. The way things are going, the LP will probably outlast the CD as a mainstream format.
Speaking of yard sales and thrift shops, you can probably find dirt cheap turntables in those places, but the chances of finding a turntable in good working condition there are small. Turntables are fairly delicate devices, and there are lots of ways they can go bad that might not be immediately obvious. That's why I don't recommend buying used turntables, unless you know the owner and he or she can verify its condition.
All turntables have a base, platter, bearing, and motor, and most budget turntables also have a tonearm. Turntables and phono cartridges are mechanical devices designed to convert the tiny wiggles of a record groove into sound, so turntables can be extremely sensitive to vibrations in your room. Springy wood floors and rickety furniture supporting a turntable won't yield the best sound. As for the turntable itself, solid construction is important, flimsy plastic turntables should be avoided. The better turntables sound better because they're "quieter," and add less of their own sound from their motors, bearings, and are less susceptible to "feeding back" the sound of the music that's playing in the room. The difference in sound quality between cheap, plastic turntables and higher-quality ones can be dramatic.
Direct-drive "DJ-style" turntables are less finicky about where they're placed than belt-drive turntables, but belt-drive designs tend to sound better. Almost all high-end turntables are belt-drive designs. It takes a long time, five to 10 or more years, for a belt to wear out.
The tonearm holds the phono cartridge that plays the record. The cartridge has a tiny diamond stylus that traces the groove, which contains the actual analog waveforms of the music that will be reproduced by your speakers or headphones. Most turntables have a cuing device for lifting and lowering the cartridge stylus on and off the record.
There's a counterweight at the back end of the tonearm that is used to adjust the tracking force of the cartridge. Phono cartridges require a specific amount of downward force, typically 1 to 2 grams, to work properly. The amount of downward force varies from one cartridge to the next; the tonearm counterweight is used to adjust this downward force. The tracking force needs to be adjusted by you, and the method also varies from one turntable to the next; check the instructions on how to do this. Some turntables also have an "antiskate" adjustment; again, consult the owner's guide on how to properly use that function.
One other setup tip: your turntable will work better if it's perfectly flat or level; use a bubble level you can buy at a hardware store to make certain your turntable isn't tilted at an angle. A sturdy stand, table, or furniture is preferable to wobbly ones.
The turntables I'm describing here are pure analog models, not USB turntables you hook up to a computer. If you already own an AV receiver with a phono input, just connect the turntable's left and right channel cables to the inputs. Some turntables also have a third ground wire; if yours has the extra wire, it should be connected to the ground connector, usually a small threaded knob on the receiver's rear panel near the phono connectors. If the receiver lacks the ground connector, just attach the turntable's ground wire to any chassis screw on the receiver's back panel. Some inexpensive turntables have built-in preamps and don't need separate preamps; that's nice, but those sorts of turntables won't sound as good as the better turntables I refer to later in this blog.
If you don't have a receiver with a phono input, or want to buy one of the better turntables, you'll need to buy a separate phono preamplifier. Prices start under $20, but I'd recommend investing $100 or more for a really decent phono preamp to get great sound.
If you're searching for a great turntable that won't break the bank, the Pro-Ject Debut Carbon ($399) is the one to buy. It looks great in gloss black, red, green, blue, yellow, light gray, or white, and its tonearm uses a carbon tube. I've had good experiences with Pro-Jects in the past, but Michael Fremer, a bona fide turntable guru, recently raved about the Carbon. He summed up his review with this remark: "I don't know what else comes close for $399, especially now that Pro-Ject includes the $99 Ortofon 2M red [phono cartridge]."