I love records and turntables. I dig the sound, and at its best, analog music is more fun, engaging, joyful, and groovy than digital (MP3, PCM, WAV, FLAC, etc.) ever is. It's not even close.
I say let analog be analog, but if you need to convert analog grooves to digital zeros and ones for convenience sake, sure, why not? But you're cheating yourself out of most of the experience. USB turntables are generally pretty lousy turntables, and I question how many vinyl newbies listen to these things and wonder what all the fuss is about. The cheap USB turntable isn't really about sound; it's about manufacturers cashing in. Yes, USB turntables deliver and you'll hear your LPs' music on your computer, iPod, or Zune. Mission accomplished, just don't kid yourself into thinking you're hearing records. You're getting a pale imitation of what's actually hiding in the grooves.
You can buy a great new (non-USB) turntable for around $400, and if that's out of reach, look for a decent used turntable. Shouldn't be all that hard to find something on eBay for less than $200, or the same price as one of those iffy USB contraptions.