I admit I'm not into podcasts, but it seems that if you're already using iTunes to retrieve your material, the easiest solution is to get an iPod, if circumventing additional steps is what you're after. The nano handles podcasts, and I believe allows for pause/placeholding when turned off (though you might want to confirm that with someone else who's more knowledgeable about the device). As for budgeting, 8GB versions are now around $125 online (Amazon).

As for 'extra features'...unless you're going after a bare bones, generic-branded device, it seems that most name brand units are video friendly these days. The days of audio-only, at least among most of the big guns in the U.S. market, seems to be a thing of the past.

If you're REALLY in a pinch for budget, Sandisk makes a couple of very capable devices. Both the Sansa Clip and Fuze in all of their flavors retail for under $90, and are usually discounted below their list pricing, particularly online. Neither will make your podcast gathering easier if you continue to use iTunes for your foraging, but both claim to be podcast capable though what EXACTLY that means is something of a mystery to me.