You'll need to build housing (and fulfill other requirements) in order for helpful NPCs to arrive. An acceptable house needs a floor, ceiling, and walls on the sides, along with a back wall, a table, a chair, and some lighting.
As you dig down, you'll come across large caverns you can explore. When you find a vein of ore to mine, a small window zooms in on the action so you can select specific blocks to hit with your pickaxe.
Once you have mined the resources and chopped down the trees, you can build your base bigger to bring in more of the NPCs that sell valuable tools and other items.
There is only one dungeon on every map and you'd be wise not to set foot in there until very late in the game when you're strong enough to fight challenging enemies.