For *outdoor/day* family and sports photos (and wildlife), the 70-300 might work. Although, I think it would be a bit long for family photos, but then that's where you could switch to your 18-135mm.

But for *indoor/night* family and sports photos, you'd probably need/want something faster/brighter (with an f-number of 2.8 or lower). For indoor/night sports, the 70-200mm f2.8 is a popular (and pricey) choice. Fortunately, for sports, you'll probably be using a fast enough shutter speed that you won't need to pay even more for the IS version. For indoor/night family photos, the 24-70mm f2.8 is again a popular (and pricey) choice, but if you can live without a zoom, there are several prime lenses that are both cheaper and faster/brighter. e.g. 35mm f2.0, 50mm f1.4, etc.