A little from column A and a little from column B. In particular it sounds like you're using an all wireless setup, which is not a good idea for more than 1-2 people. While you do have more and more MIMO routers coming out lately, there's a great deal of older single antenna (i.e. half-duplex) routers out there which will really eat into your bandwidth. Essentially every time you connect another device, the amount of bandwidth gets split an equal number of ways. Two devices: 50/50 split. Three devices: 33/33/33. Four: 25/25/25/25.

The issue isn't your DSL connection, the issue is that you're likely using an older el chepo router with an all wireless setup, and/or some of your mobile devices are sucking up a large amount of bandwidth running some kind of background sync operation. Some cell phones will only update installed apps when connected to wifi, because as much as the rates DSL and cable companies charge for going over bandwidth caps are highway robbery, the amount cellular carriers charge is like handing over your first born, and a couple of limbs... As a down payment. Apple saw fit to hard code an artificial limitation into iOS, so that any app of sufficient size will not download unless you're on wifi.

The quick and easy solution, is to make sure you're always connected via an ethernet cable. That will help significantly. Even better is connecting everything possible via ethernet cable. Leave your tablets and cell phones to connect to wireless alone. Basically, you need to give yourself a crash course in some of the basics of networking.