I'm using a tiny pipe (1.5 up, 4 down) on AT&T DSL. While I can being gaming (WoW) and downloading from download.com (a Cnet site), a friend can be downloading a very large file from my server, and the wife can be watching Netflix, all at the same time, normally without interruption. We did this as a test, it worked flawlessly and I was still downloading.
We currently have two servers, two desktops, two laptops, two phones, TV, Tivo, printer, etc, one 24port 100M switch, and two Cisco routers (because I can) running it all through a simple DSL gateway, it's not the Internet speed that's causing the problem. We can sit in the living room, watch something on Netflix, and it will go from 1080p (the 5 bar display) down to the 3 bar display.
It's not your nor my network bandwidth, it's the Netflix servers.