Sometimes it is because a Windows Update hosed the network ability of the PC. Only an image restore and a repeat of the last Microsoft updates fixed the issue. I couldn't even connect through the modem, let alone the LAN.
Most of the time, in my experience, it is because of malware, on my client's PC . They have no real time malware blocking - which is almost impossible to find now(for limited accounts), so using CCleaner is the best substitute. Using No Script on FireFox can help block advertisements that may be playing while you watch the video, so it is handy to just allow the flash video using NoScript. All you have to do is double click in the video display area of the page, and NoScript will automatically allow that video. Running as a limited user account is a requirement to help prevent infection by the more serious malware. Regular scanning by MBAM and/or SAS can really help in this area. If you run as administrator, the real time protection of the paid versions of MBAM and/or SAS can be a real help there, but personally I can't recommend running that way all the time.
If you have at least a 3Mbs per second broadband connection, and the ISP isn't up to throttling; then you should be able to watch most videos with few interruptions, but it helps to be on DSL, because the connection to the gateway is dedicated, and you are less likely to be sharing bandwidth with other customers. Sometimes it is just the web site and how much load it is taking, or the location it is in the WAN that can effect routing, and slow a particular site down. This shouldn't be the case with YouTube however, as it is optimized for all factors in this regard.
Oddly enough I was having trouble with everything but video streaming on my PC, until I got calls from my clients about the same thing. The regular HTML pages were so slow as to be like dial up!!! I called my ISP and asked them if they were throttling HTML to give more bandwidth to the streamers, and they said they'd check. Sure enough everything sped up and all traffic was normal again. If your ISP made a mistake in configuration, they could be inadvertently slowing streaming traffic in the same way.
I'm not sure of the legal ramification of throttling if your EULA with the ISP doesn't cover this area, they may be in violation of FCC rules. I believe the supreme court allowed throttling from providers if they clearly state that this will occur in their contracts with customers.
If you've never had what you thought was acceptable streaming performance, it could also be the throughput of your modem or router, or both! One would have to check the specifications of the router/modem to make sure this is not a factor. If you cabling is cruddy and has a lot of crosstalk in it, this can definitely affect performance in this area. Be sure and use the best shielded Cat 5 patch cable you can find. The Ethernet card must be capable of fast connection too, but even old card technology supported 10Mb/per second, and that is God's plenty, so unless it has been comprimised by a power surge and reduced its bandwidth, that shouldn't be a factor.
Our ISP is going gigabyte speed soon, and I will have to purchase all new Ethernet adapters and an "N" router at that time, to take advantage of this service. Most modern cards, modems, and routers are rated for this now. We had to have PPPoE capable equipment for this to work properly, so that could also be a factor if your modem or router is miss-configured or not even capable of that kind of protocol or other ISP schema that could be prevalent.
I rarely find the firewall as the culprit unless the router is mis-configured, or the PC's software firewall is malfunctioning. IF you are behind a router/firewall, it is safe to turn off your software firewall(providing no other PCs are on the same LAN and turned on), and try your ISP speed test utility, every ISP I've worked with has one on their web site. I recommend using their speed test first, so you can see if it is truely a problem between you and the ISP gateway. IF that passes with flying colors try some of the online speed test located in different areas if the world to see if their is a bottle neck near the areas you like to frequent. Be sure and turn your software firewall back on afterward and compare the speed test for both possibilities.
This is rare, but occasionally I find clients with really old smoldy media players in their PC, they never updated media player, Adobe Shockwave, or Adobe flash, or they have a really old version of Chrome with its older embedded flash version. So a person can always run Secunia PSI or File Hippo's Update Checker to find out what is running that is out of date in that area. Some folks like CNET's email update reminders, I like to use all three myself, so in case one misses something the other one covers it. Most of the time this won't affect performance, but malware can take over an older version of any of these, and make life miserable, so it is still good advice.