Could be a lot of things, but I would put money on the biggest factor being communication overhead. I would guess that the 23MB folder that took a long time had a lot of small files in it. That will have a huge negative impact on transfer speeds, while a single large file will transfer rather quickly. I can't come up with a good analogy to explain this, so maybe someone else can, otherwise go research network overhead if you really want to know the why.
I just transferred a 369 MB folder from my HP Vista machine to a 1GB flash drive in about 15 seconds, another 41 MB file in about 10 seconds, and a 23 MB folder took 5 minutes. I did them in this order. Any idea why this would be?