Some of the smaller sized version of Linux (Puppy, DSL) can be installed on a flash drive. I haven't heard anyone installing a version of Windows onto a flash drive, but I would guess it's possible. I'm sure people out there with far more knowledge than me would know of more advantages/disadvantages then I have listed below.
Booting onto a flash drive is that it is MUCH slower compared to a regular hard drive.
Not all motherboards are capable of booting to a flash drive. You would have check your BIOS to find out.
You are limited in your hard drive size.
You literally have everything in your pocket.
Here is something from Tom's Hardware that talks about install Windows XP to a flash drive.
Is it possible to use a flash drive instead of a hard drive in a desk top computer? What would be the advantages/disadvantages?