When downloading a file, you store it in a folder. So when your browser asks where to store it, you point it to a folder on the external hard disk. And that's where you find the setup program then to run it. Some browsers store a copy in their cache first; in that case you should clear that also.
Most setup programs let you choose what folder you want to install the program to. But they can put shortcuts on your internal drive (which you have to delete then), they usually insert something in your registry (and that's on your internal drive) and they might install some libraries in some other folder on the internal hard disk.
And, of course, when running they can put a lot of data on your internal hard disk also. But that's the third step, after downloading and installing. And, luckily, you don't ask about that third step!
...hard drive without saving parts of the program on the computer's internal hard drive?