Personally, I don't use an app, but the website of the bank, so there is no installing at all. For apps, in depends on the app if you can install it on an external drive or USB-stick, or not. You can only tell if you try.
But, even if it works, I doubt if it would help against identity theft and hacking.
Can I uninstall all apps that i use for monetary transactions, reload them on an external hard drive and unplug when it when not in use? would this help protect me from identity theft or hacking?