The history, as shown in the browser, is just a small .xml file. I would leave that.
Each update however makes (1) an uninstall folder in c:\windows (shown in blue with me) and (2) a log file in c:\windows (like KB917615.log). Both can safely be deleted, if you don't want to go back anymore.
Hope this helps.
Is it OK to delete the old Windows updates that have accumulated over long period? I'm thinking of making my system leaner and faster. Are they critical to the security of my PC? Appreciate feedback .