The AVG Uninstaller wouldn't work for me. It wanted me to be logged in as an Administrator, and yet when I did so, it also failed.
I was able to use the command line (as an administrator) to delete the AVG files in the c:\Program Files (x86) directory. The rmdir /s command was very handy, as it deletes folders and their contents. There was one file that Windows wouldn't let me delete, but I was able to rename it.
That seemed to do the trick. You also have to remove the toolbar from your web browsers, but you can do that with their normal settings tools. I also went back to the Control Panel's uninstall program list and tried to run the AVG uninstaller again. Control Panel said it couldn't find the file and would I like to remove AVG from the list. Oh yeah.
I also used Internet Options, Security settings, to make http://*.avg.com a restricted site. Couldn't hurt.