The battery backup should have solved your issue. I assume you can unplug the battery from the wall for a short time and you do not go down. Make sure there is nothing else in the path. Sometime depending on how exactly you get your internet there is a powered device where the internet enters the building, most the time these have their own battery but that could be bad.
A better way to see if your internet is up or not is to run constant ping commands in the back ground. Open 2 command prompt windows and on one ping a well known ip like 4.2.2.2. On the other ping the gateway your router uses to talk to the internet. (it will be the second hop in most trace routes). Now minimize the windows. When you think you have a issue you can open these and see if you have loss.
Building wiring fault is not a good thing. I would not recommend you mess with it but it sound like you have already been putting things in the outlet to test. Generally what this means is the UPS is seeing voltage on the neutral. If the hot and neutral were wired backwards to the outlet it will do this. It could also mean it is seen a large difference in the voltage between the neutral and the ground. Since the neutral and the ground are both connected to ground someplace (where depends on building codes) there should not be a voltage difference.
They make small plug in devices that will test the outlets much more safely than a multimeter.
Here's the problem I'm having: Every time I'm on my computer playing any online games or even watching hulu for example, there are times where my internet will suddenly cut out. When I'm gaming there are times where I can watch what's happening, but am unable to move or chat or do anything like that, so it seems like it's not a full disconnect. The odd thing is that most of the times that this occurs the air conditioning is on or comes on a few moments later. I know that it could be a power drop due to the wiring in the old house I'm staying in, and to try and get around that I hooked up both my modem and my router to a battery backup in the hopes it would provide it a more steady power supply.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic