It's nothing to worry about, assuming you've scanner your computer with the appropriate tools and you keep it secure.
Hope this helps.
Grif
I have 1 PC connected to my DSL modem, connected to the "phone" line
On the DSL Modem, there is a:
power light
Ethernet light
internetlight
activity light
Sometimes I will notice the activity light(green) blinking a little, even though nobody is on the computer. Now the computer is always on and internet connected. When the activity light is flashing a little, I have checked the security logs and see that no connections were made to the internet, nor were there any programs initiated. Firewalls do not show any activity either
So question is, should it be concerning that the activity light is being active on the DSL modem when there is no reason for it?
Someone once told me that if your machine is infected, you will see the activity light on the modem being active.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic