For the following, I'm assuming that your service is DSL. I don't live in the US so forgive me if my assumption is wrong.
What I need to know from you is if AT&T requires a username and password to connect to their service? In other words, were you given a username and password when you signed up that they told you to use when connecting to the Internet?
1. If you DO have a username/password...
A. When you had the modem hooked up directly to the laptop, did you have to connect to AT&T by logging in first using Windows XP's "network connections" (or similar for other OSes), like when connecting to the Internet with an old-fashion dial-up modem?
B. Did you have Internet access right away?
If situation A. is true, then hookup the modem to the router's WAN port and the router to your laptop. Next, you will need to access your router's setup page and look for something called "Internet Setup" or similar (I don't have your router so check your manual if you can't find it). You want to find a page that will let you select PPPOE and enter in your username and password. Save the configuration and now you should be able to access the Internet WITHOUT logging in with Windows XP.
If situation B. is true, then things will be more dicey since your modem is doing the login on your behalf. You will first need to find a way to access your MODEM's setup page and change the configuration from a "routed PPP" connection to a "bridged" connection (this is how my modem labels it). You will probably need to have the modem connected directly to your laptop in order to do so. Next, hookup the modem to the router WAN port and your router to your laptop. Follow the same steps for situation A above for programming your router with your login information. Basically what you've done is delegate the modem's task of logging-in to AT&T (technically creating a PPPOE session) to your router.
2. If you don't have a username/password...
Then hookup the modem to your router's WAN port and the router to your laptop. Access your router's setup page and look for something called "Internet Setup" or similar. Look for a selector that will let you choose DHCP. I don't think you'll need to enter any other info so go ahead and "Save changes". Hopefully that will work, if not then post back and we'll see what else we can do!