There are two answers to your question, all depending on what route a potential hacker might take.
The most common answer is no, they can't use your IP address, because an IP address MUST be unique to each computer. They can spoof, or fake, things to make it appear as though it came from your IP, but that's really not the same thing.
There is one way in which it's possible for a hacker to use your IP address however, and that's if they managed to slip some malware or a trojan on your system that sets up a proxy server. This then causes your system to get web pages, or whatever else, and forward it on to the hacker. To every site they visit, the logs will have your IP address, not theirs.
However, unless you're on a very high bandwidth university internet connection, or something similar, it's unlikely anyone would want to bother with the latter option. It's unlikely a hacker would take any interest in you at all to be honest. Hackers usually go after high profile targets, like say IBM or Microsoft, not random individual people. They are usually more interested in hacking into a website, and changing the pages to take advantage of IE security holes to install malware on people's systems, or to set up a system to capture sensitive information entered into a site to use/sell for identity theft and other types of fraud.
The days of the stereotypical hacker are pretty much over. Now it's not about trying to cause damage or chaos, it's a business. Social security numbers can sell for big money on the black market, just like credit card numbers, and other sensitive information. This is why malware production is out pacing virus production by as much as four fold.
So be sure you've got all the latest security patches installed for all your programs. I would strongly recommend not using any Microsoft product that can connect to the internet. That includes Internet Explorer, Outlook Express, and even Microsoft Office. Office in particular is rapidly becoming a prime target just like Internet Explorer. You can replace all of these with alternatives, such as Mozilla Firefox, Mozilla Thunderbird, and OpenOffice. Mozilla Firefox is hoping for it's 2.0 release tomorrow I believe, and Thunderbird will probably follow within a week. OpenOffice has probably the best compatibility with MS Office of any other office suite, sometimes even better than Office itself.
Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic