if it is the standard Magic Mouse, then go to System Preferences, Mouse and turn off Horizontal Scrolling.
Since installing OS X Lion, when I scroll through a webpage in Safari there is both vertical and horizontal scrolling. I don't want the horizontal scrolling to occur and I can't find a setting for this to turn it off. The pages I'm viewing aren't zoomed in, I can see the entire screen from left to right and there is no scroll bar along the bottom of the screen. Please help, this is very annoying.

Chowhound
Comic Vine
GameFAQs
GameSpot
Giant Bomb
TechRepublic