You are not alone. This is a very common problem. It happened to me too. My 360 and my Oppo DVD player worked perfectly in 1080P. Just not my PS3. It would only work in 1080i. As soon as I switched to 1080P, "Mode Not Supported"
Unfortunately, no solid fix has been found for this. My fix was to get a Monster 1000HD HDMI cable. Believe it or not, that instantly fixed the problem.
The cable shouldn't make any difference right? It is a digital signal. But for whatever reason, in this case the cable did make a difference. I wouldn't have gotten the cable, but was able to get it on significant discount.
I know this doesn't help very much, but this is only fix I could find. I scoured the net also, and found many examples of folks having the same issue. There were no fixes though other than to get a high end HDMI cable.
The Monster 1000HD HDMI cables can be gotten on ebay for "relatively" cheap.
I have a Samsung 46A650 and just got a PS3, and am having a problem displaying 1080p. The TV mentions "mode not supported", or if it does work at first, it will cut out and the screen will go blank. I have an Xbox 360, which displays games in 1080p, and it has always worked fine. I tried switching cables, using different cables in different ports, and it doesn't seem to make any difference. HDMI port 4 (the side port) seems to work better, but was wondering if there was any other workaround?