Maybe you could connect via optical or analog audio cable to avoid sending over the video?
How can I use my TV as speakers for my PC? I'm using an HDMI cable to connect my PC to my TV but I can't find any other way to listen to my PC through the TV without duplicating the image or using it as a projector. I just want to have video on the PC and only sound on the TV. How do I do it?