Seems like it should be possible to output any audio that comes in. Make sure you are not trying to bit stream anything that can;t be output via optical as well (DTS HD MA, Dolby True HD, etc)

If for some reason none of this works you should be able to connect your optical cable directly to the xbox 360 (the connector