My HT system actually shows AC on the display but no sound come out. So yes it is receiving something that it cannot decode.
But it looks like there are some solutions:
How to play an AAC-5.1 over SPDIF - instructions how to convert a file to AC3
Dvd audio is AC3-5.1 and external digital receiver plays it with no problem via SPDIF.
Modern videoes 1080p, 720p, MP4, mov, Matroska (mkv) often use audio compressed in AAC-5.1, this is good because AAC is more efficient than AC3, but that is bad for my receiver that, as commonly for a receiver, it supports only the AC3 format.
http://ac3filter.net/ - a FREE plugin that will allow the WDTV to process AAC and convert to AC3
AC3Filter is a high quality free audio decoder and processor filter. It allows media players to playback movies with AC3 and DTS audio tracks. Also it can process any audio track, much of processing options allows to adjust the sound in almost any way. It is possible to upmix any audio source to 6 channels. Filter can do multi-channel and digital (SPDIF) output. It can encode any audio source to AC3 on-the-fly and send it over SPDIF to the receiver.
BTW Vote for it HERE