Are you outputting your files from Handbrake as MP4 or MKV?
If MP4, try using an MKV container instead.
I did a little experiment with my setup after reading this thread. My old Pioneer DD5.1 amp’s display lights up to indicate when the digital signal has 5.1 channels, plus an additional dot beside the LFE channel icon flashes on whenever the bass signal is active.
With all my MKV files that have 5.1 AC3 audio (and my AVI files with 5.1 AC3), all the 5.1 icons light up and the LFE channel is present and working.
With an MP4 container with AC3, it does not register that there are 5.1 channels and seems to downmix to stereo.
I’ve even de-muxed a non-working MP4 and created an MKV using the exact same streams. The file then registered on the amp as having full 5.1 channels, LFE channel bumpin away nicely.
Another example: MP4 with 2 AAC audio tracks (Stereo and 5.1 Surround). Both tracks play through the amp in stereo.
When muxed into an MKV, the stereo track plays fine as expected but the surround track does not play at all, again as expected because it’s an old amp and can’t handle the outputted AAC surround signal!
It seems that the current WDTV Live firmware has multiple issues with the MP4 container, not just the audio sync issue. :|
-edit-
Using eac3to I was able to recode the AAC surround track to AC3 within an MKV, which then plays perfectly on my system (bass REALLY pounded during a random action scene, I had to turn it down). Here’s the link if you want to try it:
http://www.videohelp.com/tools/eac3to
It would be nice if the Live could output AAC as DD5.1 or PCM, most HT setups can’t handle AAC AFAIK? There’s an idea open in the Idea Exchange if you want to give it some support:
http://community.wdc.com/t5/WD-TV-Live-Ideas/Live-decoding-AAC-5-1-gt-PCM-5-1/idc-p/824#M165
But you’re absolutely right that the WD TV Live is a time vampire. :) Can’t just play a file, especially not an MP4, without remuxing, recoding or faffing around with it in some manner first. I have to plan ahead what film to watch so that I have a properly rigged MKV ready to play; my wife hasn’t the patience to wait for me to monkey around with a file when I realise it’s one I haven’t fixed yet.