Audio sync tests with different audio codecs and formats

*edit* link to file with a few more tests, including x264 and DTS

http://www.megaupload.com/?d=OP9JQSWD

Since I also experience some lip sync problems on many videos I decided to encode a variety of test files to see just how good/bad it is, from an audio perspective. In other words, I kept the video stream simple and constant, so that any differences are only due to the audio formats. Also these tests have nothing to do with lip sync problems like the ones related to FFW/RW etc… , these are constant throughout any video file encoded with these settings, and independent of FFW/resuming etc… the playback. I used  the current firmware for my testing  (1.01.024), it would be interesting to see if other ppl with this firmware could check these results, and also if it gets any better or not in the prerelease version.

I made a simple xvid video stream, with PAL resolution @25fps, showing 5 screen flashes in 2s intervals, and synchronized it to 5 pulses of audio encoded in a variety of audio codecs (MP3 2.0, AC3 2.0, AC3 5.1, AAC 2.0) and muxed it in different containers (AVI, MKV, MP4). The purpose of the tests was to see if with each file I could tell if either the audio pulses or the screen flashes came before the other; if you can you would most definitely see lip sync problems with a video encoded with a similar audio track. What I didn’t expect to see as well was the behavior of the WDTV with some combos. Anyhow, these were the results I got, and I think it would be interesting if other people could download these files and see if they get similar results, either with the current or the prerelease firmware (it’s just a 1.15 MB rar file).

All the files play perfectly in sync on a PC with different sound players.

MP3 2.0 (all OK)


In AVI/MKV/MP4: good sync

AC3 2.0 (good sync, unexpected decoding)


In AVI/MKV/MP4: good sync

(all of these were decoded in the WDTV to PCM and sent to the receiver as such instead of being bitstreamed as AC3, even though Digital is selected in the sound options)

AC3 5.1 (bad sync or unexpected decoding)


In AVI/MKV: (bad sync, sound has about 80/100ms negative delay with video, though it was bitstreamed as AC3)

In MP4: (good sync, but now it was sent as PCM again)

AAC 2.0


In MKV/MP4: hard to tell, but it seems like there is a small positive delay

DTS


I don’t think it is supported in either AVI/MP4 except by cheating, so I just tested it in MKV, and it seemed fine.

So… basically it seems like a mixed bag, with different problems for different types of files. MP3 seems to be the safest -though lowest quality- bet, however it gets hairy when you play stuff encoded with AC3, and possibly with AAC as well, though to a lower degree. You either get a very noticeable lip sync problem with 5.1 AC3 tracks (negative audio delay, which is more objectionable than positive one), or no lip sync problems with stereo tracks, but those are never bitstreamed, just sent as PCM. And if you mux AC3 in a MP4 container it seems that even surround tracks get decoded/downmixed, so it’s even worse than with MKV.

Downloaded your RAR and tried this out with the 1.02.14 prerelease.

Connected to 1080p plasma via HDMI (audio muted).

Optical cable to 5.1 receiver; Digital output enabled in settings.

No appreciable delay or sync issue with any file.  Connected via network share (Vista x64 PC) and media server (TVersity).

What might be an interesting second test would be to use x264 encoding on the video, as most complaints seem to revolve around x264 / AAC.  Personally, that combination works fine on my setup too (although I tend to go for AC3 passthru when possible).  Could just be that the prerelease has better audio sync?

Could be, I certainly hope so. However when I say there’s a noticeable delay (up to -100ms) on the AC3 5.1 tracks muxed in either AVI or MKV, it’s not big enough to notice it unless you are looking for it. I think I’ll make a couple more versions of the AC3 5.1 track with some +/- audio delay applied to them, to have something to compare to.

And yeah I’ll make a version with x264, not sure how it will wind up though, Xvid is pretty easy to check for frame accuracy, x264 seems to be a different story.

Btw, did you notice if the 2.0 AC3 tracks were being decoded on the new firmware, or were they correctly passed through? In the current firmware it clearly makes the difference when it decodes AC3 2.0 tracks (it displays just AC3 on screen, and PCM was clearly shown on my receiver) as opposed to 5.1 tracks, which displays AC3 (DIGITAL) on screen and the DD logo on the receiver.

2.0 AC3 tracks get decoded, not passed through.  

My receiver lights up little speaker icons for each channel when you get a 5.1 signal and just 2 icons for stereo but I get no icons from AC3 2.0 from the Live.

Same applies to 5.1 AC3 in an MP4 or M4V container, as you previously mentioned; they’re decoded and not passed through but multi-channel audio is relatively new to the MP4 spec.  This might get fixed in a future firmware.

I’ve reencoded the video stream with x264 for a few more tests, with a fairly typical profile. It doesn’t seem to change things compared to the Xvid version, my guess is that it would take more exotic parameters and much longer durations to be an influence, and in that case it would be sort of a random occurance, not a constant delay like I found with AC3 surround tracks.

I also included a couple more files with DTS sound, which seems to have much better lip sync than AC3, at least on my box. I also made a subfolder just for AC3 5.1, with different delays applied on purpose. For me the ones that sound best synced are +100ms and possibly +50ms, so I guess the real delay is somewhere in between. Unfortunately having to apply positive delay to compensate means that the audio is early, which is a lot more noticeable than the other way around, even for <100ms.

I also made a file called “variable delay.mkv” that goes from -150ms to +150ms in a single file,  which might make it easier to check.

http://www.megaupload.com/?d=OP9JQSWD

what about AAC 5.1 audio tracks in different containers…