Handbrake + mkvmerge

Hello folks.  I’ve got a couple of questions I’m hoping some bright mind might help me out with.

  1. I converted one of my favorite TV series to mkv with Handbrake’s High Profile using AC3 passthru for sound.  The problem is, out of 24 episodes, a few of them have the video running at almost 2x the speed thereby de-synchronizing the video with the sound.  To fix this I ran the files through the AVI-Mux GUI program.  It fixes the files quite nicely thereby de-muxing, re-muxing the files with the correct synchronization.  My question (Part A): Why would Handbrake do this for some episodes from the same disk but not the others?  (Part B): I read that running the files through mkvmerge 4.0 will do the same thing.  Why perform this second step if Handbrake is supposed to create compatible files anyway for the WD TV Live?

  2. Along the lines of the first question, I’ve embedded the original subtitles into the mkv.  The good thing about it is, the subtitles are synchronized properly, but for some reason, some of the subtitles appear perfectly solid while others only appear as black outlines.  Is there anyway of fixing these with either AVI-Mux GUI or mkvmerge?

Thanks in advance for your help!

What version of Handbrake are you using?

It kind of sounds like some of the episodes are using Variable FrameRate, and earlier versions of HB had issues with that.

As to the trasnparent subtitles, I believe that’s a problem with the WDTV, not Handbrake.  Other people have reported the same thing.

Here’s the mediainfo:

General
Complete name                    : C:\Work\Season 1\S01E01.mkv
Format                           : Matroska
File size                        : 167 MiB
Duration                         : 22mn 29s
Overall bit rate                 : 1 041 Kbps
Writing application              : HandBrake 0.9.4

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L3.0
Format settings, CABAC           : Yes
Format settings, ReFrames        : 3 frames
Muxing mode                      : Container profile=Unknown@3.0
Codec ID                         : V_MPEG4/ISO/AVC
Duration                         : 22mn 27s
Bit rate                         : 798 Kbps
Width                            : 720 pixels
Height                           : 480 pixels
Display aspect ratio             : 16/9
Frame rate                       : 29.970 fps
Standard                         : NTSC
Resolution                       : 24 bits
Colorimetry                      : 4:2:0
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.077
Writing library                  : x264 core 79
Encoding settings                : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=1 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=3 / b_pyramid=0 / b_adapt=2 / b_bias=0 / direct=1 / wpredb=1 / wpredp=2 / keyint=300 / keyint_min=30 / scenecut=40 / rc_lookahead=50 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00

Audio
ID                               : 2
Format                           : AC-3
Format/Info                      : Audio Coding 3
Codec ID                         : A_AC3
Duration                         : 22mn 29s
Bit rate mode                    : Constant
Bit rate                         : 192 Kbps
Channel(s)                       : 2 channels
Channel positions                : L R
Sampling rate                    : 48.0 KHz
Language                         : English

Text
ID                               : 3
Format                           : VobSub
Codec ID                         : S_VOBSUB
Codec ID/Info                    : The same subtitle format used on DVDs
Language                         : English

Chapters
1                                : 00:00:00.000 - Chapter 1
2                                : 00:02:26.054 - Chapter 2
3                                : 00:07:35.188 - Chapter 3
4                                : 00:16:52.244 - Chapter 4
5                                : 00:21:52.377 - Chapter 5

HB 0.9.4 is ages old; unfortunately, it’s the latest “Official” release.

Fortunately, you can download the latest “Nightly” build HERE:   https://build.handbrake.fr/

Make sure you set the FrameRate to “Same as Source,”  and if THAT still doesn’t work, try set it to 23.97.

Thanks for the link, TonyPhi12345.  The page says this Handbrake version is unstable in bold: “This page contains nightly builds of some HandBrake Components. These are not stable releases. They are unstable (and undocumented) code which will one day lead to a new stable release of HandBrake.  They are only recommended for experienced users and developers.”

I’m a developer, myself.  I know what all that means.  Will it address my issues, by trying this software and testing it for the Handbrake dev team?  It seems I’ll be taking a chance with this “Nightly” software.

Have you had much luck with their “Nightly” software and episodic DVD conversions to mkv?  Will I need mkvmerge afterwards as well?

No, don’t be scared.  ALL of us are using nightly builds (or the SVN ones) and have done so for many months now.  They are as stable as anything.

Thanks, Mike!  What about mkvmerge?  I read in your past posts that you use the program on all your conversions after Handbrake’s completed.  What do you use it for?  What’s your workflow like?

mkelley wrote:

No, don’t be scared.  ALL of us are using nightly builds (or the SVN ones) and have done so for many months now.  They are as stable as anything.

Well Mike, not ALL of us… :wink:

As for me, I use StaxRIP to convert my DVDs and BluRays to h264 and mkv. I used Handbrake for a while (and yes, also the nightlys) but every now and then I run into some sync problems, be it audio or subtitles. Never figured out really why…

I then tested StaxRIP, and I never got any sync issues and it runs a little faster (at least on my x64 Windows 7 box) than handbrake. Although the Handbrake GUI is a bit more comfortable, while StaxRIP gives me more freedom regarding container and codec configuration,  I switched to StaxRIP. Oh, by the way, I use the latest beta of StaxRIP.

EDIT: Forgot to say, that StaxRIP is WIndows only. So, as a Mac user, I would take Handbrake…

Cheers,

Zap

Jackster wrote:

Thanks, Mike!  What about mkvmerge?  I read in your past posts that you use the program on all your conversions after Handbrake’s completed.  What do you use it for?  What’s your workflow like?

Using Handbrake I hardly ever run into any issues.  However, I noticed following:  WDTV live and + units have issues with B frames.  So, use High Profile settings and other recommendations in this thread, but set B frames under Advanced to Zero.  I can not make sense of the type of issues encountered, but setting B frames to Zero seems to resolve sync problems with me.

As for mkvmerge.  You can use this to edit your own chapters and many other things.  However, I believe you are refering to the issue of Compression that is independent of using Handbrake.  When header compression is not set to None, some MKV files will play without Audio.  If you encounter this on an mkv file that you have, use mkvmerge to remux mkv file but with compression set to None under Extra Options in the new build of mkvmerge.

Jackster wrote:

I’ve embedded the original subtitles into the mkv.  The good thing about it is, the subtitles are synchronized properly, but for some reason, some of the subtitles appear perfectly solid while others only appear as black outlines.  Is there anyway of fixing these with either AVI-Mux GUI or mkvmerge?

It’s a Live bug. Workarounds are  (1) keep the subs as external files (SUB/IDX, SRT or whatever format your subs are in), or (2) burn-in the subs when handbraking (it takes longer time, though).

 

Cocovanna

Jackster wrote:

Thanks, Mike!  What about mkvmerge?  I read in your past posts that you use the program on all your conversions after Handbrake’s completed.  What do you use it for?  What’s your workflow like?

I use it for both DVDs and blu-rays (mostly blu-rays).  I rip the DVD/Blu-ray to disk (using AnyDVD HD) and then point Handbrake at the main movie (with a DVD you point it at the VIDEO folder, with a blu-ray you point it at the largest M2TS file in general).  I use the High Profile preset but change the output type to MKV and I pass through the AC3 or DTS main audio track (but leave everything else alone – there is no issue with b-frames with the High Profile preset, at least in versions in the last few months).

If a DVD I can also put the subs in (although I’m not so worried about subs – while I’ve been doing all the subs at times I wonder if it’s worth any effort at all, since I almost NEVER need them).  With a blu-ray the process of getting the subs is a little more complicated and I won’t go into it here, but it’s not difficult.

The result is a picture that’s perfect – indistinguisable from the original disc and usually around 25% the size of the original.  In literally thousands of discs I’ve yet to have a single issue or flaw.

Sounds like a lot of steps to save some disk space.   Disd space being so cheap and plentiful with the large external drives, why not just copy the entire DVD with Imgburn to an ISO which will have every last byte of data on the original?  sure it takes more space but it’s only one space and Imgburn is very fast too.  The WD Medial Player Live Plus now plays ISOs so it’s very simple way to accomplish what you are trying to do.  A couple of mouse clicks and it’s a done deal.

A lot of steps?  About one – other than ripping it to the hard drive (you load it up in Handbrake and make the movie.  That’s one step).

And since blu-rays average around 30GB (and my typical encode averages 7GB) even with large amounts of disc space you can run out quite easily, particularly since you have to double it (because we always have to back up our efforts, right?).  I have around 1200 movies encoded, and would need about 12TB of disc space not even counting the backup.  How much disc space do YOU have?  (If you even have half that amount I’ll be very impressed).

So, I managed to fix the subtitles problem.  I extracted the subs via MKVExtract as idx/sub files.  Once the files were set as external, everything worked out great.  The subtitles now look exactly as they should from the DVD.

Now, as for the de-synchronization problem I’ve been having with Handbrake with some of the episodes from the same DVD disc, I haven’t figured out why, for instance, Episode 1 converts properly while Episode 2 converts the video faster.  So far, I’ve surrendered to the fact that I need a third-party app to fix this issue.  I don’t know, I’m a bit apprehensive to venture into beta software for Handbrake (but hey, that’s just me).  I will not discourage anyone else from doing so.

I’ll try the B-Frames idea, setting the parameter to 0, and see how it turns out.  But, I think, I shouldn’t have to do that to achieve perfect compatibility with the Live.  The Live should have that compatibility on its own.  By the same token, the Live is a fairly versitile unit that handles almost anything I’ve thrown at it.

Jackster, c’mon…  Take a leap of FAITH!   HB’s nightlies are pretty dang solid.   MORESO than 0.9.4.

Give it a try.   If it doesn’t work out, just go back to 0.9.4, what have you to lose but a few hours of time?

And I reiterate - do NOT worry about B-frames.  Just use the High Profile preset (but do use a later version than .094, even one of the SVNs.  I’ve been using it for five months now and it works rock solid).

Ha, ha!  You guys are hilarious.  You know what?  I’ll give it a try.  :)  I’ll join the republic against The Force.  I’ll cross over to the Dark Side.  But, I’ll do it in a controlled fashion!  I’ll run the “Nightly” under Virtual PC.  If I’m satisfied with the results, I’ll let you know via this thread.

Anyway, I have 2 discs left of the series I’m converting.  We shall see what the results are!  I’ll push through them tomorrow and report back here.

Something occurred to me - you say “series” so I’m assuming this is a TV show, right?

TV shows can be the hardest things to convert, due to a whole lot of things.  Frame rates can be tricky (the infamous ST-TNG is a case in point.  These discs are among the hardest of any DVD to convert because they contain a bewildering variety of frame rates within the SAME EPISODE).  For sure you want to use the very latest Handbrake to handle some of these things, but you may also need to get on the Handbrake forum and post some of your logs to see if they can help you with a proper encode.

One thing is certain – it won’t be a Live issue that stops you assuming you Handbrake with the High Profile and use MKV as your container.  This is the gold standard for Live playback and it just works.  If it doesn’t, we need to look at what’s going on with that particular encode.

1 Like

mkelley wrote:

…Frame rates can be tricky (the infamous ST-TNG is a case in point.  These discs are among the hardest of any DVD to convert because they contain a bewildering variety of frame rates within the SAME EPISODE).  

Bingo; that was the issue I was having with 0.9.4; ST-TNG issues.

MOST would play just fine; SOME would have normal audio, but Video that raced at top-speed.  

I discovered that HB was incorrectly marking the framerate of CERTAIN episodes to 29.97 fps, whereas they’re ALL supposed to be 23.97.   

The ST-TNG DVD producers did something odd; they mixed PROGRESSIVE content and INTERLACED content quite frequently.  MOST (but not ALL) of the CGI sequences are 30i, whereas the rest of the stuff is 24p.  I think HB was keying on some opening sequences that were 30i and setting the rest of the file that way.

I found this to be very interesting from a technical perspective, so I wrote a script to take a HB EncodeLog and put a SUBTITLE on all 30i content (and it goes away when it switches back to 30p).   

Even the opening credits are 30i;  you’ll see normal video, then a switch to 30i, and a name will pop up on the screen, then fade out, then the video switches back to 24p for a few seconds until the next name pops up.  Quite bizarre.  You’ll even see switches to 30i for a few frames during some camera cuts.

This is an example (I claim “Fair Use” under copyright law for educational purposes only.)

A segment in 30i (where a credit is onscreen), as indicated by the subtitle “VIDEO MODE”

…followed just a few seconds later by a shift back to 24p when the credit is gone:

If you see a bunch of lines in your EncodeLogs that say Film->Video and Video->Film, then you have the same thing going on;  Film is 23.97p, and Video is 29.97i, and those lines indicate they’re changing back and forth.

ANYWAY, I haven’t had that problem since getting away from 0.9.4; I converted my entire ST-TNG collection successfully to MKV using HB.

1 Like

It’s also the reason why there is very likely never going to be a blu-ray edition of ST-TNG.  They’ve admitted that by screwing with the frame rates (which they did because the FX shots were done by a different house at a different rate) there’s no way they can resolve them to hi-def without *completely* redoing them (read: $$$, WAY too much money for them to spend with little possibility of ever getting it back).

A shame, really.  ST-TNG was an outstanding show that just begs for an HD master like TOS.  Then again, it was never broadcast that way so perhaps it’s for the best.