The first one (MKVtoolnix) you don't need to run for this... you just have to have it.
If you click on the second one to run it (after the "Publisher can't be verified" screen) you should see a screen like this. Click "Run Search":
It should pop open a file requester asking you where the files are that you want to check. Navigate to what folder has the movies you want to fix and click "Ok":
It may take a while to scan, depending on how many movies are being checked...
When the scan is done, you just need to click "Clean Files":
"Clean Files" is ghosted out for me, because all my files are ok. If I click the "Log" tab, it tells me "0 files need remuxing":
For you, when the "Clean Files" has done its magic, all the files in that folder will now play on your WDTV without locking it up.
But, after the previous lockups, I would reset the WDTV before trying to play anything on it.