MKV is a container type, like AVI. They are sort of a "wrapper" around which the various video and audio streams are inside. Those streams are encoded (and then decoded -- thus CoDecs) in various forms like DTS, AC3, H264.
The MKV container is relatively new ("new" being a difficult thing to define in this fast paced world -- in the last four or five years) but has become widely accepted as THE container type, replacing the cantakerous AVI container (AVI was so ill-defined that it often would not work in many situations). The Live *loves* MKV containers and it also offers advantages over many (it can hold DTS, sub titles, meta data of all sorts, etc.).
You can't rip directly to MKV (nor can you to AVI) but need to first rip and then process. But it shouldn't scare you -- Handbrake just points to the rip and using the High Profile preset (but changing the output type to MKV and passing through the DTS or AC3 audio) is all you need do. It will reduce your DVDs to about 25% of the size of them without changing quality one whit, and you can queue it up so that it will process many of them overnight (so you can rip 10 or so DVDs and have them converted to playable files by the morning). My typical encodes are around 700MB for a quality DVD movie (however, I usually encode blu-rays, which are considerably bigger).
There is a free trial of AnyDVD and, as I said, Handbrake is freeware, so you really aren't out anything to give it a shot. Many of us (most?) use this workflow -- rip with DVDFab or AnyDVD, and encode with Handbrake. Nothing plays better with the Live (which is why we call it the gold standard).