I have about 1.3TB of video files on a 4TB Seagate external hard drive. I would like to transfer them to my new WD MyCloud EX4. I have created these files by copying purchased DVDs using DVDfab. Each disc copies as its own discrete directory, which then contains two sub directories, AUDIO_TS (empty) and VIDEO_TS (which contains lots of files.) I have plugged the Seagate drive via what purports to be a USB 3.0 link into the back of the EX4.
Is there any way I can set up a backup job which contains lots of individual files, but not a whole directory´s worth? I am loathe to try and backup the entire master directory of files since I am sure that a) it will take weeks, and b) it will fall over part way anyway.
I have a WD TV Live which I have used with the Seagate drive to play videos on my TV. If the Seagate Drive is plugged directly into the WD TV then we can just click on the name of the disk and it will start playing. But playing it through the EX4 (where both the EX4 and the WD TV live are plugged via Ethernet cables into the router) I have to go to thje first individual video file in the VIDEO_TS folder to get anything to play. And then I have to manually start the next one, Useless!
I can´t get subtitles to work. Which, as I´m a fan of Euro’crime series without being much of a linguist, is rather unfortunate!
I´m sure these are all things I can overcome, but i do need help!
Sure, there are many programs that will do exactly that…
The first thing I would do is to try to do a USB backup job, it will look like it is not doing anything for a while due to file validation. Once it gets going it should be done in a day or so.
If for whatever reason that does not work, check out Synctoy from Microsoft
The best part about Richcopy or Synctoy is you can indeed copy an entire folder or tree structure and if it fails, you can simply start over again and it will automatically skip files already there (via settings)
Sure, there are many programs that will do exactly that…
The first thing I would do is to try to do a USB backup job, it will look like it is not doing anything for a while due to file validation. Once it gets going it should be done in a day or so.
If for whatever reason that does not work, check out Synctoy from Microsoft
The best part about Richcopy or Synctoy is you can indeed copy an entire folder or tree structure and if it fails, you can simply start over again and it will automatically skip files already there (via settings)
Okay, so how does one go about installing Synctoy and RichCopy, that runs on Windows, to run on Linux running on the EX4??? This EX4 user’s USB drive is hooked directly to the EX4 - if he wanted to backup an USB drive connected to his Windows, he might as well have done it manually by dragging and dropping. Maybe I am missing something
The USB backup job directly from the device under backups / USB backups should be the fastest but I don’t know for sure because I haven’t benchmarked it.
Synctoy and Richcopy do run on a windows PC. The advantage is the resume file copy system so when your network hiccups you don’t have to start over from the beginning again.
As most Windows users know, if you start a large copy process (1TB+ or 1000’s of files) the chances of it failing are exponentially related to the time it takes to copy them. The best straight windows copy procedure I ever had was a mere 4 hours. Can you imagine trying to backup 4TB having to start over every 4 hours, it would take days and days.
The best copy performance I have seen is the USB drive plugged into my Gbit laptop connected to a Gbit switch that is also connected to the EX Device. I copies as close to max USB speed as I believe is possible.
Cven better speeds were obtained by using richcopy on a PC connected to a Gbit switch that connected to both the EX4 and MyCloud 3TB.
What this says is that after 23+ hours it copied 218k of 224k files = 845 GB averaging 10MBps which is 80Mbps which does not seem very fast untill you compare it with normal copy speeds of about 30Mbps sustained. Most these files are very small (1-2 KB) so they eat up time slices like crazy.
Can only share what not to do. (and then mainly for other people who come looking for suggestions :S)
In trying to acheive the same thing I thought: plug the drives into the USB ports on the NAS, ssh in and do a straight:
mv /mnt/USB/[drive]/ /mnt/HD/Drive/mynewshare/
being USB 3 it should be about the quickest option, right? I had 4 or 5 tb’s of data to copy. I would have thought 10, maybe 15 hours. Boy! How wrong was I?! I’m at around 36 hours and counting.
Noticed the processor was very busy in the console. So then running top led me to research wdphotomerger and I found this:
Which did speed things up a bit but not that much. CPU is still sitting on 100% and spiking down to 90% every couple of seconds. Its like its in opposite world. At least its just what you’d expect hogging the CPU (the mv command and ntfs and raid drivers).
And annoyingly, mv actually failed to remove the source on random files for some reason. It seems have copied ok but left the odd file behind. Can’t see any reason for it. I’ve tried to give it as free a run as possible. nothing else has a grab on those files. Possibly its crappy ntfs drivers.
I don’t know what speed those onboard usb 3 ports are supposed to achieve, but it doesn’t seem to perform anywhere near as well as the same drives plugged directly into my intel nuc (running xubuntu).
If I had my time again, I’d probably try plugging them into a reliable and known USB 3 (in my case, the nuc or my work laptop) and copying over the network. Maybe I’d try scp or rsync or maybe even ftp to do the heavy lifting. I don’t know that I could trust the onboard usb. Don’t think its up to the task.
