NAS to NAS Transfer

Hi,

I need your advice on what is the best way to copy all files from a 3rd party NAS (D-Link DNS-320) to the WD MyCloud EX2.

I tried initiating the transfer using the remote backup function on the D-Link to copy files to the WD.  SSH connection was successful, however, Rsync test failed on the D-LInk.

So now I’m trying another method.  This time, I am using the built-in FTP download app (uses Linux wget command) on the WD MyCloud EX2.  The transfers are on the average 10MBps.  It’s quite slow don’t you think?  To transfer 4TB would take 4 days?

Both NAS are connected to a Cisco DPC3925 which supports gigabit ethenet.  All cable connections are using CAT6 cables.

Any help would be much appreciated.  Thank you.

I haven’t tried this method but I suggest you look into FXP, which is supported on the EX2. It’s basically a server-to-server direct ftp without going the circuitous route of going via your computer.

I would say if you’re getting 10 MBps read speed then that’s awfully slow…not sure though what’s the culprit here as there are too many variables…it could be the other NAS is slow, could be RAID config on your EX2 that I’ve heard slows writes down (I don’t use RAID on my EX2, I use JBOD so I don’t know for sure). Also, if the either of NAS has services like media sharing on, I’d turn that off during the duration of file transfer.

I remember seeing some instructions in the EX2’s manual about making a way to schedule backups between EX2 and another NAS…that process may or may not apply in your situation but you should check out the EX2’s manual for that info.

Copying 4TB of data will take a long time regardless but you should be getting better speed than what you are reporting. Not sure what’s the bottleneck here.

Of course, there is also the tedious option to copy the data off your NAS in chunks to your local computer first and the computer-to-EX2…you may not have 4TB space available on your computer hard drive…I wonder if that could speed up the copy process. Of course, this would be a manual 2-step process but I wonder if data transfer would speed up this way. You could see what speeds you get for a small but sizable chunk of data (like 10-20 gigs) and then extrapolate the numbers to 4TB to see if it would be better. BTW, I suggest you try this using ftp (i.e. ftp files over from NAS1 to PC and then ftp from PC to NAS2)…my suspicion is you will see better transfer speeds…of course it’s still a pain in the butt method since you’ll have to do it manually and perhaps even in batches but if you have a large external (or internal) hard drive then you can minimize the number of manual fetches and subsequent puts. I suspected this might give better transfer rates…and just now I saw this post which suggests the same method to yield a little faster rate →   http://community.wd.com/t5/WD-My-Cloud/Slow-Transfer-Speed-over-USB/m-p/687003#M8322

Thanks for the helpful tip.

Yes, I’ve been copying using the FTP method.  I grouped my video files into a few chunks of folders and created an FTP batch for each.  Speeds have improved to 25MBps, still slow though.  The EX2 somehow limits to 3 FTP transfers at a time, meaning each FTP transfer is roughly 8MBps.  I’m still finding tips on how to increase the number of FTP threads so that I can bump up the transfer speeds beyond the 25MBps that I’m seeing.  By the time I find the answers, the copy process might be completed by then.

The EX2 manual only provides instructions for backup to and from other WD products, which is consistent with what other manufacturers are providing for theirs.  To do transfers among NASes that are from different manufacturers seems a bit more tricky, which is why the methods to achieve such are usually found in the forums like what we have here.  Most recommend to use rsync, other use a combination of tar and piping it to ssh.  Did try those and the speeds are slower than the FTP method.  Thus, using the built-in FTP app was still the choice I made.

If you are using ftp, there is a file where the configurations for ftp is set…you can use the dashboard to save the configs but currently there is a bug in the dashboard UI of EX2 which prevents you from setting the external IP and therefore you cannot ftp from outside. I have reported this bug and WD has promised a fix in the next firmware release. Until then, you can manually update this config to change the settings via ssh.

So the dasboard saves those ftp settings in the file /etc/NAS_CFG/config.xml. But they get wiped out and copied over after reboot from /usr/local/config/config.xml - so it’s best to change things there (after backing up the original). It’s an xml file. You need to scroll near the very end but not at the end…just a little before the end, you will see the entries under the ftp xml tag. There you can tweak any ftp setting - but best to leave untouched the max number of clients, which by default is set to 10. Also you’ll see that max number of connections per IP is set to 5. So I am not sure where you saw that EX2 limits the number of ftp transfers to 3.

The only thing is I have not been able to have my friends ftp-ing from outside to make more than 2 simultaneous connections but I think that’s a function of my available bandwidth - because before I got the EX2, I used to use Filezilla ftp server and on that too I could not achieve more than two simultaneous connections…the rest of files were always queued by the ftp client.

The way I am doing NAS 2 NAS transfer is towards a Windows 8 PC copying directly from one place to another through samba.

I tryed using FXP towards FTP, as seemed to be the best option without requiring a proxy in between but I did not succeed.

Transfer rate is not bad, but depends directly on the size of the files being transferred, which is a normal case in any PC transfer, even within a local copy from files between folders in same HD.

genoso - So you said you are copying directly via Samba…but when you connect to the two NASes via samba, that is not a direct NAS to NAS connection…they are all getting routed via your PC, though you may be skipping the step of storing them on your PC…but the data is still flowing from NAS1 to Win 8 PC to NAS2.

it is NAS1 towards samba to PC RAM and from PC RAM to NAS2 towards samba… data is not stored in the PC.

I open network folder on NAS1 (file explorer window 1) and another one to NAS2… and just drag and drop.

Protocol used for transfering is SAMBA, and it is done using my PC as proxy because it pulls data from NAS1 and push it to NAS2… 

Simple :wink:

Obyously it is not NAS to NAS direct communication, that’s why I mention my PC is acting as a proxy.