Horrible rsync performance on wd cloud vs My book live

alirz1 wrote:
So if the same file or folder changes e.g a word doc or folder gets updated that won’t get backed up with the safe point.?

Safepoint will update any changed files/folder on the remote share it copies to. However it will delete any files/folders ln the remote share to match the source. So if you delete deom the source, its reflected on the remote.

cool. So it sounds to me that works pretty much like rsync except it copies over the entire changed file rather than the delta?

Thats how im using rsync right now anyways. So im thinking as safepoint uses a samba transfer i could utilize the full transfer performance of the cloud drive rather than the slow transfer of the rsync.

alirz1 wrote:
So if the same file or folder changes e.g a word doc or folder gets updated that won’t get backed up with the safe point.?

Sorry, didn’t mean to confuse.   Yes, it will get re-backed up, but I don’t think it’ll do “block-level” changes.   If you rename a folder, I think it’ll back up the ENTIRE folder instead of just changing the name, and then delete the old folder tree at the other end.

Or, if you make a 1-byte change to a 10-gig file, the whole 10g will get sent, not just the change.

Thanks. I understood that from your previous post.

However im trying to do a test. Using safepoint for the first time.

the first drive find the other cloud drive on the network when i search for devices under safe point.

I have the same user/password on both drives.

If i try to specify a share on the destination where i would like to store the safe point, i enter the share username and password and click submit. It just pops a message saying updated and goes away. However i do not get the NEXT button available.

If i select the public share on the destination drive, i get the next button. I schedule for later and press finish and then i get a different error  " A bad request  was made (200400)"

EDIT: i get the same bad request error if try to setup the safe point the other way around i.e from disk 2 > disk 1

I had a similar issue when doing a safepoint to a network connected (smb) USB drive. What I did was create these folders on the remote share:

_WDPROT then under this folder WDSAFE then under this folder the name for the safepoint (I used MBL_BU). That worked for me.

Thanks for the info. I’ll try that. But you know we shouldnt have to do this.

So i had opened a support request with WD, concering rsync speeds.

After they asked me a whole bunch of stuff and logs from the drive…

The person suggested that i use regular file transfer method i.e in windows, copy and paste files between my two wd cloud drives  instead of using rsync…

I just wanted to post that here as i found that response hilarious. Seems like the person had no clue what rsync was.

alirz1 wrote:

So i had opened a support request with WD, concering rsync speeds.

After they asked me a whole bunch of stuff and logs from the drive…

The person suggested that i use regular file transfer method i.e in windows, copy and paste files between my two wd cloud drives  instead of using rsync…

 

I just wanted to post that here as i found that response hilarious. Seems like the person had no clue what rsync was.

The first level support in most tech companies is always hilarious.

I’m pretty sure they aren’t trained to do anything other than basic SSH stuff.  Remember, this wasn’t meant to be used for Rsync, its focused for consumers copying files via GUI.  If you wanted to do everything via linux, should’ve just built a cheap linux server.

WD My Cloud uses 650MHz dual core CPU. Why do you think that is 1.3 GHz?

I looked in the /proc/cpuinfo  file. I know BogoMIPS is not a true cpu speed but i think it denotes an approx cpu speed?

where did you get the 650MHZ speed from?

nas1:/# cat /proc/cpu
cpu/ cpuinfo
nas1:/# cat /proc/cpuinfo
Processor : ARMv7 Processor rev 1 (v7l)
processor : 0
BogoMIPS : 1292.69

processor : 1
BogoMIPS : 1292.69

putting aside voiding your warranty etc etc…

is there a verison of rsync that be insalled on the drive that would make sure of the dual cores?

Captain_Panaka wrote:


alirz1 wrote:

putting aside voiding your warranty etc etc…

 

is there a version of rsync that be installed on the drive that would make sure of the dual cores?


The horribly low transfer speed seen when using rsync in modes that are perfect for “normal” systems are not due to the version of rsync but due to some very unusual configurations in MyCloud.

 

It looks like MyCloud is using rsync for syncing MyCloud boxes, but I have not yet found out how this really works.

 

When I rsync from my home server using the “normal” methods I get transfer speeds of 4-5 MB/s.

 

When in the home server I mount a samba share of MyCloud and do rsync to that share I get acceptable speeds above 50 MB/s.

 

Have yet to find out why this happens…

 

Well in my case, i’ve setup rsync manually between two mycloud devices connected on the same switch on a gigabit network.

Speed is not a problem when tranfering files to and from the drive lets from a windows PC using samba shares.

one of drive runs the rsync daemon. The other drives syncs to it. I get about 15MB/Sec transfer speed. Now some might say thats fast enough. I think it should be faster. Given that my older mybook live drives synced in the same way and i used to get 24MB/sec

alirz1 wrote:

putting aside voiding your warranty etc etc…

 

is there a verison of rsync that be insalled on the drive that would make sure of the dual cores?

I don’t think there is.    I think rsync is inherently single-threaded.  Seems the only way to do it is to start two instances on different ports.

I’ve been googling for the last 15 minutes and not seeing anything other than bizarre workarounds.

Captain_Panaka wrote:

alirz1, if you use rsync with it’s own protocol for transmission then on the MyBook NAS it works as expected at about 20 MB/s.

 

However, if you do the same thing on a MyCloud you get such slow speeds like 4-8 MB/s.

 

 

 

Actually, i get 15MB /sec between two mycloud devices. I’ve been playing with it more today and have been trying different values for “–sockopts” switch for rsync and now i can sustain 19 MB/sec…

i had tried mounting the share from one to the other drive, though NFS, but that was giving me the same speed i.e 15MB.

Captain_Panaka wrote:  I wonder why two users report up to 80 MB/s. Imho these either do faulty measurements or barefacedly lie…

You are comparing apples and oranges.   This discussion is specific to RSYNC, not normal SMB file transfers.  

Yes, I easilly get 80MB/s reads from the cloud.   RSYNC performance for me is the same as what others are describing.

Captain_Panaka wrote:Sorry but you are wrong again.

Wrong “again?”  Sir, I’ve never been wrong in the first place.  :laughing:

I’m sure the folks in this thread will thank you to not hijack this discussion about RSYNC.

Captain_Panaka wrote:

To what machine do you copy, what os is running on that machine at what program do you start which shows that speed.

Here I am using Windows Powershell.  It doesn’t change any copy speeds… it just allows detailed timing of commands.

PS C:\Windows\system32>net use v:
Local name V:
Remote name \\cloudnas\movies1
Resource type Disk
Status OK
# Opens 1
# Connections 1


PS V:\> measure-command {copy '.\War of the Worlds.m4v' c:\temp}

[snip]
TotalSeconds : 23.9051605
TotalMilliseconds : 23905.1605

PS V:\> dir '.\War of the Worlds.m4v'
[snip]
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 9/8/2013 8:03 AM 2128426812 War of the Worlds.m4v

Just using the COPY command on an older Gateway DX4831 (Intel Core i3 with 6GB RAM) running WIndows 7 SP1 x64 from the Cloud NAS to my internal disk across a Gig Ethernet network that’s comprised of two ethernet switches (one switch has all of the NAS boxes, and the other switch has all my PCs.)

Simple math:

2,128,426,812 bytes divided by 23.9 seconds = 89,055,515 bytes per second.   Wow.  Almost 90 MB/sec.  Even better!  :laughing:

Captain_Panaka wrote:

Copying a file of 2GB to a machine with 6GB RAM will be a pretty special situation which I can not reproduce due to lack of RAM. Nor do I have powershell so please let me know what do do on an ordinary PC with Windows 7.

Thank you.  I don’t guess you realize that you’ve just proven my point.   If you’re having performance problems, it’s obviously with your computer.   

If your computer can’t move files faster, it’s not the NAS’s fault…  Go upgrade your RAM (or whatever) and then we’ll talk.

You don’t need Powershell (as I explicitly stated above).   Windows Explorer moves files just as fast…

Here’s moving a bunch of smaller (10-100 megabyte files)…  Still very fast.

Untitled.png

And even smaller files (1-2 megabyte)  Yes, it’s only 50% of the above speeds, but that’s not atypical when dealing with small files due to filesystem overhead.

Untitled2.png

Captain_Panaka wrote:>  I still can not believe that 80 GB/s reading from MyCloud is possible. Could it be that you have bought it very early and that at this time the hardware was different from what is delivered today?

There are dozens of online reviews by third parties that have performance that high (or even faster).   But given how you’re describing things, I’m sure even that won’t sway you – it’s just furthering some vast conspiracy.

LigitReviews:   45 to 60 Megabytes per second

http://www.legitreviews.com/western-digital-cloud-2tb-nas-review_131939/3

TechRadar:  14-17 Megabytes per second on WIRELESS (they said it was much faster on Ethernet)

http://www.techradar.com/us/reviews/pc-mac/networking-and-wi-fi/network-attached-storage-nas/wd-mycloud-1188379/review/2#articleContent

PC Magazine:  63 Megabytes per second

http://www.pcmag.com/article2/0,2817,2424976,00.asp

CNET: 87 Megabytes per second

http://www.cnet.com/network-storage/wd-my-cloud-4/4505-3382_7-35828120-2.html

PC Word:  80 Megabytes per second

http://www.pcworld.com/article/2051347/wd-my-cloud-review-a-better-more-secure-alternative-to-cloud-storage.html

Trusted Review:  92 Megabytes per second!

http://www.trustedreviews.com/wd-my-cloud_Peripheral_review_setup-performance-verdict_Page-2#tr-review-summary

Electronista:  “More than 60 Megabytes per second”

http://www.electronista.com/reviews/western-digital-my-cloud.html

So why the variability?   Because they’re all using different test systems!  OF COURSE the computer used is going to make a difference.   

What NAS are you using that you can get 80 megabytes per second on your laptop?

We’ll I’ll jump in too…

Tony is right though. Samba read and write is very fast with the cloud drive. I also get 80MB + read from the drive and about 58mb+ write to it…
This thread is about Rsync speed… Which is slow.

I’m told my book live had a single CPU but faster clock speed…wdcloud has dual core but lower clock speed. So as Rsync uses single core only Rsync was faster on the my book live