Slow transfer speed for WD MyBook 4TB USB3 + WD My Cloud 4TB

My Cloud OS 3 Personal & Network Attached Storage My Cloud
1

Hi,

They are connected directly by the USB3 cable that came in the MyBook box and the copy was done at SSH session.

After SSH login to the My Cloud, I copied a large movie file. Both devices are USB3 certified but the speed is so slow, about as same as USB2, not any near to a quarter of USB3. What’s wrong?

Thanks,

Song Toan

2

Hi there and welcome to the WD community.

Have you tried to connect the drive directly to a computer to see if the speed is different? This is to verify that the drive works properly on a computer and does not do the same thing.

Hope this advise helps you out.

1 Like
3

Hi,

Have you tried to connect the drive directly to a computer to see if the speed is different?
Yes, I did that even before connecting to the WD My Cloud. My Book is NTFS formatted. WD My Cloud connected to router directly using the included Gigabit cable. My desktop connected to the router using WiFi. My data flow is: Desktop → My Book → My Cloud. The files mostly are big, not a lot of small files.

  1. Desktop → My Book: I connected My Book to my desktop first to backup around 2TB of data from local drive. The speed was 30 MB/sec which is ok to me given my PC is only USB 2.0.

  2. My Book → My Cloud: I connected My Book to My Cloud to transfer those data there in a SSH session. I expected the copy be very fast because both of the devices are USB 3.0 and the USB cable is from the box.

Unfortunately, it VERY slow. I waited for few days till the LED stopped blinking!!! The log from rsync shown that the average rate is about 10MB/s! cp gave a little better 17MB/s for a 9GB movie file. Note that during the copy, both twonky and apache2 were shutdown. ‘top’ command shown only rsync mainly used CPU.This made me crazy because it is even worse than USB 2.

To make sure that My Cloud has no issue, I copied data from the desktop to My Cloud through 1Gb wire network. Both connected to the router using CAT6a cables. The speed was 80MB/s. So the WD My Cloud seems to be ok.

Look like both devices are ok. Then where is my USB 3.0 speed? :wink:

Thanks,
Song Toan

4

You’re doing two things that aren’t very productive –

Using RSYNC is Slow — ALWAYS.    RSYNC is hugely processor intensive.   Even the folks who maintain RSYNC will tell you it’s not an efficient way of doing simple file copies.   Where RSYNC derives its benefit is doing partial incrementals.

Also, the CPU on the my cloud is not that fast.   Samba is a pretty effecient protocol even on low-power CPUs – Rsync is not.

And yes, 10 MB/s is not unbelievable when using RSYNC locally.

“cp” locally is also very processor intensive.   If you watch “top” while doing a “cp” you’ll see it consuming lots of cycles.

5

I fully agree.

Mycloud for me has been rock solid for the last 1and 1/2 years.  Never let me down.

However, mycloud safepoints to extern USB3.0 (attached to it)  is always slow around 12-14 MB/s, no matter what you use (unless mydevice is faulty :-)  and to where you transfer (currently doing safepoints to my FreeBSD box).

However, PC to the same external USB drive read/write 102-80 MB/s respectively and same from mycloud to PC. I think this is a conscious decision by WD to prefer to do safepoints behind the scene while keeping mycloud device responsive. A large safepoint (around 950GB)  and playing movies at the same time myclooud did not even cough.

6

Guys,

Thanks for your quick replies.

I use rsync to synchronize 2 backups, one on My Cloud and the other on My Book. I can’t check manually which files are old and needs to be updated. rsync will automatically do comparison and mirroring. If I need to copy one off, I would just cp the whole thing at once, no need to use rsync.

For the speed: “always slow around 12-14 MB/s, no matter what you use” → it is totally unacceptable. I don’t think a single cp command can bring it down from USB3.0 Gb/s to just 1xMB/s. I tried and saw that even when samba, apache2 and twonky are running (streaming movies to my WD TV Live), the cp still stay at almost the same rate. This shown me that the problem is somewhat at I/O, not CPU bound. I guess we should find out what the real problem is.

I will do some more test. Will try ext4 instead of NTFS.

7

Guys, I have tried a external Maxtor 500GB USB 2.0 connecting to MyCloud. Like before, the unit did not do anything else during the file copy, both twonky and apache2 stopped. ‘top’ command confirmed device’ was idle before the copy command executed. It copied a full HD movie file at around 8.5 to 9.5 MB/s, in both NTFS and ext4. The same hard disk performs at 30 MB/s if connected to PC. Definitely WD should tackle this problem. USB 3.0 certified is misleading if the device brought down to its knee by just a single file copy operation, much worse than USB 2.0!

8

What happens if the block sizes match?

I know it’s more CPU intensive to copy to mismatched file systems.

SongToan wrote:
USB 3.0 certified is misleading if the device brought down to its knee by just a single file copy operation, much worse than USB 2.0!

Don’t go down this road… USB “certification” has absolutely nothing to do with expected performance.  If you insist that it does, then your beef is with the USB Implementors they set the criteria, not WD.  WD met the requirements.

There’s not a single device manufactured that performs at full 649GB/s USB 3 speeds.  Anything less than that is only a matter of degree and opinion.

9

I am seeing a similar problem right now.  I am copying a files folder with about 400GB from MyBook 4TB to MyCloud 3TB and it is really odd.

Transfer speed was only 2-3 Mbps for over 12 hours and then suddenly, just now, it increased to 50-55 Mbps.  Nothing else changed.

Everything is wired connections on 1Gbps LAN.

10

Alright SongToan,

I did some extensive testing of the USB 3.0 speed about a year and a half ago, from a 4TB WD Cloud to a 4TB WD USB My Book as well as from the Cloud to a Vantec USB Raid enclosure via a gigabit ethernet.

Now to be fair, you cannot quote small file copying speeds as the overhead of copying files like jpegs, mp3s’, epubs etc are attrocious even if you are copying directly from computer to a USB 3.0 connected drive. Your first dozen files may get the 140MB/s speeds but once the buffer is filled everything slows down to about 4 to 6 MB/s and all this has to do with sector allocations, blocksizes, directory allocation and so on.

Thus you will always get a speed hit when copying photos, musics and ebooks which is essentially what we store these days anyways so complaining about them applies to every device, not just WD.

  1. So to get a valid speed, you should always test the speed on a large movie file, something in the neighborhood of 800MB and up.

  2. Now you should also make sure that your scanning services are turned off, those are the media scan services that keeps popping up once it has detected that you have copied a photo or a movie and takes up to 100% of your cpu thus reducing your copying speed.  

Through the USB 3.0 port on the Cloud, the last time I checked (1.5 year ago), WD maxes out at 40MB/s when copying from a WD Cloud to a WD My Book (NTFS format). The funny thing is this is the max speed you get from copying an 800MB file from your computer to the Cloud, thus the max speed might be indicative of MAX cpu software speeds rather than maxed network or USB 3.0 port speeds.  

At that time when I saw this 40MB/s, I thought that WD simply capped it to 40MB/s since that was the max speed when writting from a computer to the Cloud but as I write this post up today, I think it is just the max speed of the CPU for passing through data.

The other funny thing was when I tested the Vantec Raid drive using striped 0, which gave speeds of up to 160MB/s when connected directly to a USB 3.0 port on the computer, the max speed of the Cloub USB 3.0 was 30MB/s, leading my suspicions to code like…

if WD drive attach

   set maxspeed to 40MB/S

else (for every other generic drives)

   set maxspeed to 30MB/s

Reading from the Cloud always gives me about 80MB/s read speeds and 70-75MB/s from the attached WD My Book. The write speed for the attached My Book is 35MB/s

Thus:

My Cloud to a computer

   Read 80MB/s

   Write 40MB/s

MyBook to a computer from My Cloud

   Read 75MB/s

   Write 35MB/s

MyCloud to the My Book

   copy 40MB/s

Computer to My Book

   read 110MB/s

   write 80MB/s

MyCloud to a Vantec Raid Stripe 0 Raid

   copy 30MB/s

Computer to the Vantec Raid Stripe 0 Raid

   read 140MB/s

   write 100MB/s

Computer to a $300 router that had a USB 3.0 port

   write 18MB/s

   read 22MB/s 

These values are as a close as I can remember from a year and a half ago. When I tested them for the speeds I had needed a simple NAS that could give me the minimum of 40MB/s writes which is really the average NAS speeds for even a computer to computer NAS. Although you can get faster NAS you end up paying for the speeds and there is no need for that type of speed for an archival unit since once you copy your files over, you don’t do it over and over again.

Your testing should be in the neighborhood of these values and if not, talk to WD.

Good luck…