Low transfer speed between two PR4100

My Cloud OS 3 Personal & Network Attached Storage My Cloud Pro Series
1

Hello folks!

I’ve got two PR4100:s sitting on my desk. One 56TB and one 16TB. I use the smaller one to backup some of the shares from the larger one each night - through ‘Remote Backups’ app jobs. I’ve noticed that the speed isn’t really up there. The same thing goes when I SSH into the larger one and copy files to the other one. In general, a scp session to copy one large file between the two sits at around 27 MB/s (more like 200Mbit/s). I’ve got both PR4100 connected to a Unifi 48-port Gbit switch using two cables each, link aggregated in the switch and on the network settings of the PR4100:s I’m using 802.3ad. This, if I’m not mistaken, gives me 2x1Gbit where one client in theory could get (almost) 1Gbit and if there are two clients accessing the NAS at the same time, the whole bond could be used with (in theory) a total of 2Gbit/s.

Each disk (WD RED:s) is capable of delivering almost 180-200MB/s and in a RAID5 setup it should be even more I think.

Do anyone have any thoughts about this. Why do I not see numbers like 80MB/s instead? Is there any limitations in transfer rates over the network when I connect through SSH and copying files like that. If I copying the large file from and to itself, with cp it’s of course as quick as I can exspect but as soon as I involve network, the speed drops down to around 27MB/s.

Example with one large .zip file (9GB)

  • cp
    Copy the file to itself from SSH shell takes 37 seconds. (Read 9GB and write 9GB)

  • scp (local)
    If if copy the file from-and-to the same PR4100 (scp [filename] sshd@localhost:[destfile]) it takes over 5 minutes with a speed of around 27MB/s

I get the same speed if I do the same scp test but instead copying the file to the remote PR4100.

I’m kind of stuck here I must say. I’ve tried to back down from link aggregation on the switch and also put the PR4100:s in ‘Adaptive Load Balacing’ without any progress. Max speed is still the same. All connections are reported as 1Gbit. If I do ‘ip a’ on both PR4100:s I see that the network config is (from what I can see) correct. It’s a bond of both interfaces:

2: egiga0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
link/ether 00:14:ee:03:95:bb brd ff:ff:ff:ff:ff:ff
3: egiga1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP group default qlen 1000
link/ether 00:14:ee:03:95:bb brd ff:ff:ff:ff:ff:ff
10: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:14:ee:03:95:bb brd ff:ff:ff:ff:ff:ff
inet 172.16.1.5/16 brd 172.16.255.255 scope global bond0
valid_lft forever preferred_lft forever

Any ideas of what I can try to do to get higher speed? Anyone else who uses link aggregation together with PR4100 or any other dual UTP NAS?

Thanks is advance
// Robert

3

Interesting! A few follow up questions:

  • Are you using PR4100?
  • Are you using two network cables to one and the same switch? (in a none LACP config, just regular switching)
  • What OS and version are you using?
  • When you say that you get 80-100 MB/s - are you testing this with scp from your NAS as I am trying or are you seeing your higher speed to/from another client on your network?

I have the latest OS and firmware installed, currently OS5 and v5.17.107. I am using both Ethernet ports connected to one switch, now in a Active-Backup config as you suggested with regular switching config on the switchside - no LACP. No change in speed I’m sorry to say. The problem as I see it that I get the same slow speed even when I scp one large file (not several smaller ones) to 127.0.0.1 or the IP address on the local PR4100. No matter what IP destination I pick I get the same slow transfer speed ~ around 26-28 MB/s.

I beleive that I didn’t have this problem before, but I have both replaced my old 24p Gbit switch to a Unifi 48p Gbit switch AND gone to OS5 on both PR4100 since then.

// Robert

4

Hello

A quick update if anyone else finds this thread and have noticed the same thing as I. I get good speed when I access both PR4100, so the problem is really when I’m logged in directly on the NAS and use scp (or ssh/tar). My guess is that for some reason, the bandwidth is restricted from the shell, not to exhaust the box completely when doing background jobs or backups of sort.

// Robert