Now this might not be WD my cloud specific. But is there no way to have rsync use both CPUs while syncing?
I currently use rsync between two my cloud drives, I can see that it uses 100% of one cpu while the other core stays pretty much un used. Even though I’m using rsync with no md5 calculation, it maxes out at 19MB/sec on a gigabit lan.
My older my book live drives sync with each other at 26MB/sec. That’s because they have a single core cup but faster clock speed. Could rsync some how be compiled to be multi threaded?
That will invokde rsync (in the example) 24 times. but by separating the operations and breaking down the directories it may run faster. start with value 2 for $threads and increase gradually to find the optimum number.
It does not make rsync multithreaded though.
To make rsync multithreaded, you need to get the source, refactore it, find criticial chunks that can be threaded and recode it with threads (something like posix threads), taking care of race conditions, deadlock, rendeavous, etc… it can be done :smileyvery-happy:
Thanks. I had come across that before but havent tried it yet.
True, its not multi threaded but rather simultanoes rsync tasks from what i understand…I can see that creating more I/O on the disk which might end up slowing down the file transfer itself…
Well, all this multi-thread discussion sounds a bit noob.
My 4TB WD My Cloud has 2 procesing cores.
If one core is running the kernel, filingn system etc, and the other is running rsync, that pretty multi-threaded really.
If you want more threads, you can just run several rsyncs at the same time.
The idea was to see if rsync can use all the cpu power available when running to speed up syncs. Currently it only maxes out 1 cpu core.
Running several rsync instances would simply increase disk I/O activity and would probably decrease overal thoroughput.
Well, you could use the rsync -z flag to compress the packets before sending them.
That would give some more work for both the sending and receiving CPUs to do, compressing and uncompressing packets, whilst potentially increasing the throughput.
Thanks but i doubt that would help. The cpu is already under powered in this device.