Rsync problem (QNAP to WD EXT2 ultra) after upgrading to OS5

Hi to all and Happy new year.
Since I upgraded my EX2 ultra to OS5, I’m no more able to run my weekly backup from QNAP server. I set the rsync process following the instructions as the attached file and all was always perfect till the upgrade.
I have realized that something has changed (new rsync version updated to 3.1.3-6 and the root command now is sshd instead root), but I haven’t find the solution yet. QNAP find the connection through rsync but it gives me an error in the transmission speed.
Is anybody experienced my same issue? Have you find a work around to solve it?

Thanks!

Rsync from QNAP to My Cloud HowTo.pdf (1.0 MB)

@alibabau can you provide the rsync command from the QNAP device?
You should be able to see the full command string via SSH on QNAP by running is ‘ps |grep sync’ command after the QNAP backup job is started.

One change to rsync in OS 5 is that OS 5 doesn’t support the blowfish encryption cipher so if QNAP is using blowfish, the rsync process that transfers the data will fail.

I have the same issue after upgrading to OS5 with my Synology Hyper Backup!

I have now realized that the OS5 doesn’t support the blowfish encryption. That’s the reason. On my point of view this is a very big issue so that I have switched off the my cloud forever since it has no more reason to live.
No words to describe this downgrading.

I had the same issue after upgrading to OS5 with my Synology Hyper Backup.

Went back to OS3 and the backup continues…

Workaround:

1 Like

Many thanks! I have downgraded the firmware to the os3 version as described in the guide. It was all perfect…!!!

Never change a running system… :nerd_face:

Any developments in this? Do we still need OS3 to be able to use rsync?

This is the latest posting on rsync - I have a few comments

I am testing just using the USB backup app ( that just got updates since the original OS5 update )

The commands are not working like the older version

I am hoping someone that has looked at the source can tell what changed.


I am looking into why the copy is replacing large 100GB+ Acronis backup files when NOTHING is changed on the NAS ( you can see it moving over a extra file with a random file extension then replacing the old file

I am using copy as it is the closest to the old Mirror backup command.


this link may help on other rsync items

here is what I thing may be different

  1. USB disk format

–modify-window
When comparing two timestamps, rsync treats the timestamps as being equal if they differ by no more than the modify-window value. This is normally 0 (for an exact match), but you may find it useful to set this to a larger value in some situations. In particular, when transferring to or from an MS Windows FAT filesystem (which represents times with a 2-second resolution), --modify-window=1 is useful (allowing times to differ by up to 1 second).

So using any of the FAT/ EXFAT formatting on a external backup disk may not work well
( most suggest Unix or basic windows only )

rsync finds files that need to be transferred using a “quick check” algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file’s data does not need to be updated.

– one option is checksum —
-c, --checksum
This changes the way rsync checks if the files have been changed and are in need of a transfer. Without this option, rsync uses a “quick check” that (by default) checks if each file’s size and time of last modification match between the sender and receiver. This option changes this to compare a 128-bit checksum for each file that has a matching size. Generating the checksums means that both sides will expend a lot of disk I/O reading all the data in the files in the transfer (and this is prior to any reading that will be done to transfer changed files), so this can slow things down significantly.


Just suggesting we ask a few of our regular poster if they know from the backup source code how the command options have changes.

SBROWN wdstaff post has some good background

I use rsync from a remote server to pull data from my EX2 ultra. I was getting rsync errors which prevented data transfers to complete, for example:

receiving incremental file list
unexpected tag 88 [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(1169) [receiver=3.0.3]
rsync error: sibling process terminated abnormally (code 16) at main.c(1524) [generator=3.0.3]

Maybe this is related to your problem or just an additional one.

I discovered that the WD is supplying a bizzare version of rsync with OS5:

mc$ /usr/sbin/rsync --version
rsync version 3.1.3 protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
no socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, no ACLs, xattrs, iconv, symtimes, prealloc
Panic Action: “xterm -display :0 -T Panic -n Panic -e gdb /proc/%d/exe %d”

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
mc$

Note the “Panic Action:” line. This is obviously a debug build of rsync. Bizzare because OS5 does not provide xterm or gdb.

After I override the WD version with one from debian (raspberian) the errors go away and my remote rsync now works properly:

mc$ /usr/bin/rsync --version
rsync version 3.1.3 protocol version 31
Copyright (C) 1996-2018 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
append, ACLs, xattrs, iconv, symtimes, prealloc

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
mc$

Notice the placement of the rsync override link is in /usr/bin and not /usr/sbin.

Hi ejj, well done! could you help us describing in detail how you have updated rsync?
Thanks

I copied rsync binary from my Raspberry Pi. Then I created a persistant softlink /usr/bin/rsync to the version I downloaded from the raspbery. To make it persistant there needs to be a script to re-create the link that runs at boot. My script does other things but here is the part that creates the link: (change GOOD_RSYNC to path you placed the copied rsync binary, I suggest somewhere on /mnt/HD/HD_a2/ )

GOOD_RSYNC=/cust/bin/rsync
RSYNC_LINK=/usr/bin/rsync
if [ -x $GOOD_RSYNC ]
then
ln -sf $GOOD_RSYNC $RSYNC_LINK
echo $RSYNC_LINK set to $GOOD_RSYNC >> $LOG
else
echo $GOOD_RSYNC is missing or not executable >> $LOG
fi
unset GOOD_RSYNC RSYNC_LINK

To get my script to run a boot I installed the Internal Backups app and added a line to call it in
/mnt/HD/HD_a2/Nas_Prog/InternalBackups/start.sh

If you don’t have a raspberry Pi you should be able to download the .deb file from the raspberry repository and use archive tools to extract the rsync binary. The following to works for me:

REPO=https://mirror.usenet.farm/raspbian/raspbian/pool/main
wget $REPO/r/rsync/rsync_3.1.3-6_armhf.deb
ar -x rsync_3.1.3-6_armhf.deb
tar xvJf data.tar.xz ./usr/bin/rsync

Hope that helps.

10/22/21 Note: WD uses rsync for remote backups. It is likely they had a reason to supply a custom build of rsync. If remote backups are not working properly it would be better to ask WD for a solution.