WD MyCloud problem with rsync - better alternative firmware clean Debian or DSM5

My Cloud OS 3 Personal & Network Attached Storage My Cloud
1

HI,

I have a 4TB WDMC and i will use the WDMC as backup destination for my DS415+.

The DS support rsync for backups, so I create a backup job on the DS and start the backup on the end of the backup the job reports an error.

So I check many forums for help and find the WD Community Post:  Clean debian and OpenMediaVault on WDMyCloud!

I check also the links from Fox_exe and find DSM5 on WDMC.

  • As anybody install the DSM on the WDMC? 

  • Can anybody tell me if rsync will work on the clean Debian with the backup system from the DS

I hope anybody can help me. 

2

 end of the backup the job reports an error.

Before you start changing the firmware, why not simply try to resolve the rsync problem? If you let us know the error message you are getting, one of the many people who use rsync on the mycloud may be able to help.

Or is the rsync problem only incidental, and all you really want is to do a clean Debian install and use OMV/DAM?

3

Hi,

the only thing I want is a working Backup system, so i’m look for a solution. I don’t think that OMV with DAM will help me, I need a working rsync software. So I hope that clean Debian or the DSM is a solution, but if rsync will run on the WDMC firmware it will be perfekt.

message log

Aug 25 15:40:00 ---- synonetbkp: rsync_wrapper.cpp:720 Failed to execute rsync command. source=[/volume1/@tmp/11795/Ordner], target=[name@IP::sync/Ordner/@app/@metadata/], ret=43
Aug 25 15:40:00 ---- synonetbkp: netbkp_backup_metadata.cpp:765 Failed to execute rsync command.
Aug 25 15:40:00 ---- synonetbkp: bkp_backup_metadata.c:22 bkp_backup_metadata() fail.
Aug 25 15:40:00 ---- synonetbkp: netbkp_backup_data.cpp:73 SYNOBkpBackupCurrentMeta() fail.
Aug 25 15:40:00 ---- synonetbkp: backup.cpp:698 SYNOBkpBackupData() failed, err = 8
Aug 25 15:40:01 ---- synonetbkp: Backup task [Rsync.BackUp..Private] completes with result [8]. Time spent: [26265 sec]. [A M P T] in Byte and Count
Aug 25 15:40:01 ---- synonetbkp: size[313269061550, 313269061550, 0, 0] count[71441, 71441, 0, 0] dir[1772, 1772, 0, 0] reg[71441, 71441, 0, 0]

Or did you need a other log?

Synology NAS - rsync.error

Aug 30 18:45:27 (7960) [ERROR]: rsync error: unexplained error (code 255) at io.c(687) [Receiver=3.0.9]
Aug 30 18:49:06 (9403) [ERROR]: rsync error: protocol incompatibility (code 2) at clientserver.c(493) [Receiver=3.0.9]
Aug 30 18:49:08 (9417) [ERROR]: rsync: link_stat "/Sicherungsname/@app/NAS_20150830.dss" (in wd-rsync) failed: No such file or directory (2)
Aug 30 18:49:08 (9417) [ERROR]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1783) [Receiver=3.0.9]
Aug 30 18:49:10 (9455) [ERROR]: rsync: link_stat "/Sicherungsname@app/_Syno_NetShareArchive.db" (in wd-rsync) failed: No such file or directory (2)
Aug 30 18:49:10 (9455) [ERROR]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1783) [Receiver=3.0.9]
Aug 30 18:49:14 (9479) [ERROR]: rsync error: unexplained error (code 255) at io.c(687) [Receiver=3.0.9]
4

I start last night a backup job to get new log files, her are the resultes. I hope anybody can help me.

wd:/var/log# tail -f messages

empty

wd:/var/log/rsync# tail -f rsyncd.log

2015/08/30 21:11:33 [31722] nas.daheim.local recv Sicherung/images/ordnername/IMG_0131.JPG 861966 862110
2015/08/30 21:11:33 [31722] nas.daheim.local recv Sicherung/images/ordnername/Thumbs.db 444928 445020
2015/08/30 21:11:33 [31722] nas.daheim.local recv Sicherung/images/ordnername/@eaDir/.DS_Store@SynoResource 82 122
2015/08/30 21:11:33 [31722] sent 1378633 bytes received 316765846249 bytes total size 316722944424
2015/08/30 23:11:34 [23891] connect from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:34 [23891] rsync on wd-rsync/Sicherung/images/_Syno_BkpMetadata.db from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:34 [23891] building file list
2015/08/30 21:11:34 [23891] rsync: link_stat "/Sicherung/images/_Syno_BkpMetadata.db" (in wd-rsync) failed: No such file or directory (2)
2015/08/30 23:11:34 [23892] connect from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:34 [23892] rsync on wd-rsync/Sicherung/@app/@metadata/Bilder/_Syno_BkpMetadata.db from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:34 [23892] building file list
2015/08/30 21:11:34 [23892] rsync: change_dir "/Sicherung/@app/@metadata/Bilder" (in wd-rsync) failed: No such file or directory (2)
2015/08/30 23:11:50 [23906] connect from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:50 [23906] rsync to wd-rsync/Sicherung/@app/@metadata/ from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:50 [23906] receiving file list
2015/08/30 21:11:52 [23906] nas.daheim.local recv Sicherung/@app/@metadata/Bilder/_Syno_BkpMetadata.db 3072 3112
2015/08/30 21:11:52 [23906] nas.daheim.local recv Sicherung/@app/@metadata/Bilder/.TemporaryItems/_Syno_BkpMetadata.db 3072 3112
2015/08/30 21:11:52 [23906] nas.daheim.local recv Sicherung/@app/@metadata/Bilder/.TemporaryItems/folders.501/_Syno_BkpMetadata.db 3072 3112
......
2015/08/30 21:11:56 [23906] nas.daheim.local recv Sicherung/@app/@metadata/Bilder/Bilder EOS 40D/2011/11/2011_11_17/_Syno_BkpMetadata.db 8192 8232
2015/08/30 21:11:56 [23906] nas.daheim.local recv Sicherung/@app/@metadata/Bilder/Bilder EOS 40D/2011/11/2011_11_19/_Syno_BkpMetadata.db 13312 13352
2015/08/30 23:11:57 [23909] connect from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:57 [23909] rsync to wd-rsync/Sicherung/synobkpinfo.db from nas.daheim.local (192.168.150.2)
2015/08/30 21:11:57 [23909] receiving file list
2015/08/30 21:11:57 [23909] nas.daheim.local recv Sicherung/synobkpinfo.db 3072 3112
2015/08/30 21:11:58 [23909] sent 54 bytes received 3171 bytes total size 3072

nas> tail -f rsync.error

Aug 30 18:45:27 (7960) [ERROR]: rsync error: unexplained error (code 255) at io.c(687) [Receiver=3.0.9]
Aug 30 18:49:06 (9403) [ERROR]: rsync error: protocol incompatibility (code 2) at clientserver.c(493) [Receiver=3.0.9]
Aug 30 18:49:08 (9417) [ERROR]: rsync: link_stat "/Sicherung/@app/nas_20150830.dss" (in wd-rsync) failed: No such file or directory (2)
Aug 30 18:49:08 (9417) [ERROR]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1783) [Receiver=3.0.9]
Aug 30 18:49:10 (9455) [ERROR]: rsync: link_stat "/Sicherung/@app/_Syno_NetShareArchive.db" (in wd-rsync) failed: No such file or directory (2)
Aug 30 18:49:10 (9455) [ERROR]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1783) [Receiver=3.0.9]
Aug 30 18:49:14 (9479) [ERROR]: rsync error: unexplained error (code 255) at io.c(687) [Receiver=3.0.9]
Aug 30 23:10:43 (13693) [ERROR]: rsync: link_stat "/Sicherung/images/_Syno_BkpMetadata.db" (in wd-rsync) failed: No such file or directory (2)
Aug 30 23:10:43 (13693) [ERROR]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1783) [Receiver=3.0.9]
Aug 30 23:10:43 (13694) [ERROR]: rsync: change_dir "/Sicherung/@app/@metadata/Bilder" (in wd-rsync) failed: No such file or directory (2)
Aug 30 23:10:43 (13694) [ERROR]: rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1783) [Receiver=3.0.9]
Aug 30 23:11:06 (13734) [ERROR]: rsync: writefd_unbuffered failed to write 5 bytes to socket [sender]: Connection reset by peer (104)
Aug 30 23:11:06 (13734) [ERROR]: rsync error: rsync service is no running (code 43) at io.c(687) [sender=3.0.9]

nas> tail -f messages

Aug 30 22:03:34 nas afpd[23717]: [socket.c:164-E] read: No route to host
Aug 30 22:03:34 nas afpd[23717]: [dsi_stream.c:518-E] dsi_stream_read: len:-1, No route to host, flag [12], stored[0], read_count[1059906]
Aug 30 22:03:56 nas afpd[3202]: [socket.c:164-E] read: No route to host
Aug 30 22:03:56 nas afpd[3202]: [dsi_stream.c:518-E] dsi_stream_read: len:-1, No route to host, flag [140], stored[0], read_count[16485126]
Aug 30 22:03:56 nas afpd[3202]: [dsi_stream.c:518-E] dsi_stream_read: len:0, unexpected EOF, flag [12], stored[0], read_count[16485126]
Aug 30 23:11:06 nas synonetbkp: rsync_wrapper.cpp:720 Failed to execute rsync command. source=[/volume1/@tmp/9386/Bilder], target=[backups@10.10.10.5::wd-rsync/Sicherung/@app/@metadata/], ret=43
Aug 30 23:11:06 nas synonetbkp: netbkp_backup_metadata.cpp:765 Failed to execute rsync command.
Aug 30 23:11:06 nas synonetbkp: bkp_backup_metadata.c:22 bkp_backup_metadata() fail.
Aug 30 23:11:06 nas synonetbkp: netbkp_backup_data.cpp:73 SYNOBkpBackupCurrentMeta() fail.
Aug 30 23:11:06 nas synonetbkp: backup.cpp:698 SYNOBkpBackupData() failed, err = 8
Aug 30 23:11:07 nas synonetbkp: Backup task [Rsync.BackUp..Private] completes with result [8]. Time spent: [15723 sec]. [A M P T] in Byte and Count
Aug 30 23:11:07 nas synonetbkp: size[316722944424, 316722944424, 0, 0] count[72182, 72182, 0, 0] dir[1785, 1785, 0, 0] reg[72182, 72182, 0, 0]
5

 /Sicherung/@app/@metadata/Bilder

My suspicion would be those ‘@’ characters in the path: are they allowed in Debian?

Or is that path automatically generated (but failing auto-generation)?

6

The @ in the path is automatically generated by the backup job.

the backup job will run to the end and then stops with an error.

I found this in the Synology Logs in the UI:

[Network to share][Rsync.BackUp..Private] Backup folder [/Bilder] failed. ([43] Connection to the destination server is timeout. Please check the following and try again:The destination server is connected to a stable network.Backup client and server is busy.)
7

Can you run the rsync command to another destination?

I note the reports of protocol errors: what versions of sync are you running? Maybe there’s been a protocol change?

Sorry, I’m not an rsync expert; I was hoping other users might pick up your problem. Some usfeul-looking threads come up with a search:

http://community.wd.com/t5/forums/searchpage/tab/message?filter=labels%2Clocation&location=forum-board%3Amycloud&q=rsync

If no-one else comes up with any suggestions, I’d have a trawl through those for ideas.

8

I look for another rsync destination and I will look what version run on the devices.

But it will take a few moments

9

Check rsync version, hope anybody can tell me what is wrong.

Synology NAS

NAS> rsync --version
rsync version 3.0.9 protocol version 30
Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    no socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, no ACLs, xattrs, iconv, no symtimes

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.

WD mycloud

WDMYCLOUD:~# rsync --version
rsync version 3.0.9 protocol version 30
Copyright (C) 1996-2011 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes

rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
10

Just wondering and kind of a stupid question…

Have you tried a manual rsycn with some basic shares and few files with no special characters anywhere? I would try that first and in both directions…

11

No I don’t try a manual rsync. I will try it if my WDMC will start normal working after the firmware update :frowning:

The capacity in the UI will only show -

something like that?

rsync -aP /volume1/Images/ root@wdmycloud:/DataVolume/shares/mybackups
12

Command is fine for testing, folders, I would try some with very few and small files but up to you.

See this if it helps…

https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories-on-a-vps

http://www.tecmint.com/rsync-local-remote-file-synchronization-commands/

For long term, you should not allow root to SSH remotetly nor use it for rsync, it is just best security practice. Create a user for it. Plenty of info out there as well.

13

I try the follow command:

NAS - rsync -aP /volume root@IP:/DataVolume/shares/rsync

rsync: writefd_unbuffered failed to write 4092 bytes to socket [sender]: Broken pipe (32)
rsync error: rsync service is no running (code 43) at io.c(687) [sender=3.0.9]

Is there another log were I can find more informations why the rsync end in an error?

14

I try it with 

scp -r /volume1/BigFiles root@IP:/DataVolume/shares/rsync

 and

scp -r /volume1/SmallFiles root@IP:/DataVolume/shares/rsync

both backup run without an error. 

  • So why rsync quit the backups with an error?

  • Will rsync work better with an clean Debian?

  • Has anybody install DSM on the WDMC? Who is the performance?

15

Not sure nor a Linux expert, but there might be a big clue on your previous post…

rsync error: rsync service is no running (code 43) at io.c(687) [sender=3.0.9]

:wink:

16

Yes, I also see the error, but the transfer was run until the end and than stops with en error.

17

I rebuild my WDMC with Firmware 03.04.01-219 and test the rsync again with an normal User. At the End of the sync I get the old error. So I tried it with root as user and “:dizzy_face:amazing:smileyvery-happy:” the sync was runing with any error. I know that the root user is not optimal for the sync but it works. Does anyone have any idea why it works with the root user and not with a regular system user. Apart from the statement that it is the root user.

18

I am struggling with the NAS, and I am thinking to throw it through my window… aarrrrgh.

When I connect to it over SSH, I get connection: everything is fine. But, when I connect to it, and then, I try to connect to another NAS using rsync, ssh, scp, whatever, it doesn’t work. The error that is shown is:

no matching mac found: client hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96 server hmac-sha2-256,hmac-sha2-256-etm@openssh.com,hmac-sha2-512,hmac-sha2-512-etm@openssh.com,umac-128-etm@openssh.com,umac-128@openssh.com
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(601) [sender=3.0.7]

The version that uses my WdMyCloudEx2 seems very outdated by the way, because:

root@WDMyCloudEX2 root # ssh -v
OpenSSH_5.0p1, OpenSSL 1.0.1m 19 Mar 2015

Don’t know what to do, because the other NAS uses OpenSSH_6.8…

$ ssh -V
OpenSSH_6.8p1-hpn14v6, OpenSSL 1.0.2g-fips 1 Mar 2016

Tried several workarounds with no success. People, need help from you!

Thanks in advance.