Rsync from My Cloud v04 to attached usb storage

BeKn2 · December 7, 2017, 3:17pm

As Automatic Safepoint don’t work for me, I’ve made my own backup system. For the last 2 years I have mirrored user data on My Cloud directly to a My Passport Ultra usb drive.

A crontab entry starts the script file /usr/local/bin/myBackup every morning at 5 o’clock. Rsync is set up to mirror user content on /DataVolume/shares/. When rsync is finished an email is sent (by sendEmail) to me. This email shows me the rsync exit values for each share, so I can see if something went wrong.

Also the last update of the history file and it’s content is shown so I can see if someone has been tampering My Cloud via SSH.

However, there is one small irritating issue. Sometimes I receive rsync exit values different from 0 (meaning something went wrong). This happens maybe 2-3 times in a month.

I have searched WD Forums a lot, and these two threads describe this problem in more details:
https://community.wd.com/t/please-add-vatiable-timeout-value-for-usb-backup-rsync-in-mycloud-mirror-firmware/97134

I wonder, how can the timeout value of 30 seconds in usb_backup program be changed to a higher value?

alex.s · December 14, 2017, 12:19am

Hi BeKn2,

Maybe you should try contacting WD’s Technical Support about this.

To Contact WD for Technical Support

https://support.wdc.com/support/case.aspx?lang=en
The link below will allow you to call support.

BeKn2 · December 14, 2017, 1:14pm

Hi dswv42.

Thank you for professional information, I appreciate it a lot.

I have a My Cloud v04.05.00-320 and after a closer look there seems to be no usb_backup file on it. On my device, file is not in /usr/sbin/ and not in /usr/local/modules/usrbin/ directory. I also tried:

find / -name usb_backup -print

that search from root and downwards thru the hole file structure. It took about 5 minutes, but file was not found.
If anyone have some hints I would be grateful, else I may contact WD support.

hpangels · February 28, 2018, 5:10pm

Hi BeKn2,
I am trying to do exactly what you’re describing on my EX4 - would you so kind and share the details, maybe scripts I could adapt? That would be awesome!
Thanks,
-hpangels

Bennor · February 28, 2018, 6:31pm

The My Cloud Mirror firmware is DIFFERENT than the first gen single bay My Cloud firmware. As such files present within the Mirror firmware are not going to be found within the v4.x single bay My Cloud firmware. I believe the My Cloud Mirror like other v2.x My Cloud models use a form of Busy Box (Linux) where as the v4.x single bay My Cloud uses Debian Linux. And no, one cannot update a first gen single bay My Cloud to use the second gen v2.x single bay My Cloud firmware.

BeKn2 · March 1, 2018, 11:53am

Yes I know. The reason I was searching after /usr/sbin/ was that dswv42 had a solution to the problem, but we later found out that he/she use the second gen v2.x, so the post was deleted.

BeKn2 · March 1, 2018, 12:25pm

I have translated my script to English and changed it so it is more anonymous. Before you implement it:

It is made for use with My Cloud v04.05.00-320 and an USB attached to it
WD Warranty may be voided if something goes wrong when using SSH
You must have root access to your device terminal
Several lines must be changed to adapt it to other devices, see comments in script file

#!/bin/bash
#--------------------------------------------------------------
# Mirror files on My Cloud v.04 to attached USB drive
#   Version : 2.0
#   Date : 20170815
#   Script help: https://www.tutorialspoint.com/unix/index.htm
#   Experimental settings:
#    1. Sleep is used to give Linux time to allow the 
#       shell to not conflict executing commands.
#    2. --bwlimit=1000 parameter to rsync
# -------------------------------------------------------------

#*** Function definition - start ******************************

# 3 parameters are passed:
#   $1: rsync log-file
#   $2: rsync SRC
#   $3: rsync DEST
Rsync_and_set_email_msg_lines () {
   sleep 9
   rsync -a --delete --bwlimit=1000 --log-file="$1" "$2" "$3"
   if [ "$?" -eq "0" ]
     then
       msgLn=$2" Rsync exit value is "$?". Ok."
   else
     msgLn=$2" Rsync exit value is "$?". Error, check logfiles."
     errorFlag=true
   fi
return $TRUE
}

# *** Batch script - Start  ************************************
startTime=$(date +"%T")
errorFlag=false
usbDevice=M36
nasDate=$(date +"%Y%m%d")

# Warning: Mirroring catalog in Bob must exsist!
# Check SSH history to detect hacker attack - copy history file 
# to user Bob
sshLogfile="/DataVolume/shares/Bob/Mirroring/"$nasDate"-ssh.log"
cp ~/.bash_history $sshLogfile
chmod 644 $sshLogfile

# Find last update of history file
sshLastUse=$(stat -c "%Y"  ~/.bash_history)
sshReadableDate=$(date -d @$sshLastUse)

myLogFile=$nasDate"-M31_intern-"$usbDevice".log"

# Warning: Mirroring catalogs in shares must exsist!
logFileBob="/DataVolume/shares/Bob/Mirroring/"$myLogFile
logFileAlice="/DataVolume/shares/Alice/Mirroring/"$myLogFile

src1="/DataVolume/shares/Bob/"
src2="/DataVolume/shares/Alice/"
src3="/DataVolume/shares/Public/"
src4="/usr/local/bin/myBackup"

# Warning: dest1-dest3 catalogs must exsist on usb device!
dest1="/var/media/My_Passport/OurBackup/Bob"
dest2="/var/media/My_Passport/OurBackup/Alice"
dest3="/var/media/My_Passport/OurBackup/Public"
dest4="/DataVolume/shares/Bob/Mirroring/Linux rsync"

Rsync_and_set_email_msg_lines "$logFileBob" "$src4" "$dest4"
message4="$msgLn"
Rsync_and_set_email_msg_lines "$logFileBob" "$src1" "$dest1"
message1="$msgLn"
Rsync_and_set_email_msg_lines "$logFileAlice" "$src2" "$dest2"
message2="$msgLn"
Rsync_and_set_email_msg_lines "$logFileBob" "$src3" "$dest3"
message3="$msgLn"

# Send email using sendEmail - http://caspian.dotconf.net/menu/Software/SendEmail
if [ "$errorFlag" = true ]
   then
     subject="My Cloud Mirroring - Error detected."
     message="Something is wrong, check logfiles.\n\n"$message1"\n"$message2"\n"$message3"\n"$message4
   else
     subject="My Cloud Mirroring is ok."
     message="Everything is ok.\n\n"$message1"\n"$message2"\n"$message3"\n"$message4
fi

msgHeader="Hi..\n\nStarttime: "$startTime". Internal mirroring of M31 to "$usbDevice".\n\nResult: "
msgFooter="Rsync exit values, see: https://www.samba.org/ftp/rsync/rsync.html\n\n"
msgFooter="$msgFooter""Logfile "$myLogFile" is stored\n"
msgFooter="$msgFooter""in catalog Mirroring for user Bob og Alice.\n\nLast update of history-file: ""$sshReadableDate""\n\nFrom\nMy Cloud "
message="$msgHeader""$message""\n\n""$msgFooter"

# Warning: Following settings must be adjusted to individual values:
fromAdress="myCloud@domene.com"
serverOut="smtpin.domene.com:587"
username="My Cloud"
password="myPassword"
toAdress="my-address@domene.com"

sleep 9
/usr/local/bin/sendEmail -f "$fromAdress" -t "$toAdress" -u "$subject" -m "$message" -s "$serverOut" -xu "$username" -xp "$password" -a "$sshLogfile"

hpangels · March 1, 2018, 6:21pm

Thank you very much for that!