@Darren_Ryan
I had the same task to solve and ended up with creating a little custom WD-App that can be installed on the cloud device. This way it persists all settings across reboots and firmware updates and it seems like the only valid option to achieve this.
I’ll give a little intro how to build this custom app that uses RSA keys to connect to the remote server and uses rsync to download all backup files from the server to your local WD Cloud device. I used a WD MyCloudMirror Gen2 here, but I’m sure this script can also be used for other models.
All custom apps are built with Docker, a virtualization software that keeps every application in it’s own little container with it’s own Operating System - yes. We need to develop the custom app on the WD device itself, because it has limited capabilities and that way we can spot issues a bit better. Check http://developer.mycloud.com/MCDSDKindex.html for details about how to develop apps for the WD Cloud Devices.
1. Enable SSH on your WD Cloud device
- Go to the admin dashboard and open Settings → Network
- Enable ssh and create a password. The ssh login username is sshd
- You can use any SSH client such as Putty.
2. Connect to the device via SSH
Once connected, first check the docker version installed on the machine. Type:
docker version
if should display something like this:
Client version: 1.7.0
OS/Arch (client): linux/amd64
For EX2 devices, it’ll probably display armv7.
3. Docker Base Image
Now before you can create an app, you need to install the base image.
Installing Docker base image on My Cloud DL2100 and DL4100:
docker pull ubuntu:14.04
Installing Docker base image on My Cloud EX2100, EX4100 and Mirror - Gen 2
docker pull armv7/armhf-ubuntu:14.04
4. Get familiar with docker commands
If something does not work or you screwed things up, it’s good to know at least these simple commands to get an overview:
show installed images:
docker images
show running container:
docker ps
show all available container:
docker ps -a
start new container with direct shell access:
docker run -it --name="helloworld_build" -v /shares/Public/helloworld:/helloworld ubuntu:14.04 /bin/bash
stop container:
docker stop helloworld_build
remove container:
docker rm helloworld_build
remove image:
docker rmi ubuntu:14.04
5. Prepare Docker App
In order to build the app, we need a working folder first. Goto your Public folder and create a folder. I’ll use RSyncSSH so it’ll be located at /shares/public/RSyncSSH/
Within that folder we place the files of our application. I ended up with this directory structure:
- backup-rsync.sh
- build/
- Dockerfile
- keys/
- root.crontab
- start.sh
My backup rsync script looks like the following (adjust this for your needs):
backup-rsync.sh:
#!/bin/bash
# Simple backup with rsync
# local-mode, tossh-mode, fromssh-mode
NAME="MyServer"
SOURCE="/backups/"
TARGET="/shares/Backup-MyServer"
#RSYNCCONF=(--delete)
RSYNCCONF=
# check local mountpoint
#MOUNTPOINT="/mnt/HD"
#MAILREC="mail@myserver.com"
NOTIFYURI="https://myserver.com/backup-notify/?data="
SSHUSER="wdbackup"
FROMSSH="myserver.com"
SSHPORT=22
### do not edit ###
MOUNT="/bin/mount"; FGREP="/bin/fgrep"; SSH="/usr/bin/ssh"
LN="/bin/ln"; ECHO="/bin/echo"; DATE="/bin/date"; RM="/bin/rm"
DPKG="/usr/bin/dpkg"; AWK="/usr/bin/awk"; MAIL="/usr/bin/mail"
CUT="/usr/bin/cut"; TR="/usr/bin/tr"; RSYNC="/usr/bin/rsync"
LAST="last"; INC="--link-dest=$TARGET/$LAST"
#LOG=$0.log
LOG="${TARGET}/backup.log"
$DATE > $LOG
if [ "${TARGET:${#TARGET}-1:1}" != "/" ]; then
TARGET=$TARGET/
fi
if [ "$LISTPACKAGES" ] && [ -z "$FROMSSH" ]; then
$ECHO "$DPKG --get-selections | $AWK '!/deinstall|purge|hold/'|$CUT -f1 | $TR '\n' ' '" >> $LOG
$DPKG --get-selections | $AWK '!/deinstall|purge|hold/'|$CUT -f1 |$TR '\n' ' ' >> $LOG 2>&1
fi
if [ "$MOUNTPOINT" ]; then
MOUNTED=$($MOUNT | $FGREP "$MOUNTPOINT");
fi
if [ -z "$MOUNTPOINT" ] || [ "$MOUNTED" ]; then
if [ "$SSHUSER" ] && [ "$SSHPORT" ]; then
S="$SSH -oStrictHostKeyChecking=no -p $SSHPORT -i /root/.ssh/id_rsa";
fi
if [ "$S" ] && [ "$FROMSSH" ]; then
$ECHO "$RSYNC -av -e \"$S\" $SSHUSER@$FROMSSH:$SOURCE ${RSYNCCONF[@]} $TARGET" >> $LOG
$RSYNC -av -e "$S" $SSHUSER@$FROMSSH:$SOURCE ${RSYNCCONF[@]} $TARGET >> $LOG 2>&1
if [ $? -ne 0 ]; then
ERROR=1
fi
chmod -R 777 $TARGET
fi
else
$ECHO "$MOUNTPOINT not mounted" >> $LOG
ERROR=1
fi
$DATE >> $LOG
if [ -n "$MAILREC" ]; then
if [ $ERROR ];then
$MAIL -s "Error Backup $LOG" $MAILREC < $LOG
else
$MAIL -s "Backup $LOG" $MAILREC < $LOG
fi
fi
if [ -n "$NOTIFYURI" ]; then
if [ $ERROR ];then
#wget -q "$NOTIFYURI:Error Backup:" < $LOG > NUL
LOGCONTENT=$(<$LOG)
wget -qO- "${NOTIFYURI}ERROR Backup $NAME: $LOGCONTENT" &> /dev/null
else
#wget -q "$NOTIFYURI:Backup $LOG" > NUL
LOGCONTENT=$(<$LOG)
LOGSPLIT=$(echo "$LOGCONTENT" | grep -A1 ^sent)
wget -qO- "${NOTIFYURI}Backup complete: $NAME - $FROMSSH $LOGSPLIT" &> /dev/null
fi
fi
I didn’t get it to work to send the notification emails, but I successfully call a URL after the rsync-backup is complete, so I can notify myself or write another log file entry.
Next the Dockerfile. This is used to install and prepare additional services to the OS within the app container:
# The FROM keyword must precede all other instructions. This specifies the Base image.
FROM armv7/armhf-ubuntu:14.04
MAINTAINER ikkez
# install requirements
RUN apt-get -q update
RUN apt-get install -y openssh-client && \
apt-get install -y rsync && \
apt-get install -y wget && \
apt-get clean
# copy backup keys
RUN mkdir /root/.ssh
COPY keys/id_* /root/.ssh/
RUN chmod 600 /root/.ssh/*
# copy start script
COPY backup-rsync.sh /backup-rsync.sh
RUN chmod u+x /backup-rsync.sh
# copy container start script
COPY start.sh /start.sh
RUN chmod u+x /start.sh
#set timezone
RUN rm /etc/localtime
RUN ln -s /usr/share/zoneinfo/Europe/Berlin /etc/localtime
# setup cron tasks
ADD root.crontab /etc/cron.d/hello-cron
RUN chmod 0644 /etc/cron.d/hello-cron
RUN touch /var/log/cron.log
RUN /usr/bin/crontab /etc/cron.d/hello-cron
# add group to be able to set ownership correctly after transfer
RUN groupadd share
# The command to run when your Docker App container starts.
CMD [ "/start.sh" ]
####
Now the root.crontab file. This contains the crontab settings. I let my backups run at 2am. Adjust as you like.
0 2 * * * /backup-rsync.sh > /dev/null 2>&1
#* * * * * echo "Hello world" >> /var/log/cron.log 2>&1
# An empty line is required at the end of this file for a valid cron file.
And finally the start.sh file. This is used to start and stop the container properly:
#!/bin/bash
# When using a bash script as the entrypoint, you need to handle SIGTERM properly,
# Otherwise, Docker may fail to stop your app and forcefully kill it.
# An alternative is to use supervisord as the entrypoint to start your app. In that case,
# you will need to apt-get install supervisor in your Dockerfile.
function sighandler()
{
kill -TERM $PID
}
trap sighandler TERM INT
echo "Starting RSyncSSH App"
exec /usr/sbin/cron -f
PID=$!
wait $PID
The file at /build/app.json contains some required information to install the final app on the WD device:
{
"SDKVersion": "1.0",
"Name": "RSyncSSH-myServer",
"DisplayName": "RSync Backup myServer.com",
"IconFile": "icon.png",
"Description": "RSyncSSH Backup from remote Server",
"Description.en_US": "RSyncSSH Backup from remote Server",
"Description.cs_CZ": "RSyncSSH Backup from remote Server",
"Description.de_DE": "RSyncSSH Backup from remote Server",
"Description.es_ES": "RSyncSSH Backup from remote Server",
"Description.fr_FR": "RSyncSSH Backup from remote Server",
"Description.hu_HU": "RSyncSSH Backup from remote Server",
"Description.it_IT": "RSyncSSH Backup from remote Server",
"Description.ja_JP": "RSyncSSH Backup from remote Server",
"Description.ko_KR": "RSyncSSH Backup from remote Server",
"Description.no_NO": "RSyncSSH Backup from remote Server",
"Description.nl_NL": "RSyncSSH Backup from remote Server",
"Description.pl_PL": "RSyncSSH Backup from remote Server",
"Description.pt_BR": "RSyncSSH Backup from remote Server",
"Description.ru_RU": "RSyncSSH Backup from remote Server",
"Description.sv_SE": "RSyncSSH Backup from remote Server",
"Description.tr_TR": "RSyncSSH Backup from remote Server",
"Description.zh_CN": "RSyncSSH Backup from remote Server",
"Description.zh_TW": "RSyncSSH Backup from remote Server",
"Vendor": "ikkez",
"VendorURL": "http://www.myServer.de",
"SupportURL": "http://www.myServer.de",
"Version": "1.0",
"ProductArchitecture": "armv7",
"DockerImageFile": "rsync-ssh.tar.gz",
"ConfigURLPort": 80,
"ConfigURLPath": "",
"ConfigAllUsers": true,
"Configuration":
{
"Network":
{
"Mode": "host"
},
"Volumes":
{
"RequireAllVolumes": true
}
}
}
The image file at build/icon.png can be any icon for you app. Mine is 363x363px and does work fine.
The key files at keys/ are used to connect via SSH to the remote server. The private key file does NOT have a passphrase set, as the installed OpenSSH version in this container isn’t able to deal with it and there doesn’t seem to be a way to upgrade it 
The public key needs to be added to your remote servers .ssh/authorized_keys file of course.
Now the files are ready. Please ensure that the owner of this files is root. You can change the file owner like this:
cd /shares/public/RSyncSSH/
chown -R root:root *
Please note that you probably cannot change the files anymore via the network shared folder once you’ve changed the owner, but you can use a software like WinSCP or Transmit (Mac) to connect via ssh and open&edit files that way.
6. Build the Image
cd /shares/public/RSyncSSH/
dos2unix start.sh
dos2unix backup.sh
docker build --rm -t ikkez/rsync-ssh:1.0 .
when you have edited the .sh scripts with a windows editor, the line endings could have changes, which could result in a failed script execution. That’s why I’ve added dos2unix, just to make sure this issue doesn’t steal your time ( like it happend to me).
7. Create and run new container
Start the container in background:
docker run -dit --net="host" --name="RSyncSSH" -v /shares:/shares -v /mnt/HD/HD_a2:/mnt/HD/HD_a2 ikkez/rsync-ssh:1.0
optionally check some things manually:
check system time: (important if you want to run exact sheduled backups)
docker exec RSyncSSH date
You can change the timezone in the Dockerfile above. I’ve used Europe/Berlin.
Manually test the backup-script:
docker exec RSyncSSH ./backup-rsync.sh
If that works, it’ll take a while to complete and free the terminal, depending on how much it has to download. If you can find the files in the TARGET shared folder that was specified in the backup script, you’re ready to go.
8. Create App Package:
Export container, this can take some minutes:
docker save ikkez/rsync-ssh:1.0 | gzip > build/rsync-ssh.tar.gz
Create WD Package:
cd build/
tar -cvf rsync-ssh-app_myserver.tar -C . .
The dots . are important.
9. Clean up:
docker stop RSyncSSH
docker rm RSyncSSH
docker rmi ikkez/rsync-ssh:1.0
10. Download & Install
Now download the rsync-ssh-app_myserver.tar file from public shared folder and install it via web interface on your device. On the Apps Tab there is a link “Install an app manually”. Click this link, select the downloaded .tar file and wait for it to finish. After some time the new container has installed and should show up as usual application:
I have zipped all important files from this little guide here: http://bit.ly/2rhFMnH but not sure if I’ll keep them online there forever.
I hope this can be helpful for anyone to dive into creating custom apps or just to get a nice rsync backup to work. It took me some days to figure out all this stuff so leave a like if it was helpful. Thank you and have a nice day 
