Backup to/from external drive

Hi and thanks in advance for your ideas…

I’m using my MyCloudMirror for photos, mainly. I have a descrepancy between the two drives and I need to consolidate the information. Example: Mirror drive holds (ORIGINALPhotos + ABC) and Backup drive holds (ORIGINALPhotos + XYZ). I do want redundancy between the Mirror and the portable drive, but I don’t want multiple copies of photos on the drives if possible. What I’d like is (ORIGINALPHOTOS + ABC + XYZ) on both then do a better job of using one drive more consistently as the source drive.

This sounds like it should be very easy but somehow I’m having trouble. What basic steps will be the fastest and most efficient way to make sure that I’ve got all the files but no real duplicates?

Thanks, MD

I have a WD MyCloud Mirror. The primary value is realtime backups of my MacBook laptop, on which I do all development work. Well, last week my backup process, that had been working well for over two years, was failing.

After much research and testing, I discovered that the Linux server core of my WDMCM had completely filled a temporary file systems with logs from several different processes. This caused the system to be severely sluggish and impacted the backup process to the point of failure. Good thing I had turned on SSH and setup SecureCRT on my iPad with ROOT access credentials. Once those logs files were deleted, things began to work again.

However, my MacBook TimeMachine backups on the NAS were now failing integrity checks, and I had to start a new backup. I had felt ‘safe’ for the last couple years that the NAS with two hard drives, used in a mirror function, basically making a realtime backup of all data stored on the NAS, was a solid and dependable backup process.

So, I had to rethink my back strategy and figure out a way to provide a THIRD backup (my MacBook as source, the NAS as primary backup, a third device to hold a copy of the data stored on the NAS).

The WDMCM has a USB interface and backup process, however, based on my own experience and much of what I have read online, the NAS just does not have the ‘horse power’ to backup from the primary shares to the attached USB hard drive. My understanding, based on much of what I read, using the USB interface to transfer data was taking excessive processing time causing other functions to be severely delayed, things like the primary job of making backups of my MacBook.

After quite a bit of research, finding ‘bits and pieces’ of information, I devised a plan to use a Raspberry Pi 4 computer with an attached (USB) 4TB hard drive, as my THIRD backup. It took a lot of trial & error, but I setup the Raspberry Pi as an rsync server, NOT using the rsync daemon.

Using SSH Keys, I can issue a single rsync command on the WDMCM to ‘push’ entire shares to the Raspberry Pi. I connected to the WDMCM with ssh. While logged in as root, I ran ssh-keygen. Then copied the .ssh/id_rsa.pub file to the Raspberry Pi user backupofmycloud .ssh/authorized_keys.

Here is a copy of the script file I run as a cron job on the WDMCM:

#!/bin/sh
rsync --archive --quiet --compress --recursive --times --perms --partial
–links --delete --copy-links --delete-after --log-file=/tmp/rsynclog.txt
/shares/Public/ backupofmycloud@192.168.1.141:/home/backupofmycloud/shares/Public/

The Raspberry Pi is not super fast, and because I’m using the LAN interface, the rsync process on both the WDMCM NAS and Raspberry Pi are not exhausting CPU resources. When the rsync transfer is active, I still have good response from both.