Alternative to Safepoint - reposted

#2

Thanks for sharing.

1 Like
#3

Hi

I’ve been a Mycloud user for 3-4 years already with 4 different Mycloud/Mypassport products that I have been using at my home and office. I have tried many many times to make safepoint work with the different advice given throughout the years to no avail. I actually gave up on it and have just been using Ubuntu Deja Dup on a dedicated device which basically defeats the purpose of Mycloud. Until I saw your post.

I just want to thank you for sharing this to the community even if I have yet to try it myself (just about to- just clearing up one mypassport drive to use as my backup). I just registered here on this forum to be able to thank you.

Keep it up.

1 Like
#4

@ nicktee55

I’m happy to report back that I was able to make this one work except for the email.
For the scheduling I wanted the Mycloud to do rsync twice day a since there are a numbers of pc’s sync"ed to this device whose files (mostly MS office related) are constantly updated throughout the day. So I used this particular cron command instead for rsync’ng at 12 am and 12 pm:

00 01,12 * * * /shares/system/Tasks/backup.sh

However, on the email part I’m not yet able to make it work and have yet to figure out the problem.
Will post back for any development.

Thanks, man.

#5

Might be your email server. With Gmail I had to change a gmail setting that allowed ‘less secure apps’

gmail security settings

#6

Email is just a nice touch.

Alternatively, get it to simply write a message to a log file. Put the log file somewhere in admin user space, so you can see it from the network without having to SSH in.

#7

When I run the python script I get syntax errors. I even typed the code by hand instead of copy/paste

#8

Looks like the forum formatter has mangled the code, interpreting some characters as format commands. There’s certainly a missing # from the first line, which I’m sure should read

#!/usr/bin/env python

Oh; I’ve already tried to correct that in the original thread:

#9

More than just that:

File “./kkk.py”, line 2
– coding: utf-8 –

#10

I can confirm that the forum formatter has mangled the first two lines for reasons best known to itself. I’ve managed to now post the lines correctly by adding a space at the start of lines 1 and 2.

First two lines should be (if you copy and paste please delete the space I have had to insert at the beginning of each line)

#!/usr/bin/env python

-- coding: utf-8 --

There should also be a leading # on the line “send email”

I think it should be OK now, but any difficulties have a look at arty2’s post at Sending email with My Cloud

#11

Use the ‘code formatting’ to stop it interfering with code; the </> icon in the format bar.

I guessed right about the utf-8 line, then…

#12

See my full corrections at the link i posted. Was the link that hard to follow?

#13

Great post. Set everything up, checked everything working, and then kicked off the cron only to discover … half an hour later … that it was part way through the third share and had stopped copying the first two and these were incomplete.

Stopped everything and dry ran the first rsync to be greeted with the error.

Stack Smashing Detected … Terminated.

So I’m currently using cp -R to make a full copy of all the shares before re-enabling the crontab entry in the hopes that it will cope.

I’m presuming that it’s the volume of files that it is trying to sync up. If I make a few changes then I can imagine it will work, but if something touches a large number of files, this is going to hit again.

Any idea how to avoid it?

#14

Hi DarkRayven

Sometimes I have discovered that rsync can have problems with corrupted files, specially jpg and mp3 for some reason. Rsync tries, and tries again several times, and sometime quit or give up.

Have you tried rsync with log-file output, or double or triple verbose (-vvv)? May be you can find a clue there.

Also, you can try to look at what exit values rsync is returning to investigate further.

#15

Tracked it down to an apparent lack of memory issue.The initial copy was going to be over 100,000 files (and that was just in one directory structure). If I ran the same rsync command again, it skipped the files that were copied and picked up from where it left off, only to fail again later. I think it had too much to cope with (2.4Tb of files in the first run)

I ended up aborting this for the initial copy and using

cp -Rp

instead to make an initial copy.

Once that had been done (took about 12 hours), I scheduled a backup script using

rsync --archive --delete

to keep the mirror copy in sync.

Touch wood it has worked so far, but I’ve had it send me an initial email before the backup starts warning me that if I don’t get a completed one afterwards, to check for fails.

#16

I run this script once per week - backup between MyCloud and older MyBookLive:

mount //192.168.10.3/NAS_BACKUP /media/RSBACKUP -o password=“welc0me”

rc=$?
if [ $rc != 0 ]
then
/usr/bin/python /shares/System/backup-email.py “Message from WD MyCloud” “Failed to mount remote share to RSBACKUP”
exit
fi

sleep 6
cp /dev/null /shares/System/backup.log
sleep 3

rsync -rtDvH --delete --log-file /shares/System/backup.log --exclude-from /shares/System/exclude.txt /shares/ /media/RSBACKUP >/dev/null
rc=$?
sleep 6
umount /media/RSBACKUP
sleep 3

if [ $rc != 0 ]
then
/usr/bin/python /shares/System/backup-email.py “Message from WD MyCloud” “System backup has failed” 2>/dev/null
else
/usr/bin/python /shares/System/backup-email.py “Message from WD MyCloud” “System backup has been completed” 2>/dev/null
fi

#17

Hi whsbuss

Nice mirror script…

I see that you are using -H option (preserve hard links), so this must be a mirror to another Linux file system.

I mirror my files to a My Passport Ultra connected to My Cloud. My Passport uses a Windows file system (maybe exFAT).

I‘m considering formatting My Passport to ext4 so I can have a more complete backup like yours. Is rsync working ok every week, or is the exit value sometimes different from 0?

Also I wonder why you stop execution of batch-file for 3 and 6 seconds. Is it necessary?

#18

So far, rsync has worked flawlessly for a year now. Be advised that when you update the firmware the mount point on /media/RSBACKUP will be removed and will have to be manually reconnected.

I’m an old Unix guy and I like giving the operating system a few seconds to allow the shell to not conflict executing commands.

#19

Bumping this old thread. Used a variation of the code whsbuss posted above to backup select folders to an attached USB hard drive. While the following code isn’t elegant and there are probably better ways to code it, it does appear to work for me.

The following mail.py code will use Yahoo.com email and will attach the backup.log file to the email.

#!/usr/bin/env python

# -*- coding: utf-8 -*-
import smtplib
from email.header import Header
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from email.MIMEBase import MIMEBase
from email import Encoders
from getpass import getpass
from smtplib import SMTP_SSL
import sys  

# edit the areas between the quotes in the line line below with your own settings. 
# Change the backup.log file location to match file to be attached to email.

login, password, server, recipients, logfile = "full.Yahoo.Email.Address.Goes.Here", "Yahoo.Password", "smtp.mail.yahoo.com", "Recipient.Email.Address", "/DataVolume/shares/system/backup.log"  

# send email

subject = sys.argv[1]
body = sys.argv[2]
msg = MIMEMultipart()
msg.attach(MIMEText(body, 'plain', 'utf-8'))
msg['Subject'] = Header(subject, 'utf-8')
msg['From'] = login
msg['To'] = recipients
part = MIMEBase('application', "octet-stream")
part.set_payload(open(logfile, "rb").read())
Encoders.encode_base64(part)
# Change file name in following line to match filename of attached file
part.add_header('Content-Disposition', 'attachment; filename="backup.log"')
msg.attach(part)
s = SMTP_SSL(server, 465, timeout=10)
s.set_debuglevel(1)
try:
    s.login(login, password)
    s.sendmail(msg['From'], recipients, msg.as_string()) 
except Exception, error:
    print "Unable to send e-mail: '%s'." % str(error)
finally:
    s.quit()

The reason for using this code was due to Safepoint being an all or nothing affair and wanting a backup process that would backup a select number of Shares and then email (with log file attached) me. The log file will show what files (if any) have been mirrored to the USB hard drive.

Can post full information/code for anyone interested.

1 Like
WDBCTL0020HWT-00 vs WDBCTL0020HWT-10 OS/Firmware ver & Upgrades
#20

The email by python won’t work for firmware v 2 xxx.
Internally, it is a busybox. Its python does not have SMTP_SSL

#21

This thread was generally discussing alternatives to Safepoint feature which the second gen v2.x My Cloud does not have. The v2.x firmware has a different feature (with different options) called Backup.

The second gen v2.x single bay firmware is different than the first gen v4.x single bay My Cloud firmware. As such it is not surprising that certain packages in the older v4.x firmware is not present in the newer v2.x firmware which uses among other things BusyBox.