My Cloud Mirror can't backup to USB

Hi All,

There are two threads around USB backups - I can say as of today - I have been able to run usb backups - but not scheduled.

I have enabled SSH and logged in and it appears that RSYNC is installed. Here is an article on how to run rsync:

http://www.howtogeek.com/135533/how-to-use-rsync-to-backup-your-data-on-linux/

It is my assumption that scheduled or repeating backups to USB drives is not enabled as they can be removed and cause and issue if they are not there…

My USB Drive will always be there so I am going to attempt to make a RSYNC cron and will post it back here for everyone’s reference.

To create a recurring USB backup - you will want to first enable SSH for your MyCloud (in my case a mirror).

Instructions on enabling SSH can be found here:

http://wdc.custhelp.com/app/answers/detail/a_id/10435/~/how-to-enable-ssh-(secure-shell)-on-a-wd-my-cloud,-mirror,-ex2,-or-ex4-personal

Remember your user name is _ sshd _ and the password is what you set yourself.

:dizzy_face: If you have never used SSH before, you may want to avoid doing this all together. SSH logs you in as the user “root” which is the all powerful, capable of destroying everything, user. If you do this, I make absolutely no warranty you won’t accidentally turn your drive into a brick.

That said - the examples I am giving below work for my specific setup - you will need to adjust them to your own.

================================

Using your favorite terminal app - open up the ssh connection using your my cloud’s ip address:

ssh sshd@192.168.0.1

Because the MyCloud uses a self signed certificate - it will likely require you to agree to connect to it by typing “yes”

Once in, you will want to discover your usb drive’s name on the system itself - use the commands:

cd /mnt/USB/

ls

You will see in the folder list the USB drives that are mounted.

If you will be backing up to a specific directory on the drive, you will want to browse into the drive keeping track of your folder hierarchy - in my case, it is:

/mnt/USB/USB2_c2/300m/Backups/

To create the above browsed into the folder:

cd /mnt/USB/USB2_c2/

Created my base directory:

mkdir 300m

Browsed into that folder:

cd 300m

And made my backup directory:

mkdir Backups

===========================

Now that you have your backup location, you need to know where the shares are…

In ssh type:

cd /mnt/HD/HD_a2

ls

And there they are… along with all your other files for your system  — if you get a directory not found, cd /mnt/HD/ and then do ls to see what your drive is called, then cd into that… and you should see your shares.

In my case, the utility WD provides is limited in that it will not let you backup the entire shares director - rsync will. If you want to backup a specific directory within a share, make note of the hierarchy to get to that folder like we did for the backups folder.

So now we should know the basic syntax for our rsync command - in my case to create a backup of all my shares that is synchronous :

rsync -r -av --delete /mnt/HD/HD_a2/ /mnt/USB/USB2_c2/300m/Backups

However, we want this to be run repeatedly - so we need to tell it to run as a cron job.

To understand cron - see the article here:

https://help.ubuntu.com/community/CronHowto

Open your cron tab typing in your terminal:

crontab -e

This will open in the hardcore editor - vi - so be careful - you can’t just type in this…

move your cursor to the bottom line using your arrow keys.

Press the letter “o” to inset a line below.

Remember cron jobs are laid out with each of the sections separated by a space, with the final section having one or more spaces in it. No spaces are allowed within Sections 1-5, only between them. Sections 1-5 are used to indicate when and how often you want the task to be executed. This is how a cron job is laid out:

minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday), command

So for mine, I want it to run my backup every day at 3 AM - so I typed in:

00 03 * * * rsync -r -av --delete /mnt/HD/HD_a2/ /mnt/USB/USB2_c2/300m/Backups

After you finish, press the esc key.

Type “:w” (without the quotes) to save the file.

Then “:q” to quit.

There you go, you now have a scheduled backup to your usb drive.

2 Likes

Thanks very much BG-300m for the detailed instuctions. It’s a good workaround.   I’m having the same problem trying to set up backup jobs in My Cloud Mirror. Nothing I do seem to get it to work and I’m on 1.05.30 firmware. 

Further to my post… there were two problems…

  1. some invalid file names… deleted those files and all was well with backing up that share

  2. one share with a large number of directories/files (15k+ directories, 45k+ files) was hitting a time-out on rsync (the procgram that runs to do a backup).  The default time-out is zero (unlimited) but WD decided that something less than that was appropriate and so they coded that in the backup command.  If you SSH into the box and run the rsync command yourself, the default, unlimited timeout, prevails, and the backup worked OK for me.  So I still can’t have it run automatically via the web interface, but at least i can run it myself.

So WD support is now saying that my box is defective and offering to replace it. I am having a difficult time getting the support org to explain why they feel this is a hardware issue and not a software one. I spent my entire working career in the high-end computer business, and I must say this sure doesn’t look like a hardware issue, although i’ve been wrong before.

When I look at the actual process running it is this:

14398 root      2496 S    rsync --timeout=30 --job-name=test!_usb -rlptDHq --delete /mnt/HD/HD_a2/test/ /mnt/USB/USB2_c1/test/test

and that fails with this message:

du: write error
rsync: writefd_unbuffered failed to write 29 bytes to socket [sender]: Broken pipe (32)
rsync error: timeout in data send/receive (code 30) at io.c(1530) [sender=3.0.7]

When I SSH into the box and run the rsync command without the timeout parm, it works.

So why is this a hardware problem that will be fixed by replacing the box?

Wasn’t sure if anyone might know the answer to this question. I have a NAS to USB backup running that has finished successfully. I set it up to synchronize instead of copy. The synchronization worked, all my files were copied successfully, but I’m trying to determine if I have to trigger this backup every time i add or delete or if the synchronization will continue to sync whenever files are added or removed from the Mirror. Earlier I added a text file to my NAS then waited to see if it would trigger the sync on its own but it never did. When I manually triggered the backup it did copy the file over. I was under the impression that the sync would initiate on its own. Am I wrong about that? Thanks everyone, I very much appreciate any advice and info. ghost.

No, you’ll have to trigger it manually.

If you want a timed recurrence, you’ll have to go in via SSH and set up an rsych/crontab combo along the lines of the code a few posts back in this thread. And even that’s not going to synch immediately when files get updated of course, just when the crontab triggers it.

And doing so may possibly cause warranty issues if things go wrong, as indicated by the warning you get if you enable SSH.

Darren is correct.  “sync”, in this case, means that when the backup runs it will sync the USB files with the NAS. The other option is “copy”.  “sync” copies only new/changed files and deletes files from the USB that no longer exist on the NAS.  “copy” will copy all files, whether they changed or not, onto the USB.( I haven’t checked what it does about deleted files. )

I think that’s a slightly different meaning to the word “sync” than what you have in mind.  :slight_smile:

Kicking off the sync job when a file on the NAS changes is a more complex process and not one that is available with this NAS.  When you think about it though, that’s exactly what the mirroring in this NAS does. Whenever you change a file, both drives in the mirrored pair are updated with the changes, so you always have a backup copy, assuming you are running with the RAID-1 option.

Thank you for your time sir, I very much appreciate the information.

Thank you sir. I had a catastrophic data failure loss a few months ago and learned my lesson the hard way about setting up a NAS to protect my data. I do have the RAID option turned on and it is mirroring so I’m good to go on that note. I also have the mirror hooked up to an APC in case of power loss, that is how I lost my data a few months ago. My Buffalo NAS Linkstation wasn’t plugged into an APC, storm, power loss, some kind of activity was going on with the NAS when it happened, and, data corrupted…I’ll just trigger the USB backup myself every once in a while to make sure anything I copy gets copied over to the USB. Thanks again sir.

Just to play devil’s advocate for a moment, the RAID1 MCM set-up is better than a single disk, but it is still a little susceptible to problems. For example if you have a power outage at the wrong time, or a burglary , or indeed a physical catastrophe which affects the unit (power spike, flood, fire, dropping/damage etc) it could be that both discs are affected as they’re in the same place and driven by the same hardware and circuitry. Hence using USB is a better option sometimes as it’s two distinct drive electronics, especially if the USB drive is then kept separately from the NAS (or stored somewhere like a fire safe).

RAID arrays are designed to overcome/survive individual drive failure, but are still open to issues which could affect both drives at once. Depending quite what you are storing on there (e.g. company-critical info for a business for example) then it may need consideration.

Probably not applicable in this case, but as a general comment it may be worth including.

That is definitely sound advice sir and I appreciate it. What do you think about possbily purchasing another WD My cloud storage device that would connect to my router like the mirror doe? I was curious if I might be able to set up a remote backup from the mirror to another My Cloud device. The remote backup feature looks like it has incremental and scheduled back up features available. Thanks again for the info. Have a good weekend.

There are 4 options generally available:

  1. Do nothing, and just use the RAID1 in the MCM to cope with drive failures but take the risk of physical or mechanical issues which affect both drives simultaneously.
  2. Buy a second MCM and use the internal “Remote Backup” option to backup from one to the other.
  3. Use the internal “Cloud Backup” option to back up to either an Elephant Drive or Amazon S3 account.
  4. Hang one or two USB3.0 drives from the MCM and use the USB backup option to back the MCM up to the USB drive (and optionally then remove that drive and store it safely elsewhere).

Which one is required and best is very much a case-per-case basis decision, depending on individual circumstances, and the value/importance of the data stored on the MCM.

Personally for example I just use half of my MCM (it’s in JBOD mode) for PC backup (via Smartware) and the other half for media storage and serving. That second drive is duplicated on a 5TB WD elements drive which is connected to the MCM via USB (and indeed much of it also on a MyPassport which I take with me when travelling). But in my case data loss would be an inconvenience rather than a disaster, so I don’t lose sleep over the risk.

I received an update yesterday from WD on this issue.  They have accepted it as a firmware bug and are working to fix the issue in the next firmware release. There is no release date available. 

Until such time, I can easily run the rsync command myself, so I’m happy with this course of action.

It does appear that the level 1 tech support folks are working from a faulty script though… basically a set of steps to try various things, and if the problem isn’t resolved, replace the box.   This is fine for hardware problems, but does not address software problems, since the replacement box will have the same software.  Given the amount of software in these products, WD may wish to consider providing better guidance to their level 1 tech support folks.

…just wanted to let the rest of you know the outcome.

1 Like

DarrenHill wrote:

There are 4 options generally available:

 …

  2.Buy a second MCM and use the internal “Remote Backup” option to backup from one to the other.

  4.Hang one or two USB3.0 drives from the MCM and use the USB backup option to back the MCM up to the USB drive (and optionally then remove that drive and store it safely elsewhere).

Hello,

I don´t have experiences about backups to a second MCM via LAN. The risk of loosing both system by flood, storm, fire, theft,… may decrease by the distance, but still be there.

My experiences about backups to a USB-HDD are very negative. I had to find out that the system reacts very delicately to loss of power. In my case the power of the USB-HDD was lost without logging off the USB-Drive in the dashboard. I am rather sure that the HDD even was in sleep-mode when it happend. After that the USB-Drive worked fine with my PC and also was recognized by the MCM. But the MCM was unable to connect to the Drive as a share. Attached to the USB-Slot of the MCM the drive also could not be used over LAN.

Even after several phone-calls and many emails the only suggestion WD Support could make, was to reformat the USB-HDD.

After doing so, the HDD works fine again as a backupdrive for my MCM. That means all safety was lost for time of reformatting and doing the new backup job.

WD told me that they knew about that instability of the system in case of improper disconnection. They expressed their deep regret on that fact, and they promissed to work on that issue. But that happend almostly half a year ago and I still haven´t heard about a solution since then.

Sorry, but for that reason I can´t advice USB backup to a single HDD as a safe and convenient backup method.

Greetings,

  Wein-Franke


Update:

Meanwhile a member of the German WD community found out, that a system file “.smbm.xml” in root directory the USB-Drive seems to be the cause of the trouble. In some cases it helped to replace it by a previous copy. But yesterday I had really serious trouble by doing that. I had to reset the MCM to factory default and reset the USB HDD again.

Wein-Franke

I see this thread has been running for quite some time and I wonder if anyone at WD is taking any notice!? To be able to backup your data from the My Cloud Mirror to an external USB drive is a stated feature and this NEEDS TO WORK. I have recently had a MyBook Live Duo fail on me and while WD were kind enough to have my data recovered, the drive was away for weeks and I’m yet to copy stuff back to how it was structured after they dumped everything into the Public folder. As a result of the major pain I just went through they recommended to backup to another external drive - which is a bit of a laugh since the MCM is my backup and using RAID0 I would have thought I’m covered - but if this backup to external doesn’t even work, what’s the point?

I have connected my WD My Passport for Mac, 3TB drive, it is recognised, I’ve setup backup jobs first for several then only for one folder and NOTHING HAPPENS. First of all, when I click on OK it just sits there. No confirmation. No error. It just eventually drops back to the login screen and when I login again to see if the backup job has been saved there is nothing ans also no data arrives in the USB drive, which shows up as another share under My Cloud in the Mac Finder…

THIS NEEDS TO BE RESOLVED! COME ON, WD, HOW HARD CAN THIS BE!? WE’RE TALKING ABOUT COPYING DATA FROM A TO B. THIS ISN’T SOME ROCKET SCIENCE TASK…

I look forward to an update ASAP.

I’ve noticed a few things regarding backups  using rsync

  1. When I connect the USB drive, it becomes a share with Media Server enabled by default.

     I do not want Media Serving my backups so I disable it.

  1. Mirror doesn’t like heavily populated USB drives formatted with NTFS (don’t know why - just is)

SOLUTION:

My backup USB drive is formatted in FAT32 and works great for the intended purpose.

Good evening,

Your comments regarding USB related backups are being escalated to WD Engineering Teams.

Regards,

Samuel Brown

I think I found the problem.
I duplicated the similar backup (from NAS to USB drive, which is attached to NAS port) that failed perviously as many of you has experienced.

When I tried to copy from the NAS to a USB drive (formatted NTFS) using a windows machine it also failed. USB drive was not attached to the NAS but it was attached to a windows machine. I was transferring over the ethernet to a USB attached to windows machine. It gave me an error 'Path too long". So NTFS has a maximum path name limitation. WD won’t tell you that. It is similar to copying files bigger then 4GB to a FAT drive. It will fail. WD should have warned us why it failed!

So other options are formatting the external USB drive (attached to NAS) to EX4/EX3 if you are using unix or Mac format HFS + J. They seem to have less restrictions.

So I think what is happening is we are filling our NAS drives by copying files to a longer paths. then it is hard (almost impossible )to copy back NTFS drives!.

When you use bigger data (tests says 40-100GB etc), it seem to have failed due to the full size of the back up but it is failing because statistically there is a higher chance of having a longer (incompatible) pathway.

I will keep you posted with more detailed updates.

Do I get a job at WD now ?

Update 1: It has been 4 hours …Still working !No error message…

UPDATE 2 : IT DID NOT WORK FOR MAC FORMATTED EXTERNAL USB DRIVE!

UPDATE 3: TRYING LINUX EX4 USB EXTERNAL DRIVE FRO BACKUP…

UPDATE 4 : LINUX EX4 DID NOT WORK and GAVE THE SAME ERROR (IT WAS ALSO TOO SLOW)

UPDATE 5: There is a new firmware update , claiming it solved the problem (firmware date 3/29/2016). Has anyone tried it? Thanks…

Hi thanks for the update. I too have been having this problem and have just reset my 4TB My Cloud and copied 2TB of info back on to it. I was about to re-format my back up drive to HFS+J but your post saved my from doing this.

I have raised this issue with WD and the said it would be raised to a level 2 issue but having read back through the posts this just seems to be something they say as this problem has been going on for over a year.

Quick question for anyone having the same problem.

Prior to the latest up date which introduced the “Backup” process was there the “Safepoint” feature and if so did this work?

Prior to a Mycloud I had a Mylive Duo and the Safepoint feature was excellent and worked great, never had any issues with it. I’m wondering if the Safepoint was a feature on the My Cloud and it worked can’t they just put it back in???