currently generating a backup of all my shares using OS 5’s USB Backups app.
i plan to install new drives in the NAS when that is done.
Do I have to recreate the Private shares on it before restoring from the USB backup, or does the restore process add the Private shares back onto the NAS ?
Assuming with the Public share, it’s straightforward and everything within the backed up Public will be put back into the new Public.
any tips or good practice to look out for when doing this ?
what file handling protocol does the UBS Backup app for OS 5 use ? raync, cp …
does the app do checksumming or verify the files after transfer ?
i dont quite know how a restoration from the backup would work in detail, have never done it before. i am willing to manually transfer files back to the NAS using rsync, if thats safer than using the app’s restore option.
any advice would be appreciated, thank you
For those who had the same questions about this that I do, having gone thru this process just today, I’ll answer my own post and hopefully that will spare others who keep searching and get no answers, neither from experienced members on this forum, nor from WD Staff who probably know the answers.
Firstly, removing your drives to swap with new or anything along the lines of recreating the Volume on the EX2 WILL wipe out the apps you have installed !! proceed with caution on that, if you have apps that you may be relying on in order to restore or recover stuff you put on an external drive before doing your swap. That includes WD’s own USB Backup app. I am somewhat surprised that happened, thinking apps and app data would be stored somewhere on the board and not the drives which can be swapped by a user. Disappointed about that, but i’m not at a complete loss – and I know some disgruntled forum trollers on here would read along and drumroll hoping to hear I lost all my data so they can laugh – i have 3 duplicates of my data, so I feel comforted about that.
Since you WILL lose the USB Backup app entirely, when you swap new drives into your ‘lovely’ EX2 enclosure … you will need first need to recreate all the share structure you had before. I recommend you create all your shares from the dashboard, set your permissions, options etc. at this point in time. It is the fastest and easiest way to do it. Do not try to copy a backed up share onto the NAS it will just be a folder and not a proper share.
Brush up on using terminal/putty, using ssh (enable it in the dashboard first), and basic linux copy commands such as cp or rsync. I chose to put back all my files using cp, from USB → EX2. To do this, you first ssh into your NAS, then issue a few cd.. commands until you’re at the root folder and can see the mnt directory. if you ls that directory and recursively look in HD and USB you will be able to locate the share folders (in HD) as well as the contents on the USB you want to put back on your NAS.
Note the paths of your source (USB) and destination (NAS), write them down if you want or just list them in a terminal window so you can reference. begin to copy your stuff using the command of your choice back to the NAS. I chose to use cp with the -rvT options. I chose these because I wanted to copy only the contents (sub-directories and files, recursively) and without copying the source folder to the destination (that is what the T option is for). So I did this for all my shares, from the smallest size share, to the largest, using individual cp commands for each share.
Just a word of caution, it would be wise to keep the computer you’re using to ssh awake/on while the copy is working, and keep the terminal/putty obviously open do not close or exit out while it is copying.
i switched from a single drive WD red to a dual drive Brand X 7200 RPM RAID 0 configuration. the performance gains when accessing files from file explorer (or finder) are VERY noticeable, and ssh copy operations look they are on adrenaline compared with the outgoing single-drive 5400 RPM arrangement. It will still take some time to copy TB’s over, but i estimated that it took about 1/3 the time to manually restore my NAS content now as opposed to the pretty sad 50+ hrs that it’s taken in the past.
although impressed by the performance boost i got, i was let down somewhat and in the dark completely with no detailed information in the manual about scenarios users would encounter outside of the ‘initial setup and get going’ that you find provided by WD. They tell you absolutely nothing about what happens to your configuration, apps, etc. when you change the Volume type or try to add disks in JBOD and cross your fingers hoping you’ll still have access to what you’ve already configured. obviously changing disk arrangement erases content, but everything a user can setup in the dashboard in my opinion should be intact and shouldn’t be dependent on the drives in it to store settings.
as i suspected, the USB Backup app was useless for a recovery. WD could have done a better job describing how it actually works and point out that YOU CANNOT use the RECOVER button after creating a new volume with new drives – because the freakin’ USB Backup app disappears, along with the Job(s) you created ! so when you swap drives in the NAS, prepare to copy your stuff back to it the manual way such as I described. if you wish to forego using ssh and prefer to do it with file explorer or finder … i very strongly recommend that you directly connect the NAS and backup drive to your computer to copy files as opposed to doing it over a network or wifi – copying over wifi will take a very very long time if you have TB’s of data
Good stuff.
Just a question: Do you think the performance boost is more attibutable to the 7200rpm drivers, or the Raid0 striping of the drives?
Just a few more questions: Not doing TB of transfers over WiFi is obvious (at least to me). . if you have a GB network; what kind of speed difference do you think there is on direct connect vs wired over the GB network?
Another observation: In the older MyCloud software, there was a “button” to “save” your software configuration. . . . I would have presumed that using that feature would allow you to restore your user configuration. . . including the backup jobs??
having been through this process on a few WD NAS systems your write up is much better than my old notes. ( Thank you )
(1) When they changed from build in backup to app backup to USB I had the problem of needing to re-do settings and have always done that by re-typing. I do save the config files but if you have even done Motherboard BIOS firmware updates they are almost useless on a different firmware number.
(2) Raid 1 for EX2 units and Raid 10 for PR units for the extra write speed
(3) restore takes a few days and you may lose a RAID disk unit so I have extra disk handy an well as at least 2 full backups before starting and they can take a day to make a copy.
(4) One time with the older OS3 firmware on the same NAS unit I did try just using the restore option
on the USB backup and had no problems. I was only changing disk size.
–
Before doing a restore or re-placing an old NAS unit - I have original data backup via Acronis (pc) or CCC ( mac) and at least 2 backups to external USB disks via the WD backup.
Best to adjust the Raid vol so it fits your backup disk when setting up the system.
Thanks again for your write up – would have save me from having to get a second NAS unit to test how restore works,
( that second unit is now used to test OS5 )
additional notes
some additional notes to provide you. when putting my things back onto the NAS with new drives installed and subsequently brand new blank-slate Shares, be careful with file permissions. when i put everything back, I noticed that the Share folders contained within /shares carry root:root as owner:group. These are the physical folders associated with the shares created in the dashboard. since I copied sub-directories and contents into these folders, the permissions for those items were not inline with the user privileges according to each share. that caused some problems with regard to mounting and reading/writing from the shares from Windows or macOS.
to resolve this, I had to ssh back into the NAS and reassign permissions to all the sub-folders recursively, so that all the folders and files I added back were assigned the owner:group that allowed me to work with the shares when they are mounted in either windows or macos. the important thing is to make sure you map or mount with the same user credentials you have assigned read/write on the respective shares in the dashboard – otherwise the content will be read-only when you are accessing it in File Explorer or Finder.
to make things simple for me, i setup all my shares to be accessed via one user, and that way i assigned the same owner:group to everything using chown.
i caution on using this method, you must know what you’re doing and be certain you’re hitting the content with the correct owner/group. don’t change owner:group for any of the folders the NAS created that are associated with the shares themselves.
i don’t know how much more one factor contributes vs another in terms of performance. i think they both contribute but moreso the RAID arrangement. data will write faster to a stripe, in theory. going from one disk to n disks in RAID 0 would probably carry with it some order of n increase in speed. RPM’s probably contribute as well but don’t think as much as spreading the read/write duties to n disks. i’ll guess and say a 4 disk RAID 0 will be at least 2x as fast as a 2 disk RAID 0.
speed vs. connection type, idk honestly. i think ethernet is faster than USB ? if you connect the NAS to a computer via ethernet and another device to the computer via USB, throughput is only as fast as the slower connection.
the save configuration feature is still there, but i did not think to restore it. at the point i am now, somewhat of a headache configuring everything in ssh as well as matching up users in the dashboard to the shares … so i probably won’t restore the configuration if there’s a chance it will mess up the changes i made to the users/shares.
no problem, i wanted to post to help at least some folks …
i am somewhat cautious about restoring config files, whether it’s this NAS or a Router. sometimes the config files carry over with them ‘stale’ bits that might not stick on a newer firmware or might cause a problem that’s transparent. looking back, i was better off this route and just keep config backups for last-resort but rarely ever did a config file restore.
i didn’t explore the raid setup you mention but it sounds interesting. my limitations where the cost associated with getting some beefy drives that will give me the space i wanted and do it in RAID 1 as well. instead i architected my setup in RAID 0 to maximize storage with smaller drives and RAID 1 to maximize redundancy with larger drives for backups. keep almost everything living on the NAS as the primary copy of data. even if i combined all of my household computers’ storage, i would not be able to match the storage i have on the NAS. and so data/content/media reside there, and i use other backup methods on my computers to do incremental daily jobs to save all other data that lives on my computers. all of the daily pc/mac backups i do go straight to my NAS. so when I run the backup app every day or so, everything that changed/added on the NAS is duplicated twice on the RAID 1 USB enclosure. as added piece of mind, the USB RAID 1 setup is formatted to HFS+, and I will connect an additional USB drive formatted in ext4 to the NAS as well (so, both USB ports will be used). it will be used to spin off a 2nd independent backup job and that will give me a total of 3 duplicates of my NAS shares.
my setup sizing is not perfectly matched, and the backup array is smaller than the NAS array, but when space gets tight, i will swap the backup disks into the NAS to make a larger RAID 0 and buy additional disks to design a new backup solution. i currently have 12TB on the NAS (2x6TB) and 8TB on the USB RAID 1 (2x8TB). I still have about 3TB of free space on the USB after backing up, which is a few years or so for me. another caution to anyone reading this: i think my original MyBook backup drive got corrupted because I was cramming almost 5TB of stuff onto a 5TB WD Blue drive. The whole thing was probably overfilling but showed me I had about 50GB of free space on it when I found out it pooped. this time around, i would swap out the backup disks when i reach 250GB of free space remaining … or 1/4 of a Terabyte, just for piece of mind.
recovering my files via cp I cannot regret at all, it was really that fast. 5TB was copied over from the USB drive and total working time in the terminal I would say about 10 hours. i am estimating, because i had to manually cp each share. the largest share i started copying last night, but this morning i noticed that the NAS auto-updated the firmware overnight so it’s killed to copy process that was cooking overnight in ssh. just had to go back and recopy that segment of the job for confidence that no files were corrupted.
that’s another thing about swapping to new volumes, the firmware auto-update switch toggled itself back on – go figure.
That’s a fair bit to digest. . . gonna take a few more reads to properly comprehend.
Two thoughts up front:
Never have auto-update turned on 
(I drives me wild when that happens on Win10 – -always at the wrong time)
(I don’t believe in being an early adopter of new firmware. . . .I will let others test it to see if it’s ok)
I hear you on the “stored configuration” restore. Personally, I like occasional “fresh starts” to clear out the cruft (admittedly, I don’t actually do it that often). I recently realized my main desktop machine has 6 years of turbotax installed. . . which tells you when I last updated the O/S. . . . . in the same vein. . . the real challenge is configuring a system and installing software when you are actually NOT on your home continent.(Don’t ask me how I know this)
sorry for the scripture … i think detailed and my thoughts don’t materialize in a fluid way when i write stories like this …
agree w you, i always keep manual control of my hardware. one thing i hate about windows 10 is it’s update feature being automated by default unless i purposefully break it - then it becomes a pita to fire it up again every once in a while in order to get updates when i want them.
with my arrangement setup on the NAS, i started a new USB backup job, to get that established to the point where my daily syncs are small chunks that don’t take more than an hour to do.
maybe a better way for folks to visualize what i’m doing in the big picture:
install new drives and configure NAS again → copy everything from USB to NAS → fix/chown file and folder permissions on the NAS via ssh → delete everything on the USB → start a fresh USB backup job NAS → USB. in that order.
during this first run, while it is still cooking the first backup, i went into ssh to do a top, to see what it’s really doing:
rsync --timeout=60 --job-name=NAS_BACKUP_1!_usb -rlptDHq --delete /mnt/HD/HD_a2/Public/ /mnt/USB/USB1_c2/NAS_BACKUP_1/Public
top output wasn’t as detailed as watching the verbose output of cp in terminal (where i could see which file it was copying at that point)
in the above, it is now working on the entire Public share (about 4TB). and that answers my other question above - it is using rsync.
| Forums | News and Announcements | Register | Help | Forum Guidelines |
Copyright © 2021 Western Digital Technologies, Inc. All rights
reserved.
|
Trademarks
|
Privacy
|
Community Terms of Service
|
Site Terms of Use
|
Copyright
|
Contact WD
|