Use SSH or FTP to copy files on WD My Cloud

My Cloud OS 3 Personal & Network Attached Storage My Cloud
1

I have a single bay WDMYCLOUD running firmware 04.05.00-353 on a local Windows 10 network and with a direct port enabled connection.

Bulk File Copy and File Backup are routinely failing. I get to get the data off the disk while I still can.

It has never been fast, but though I can still open and use files stored on it, and my Media server can stream satisfactorily from it, any attempt at bulk copy, bulk move, backup etc is a level of painful slow that is just insane. Though the movies are replaceable, it has just under 200,000 data files that i have to get off it, and the backup has not completed successfully for so long I would have no clue where to start identifying what my backups are missing.

I have listed below the various steps and checks i have taken to try and figure out what is causing the problem, but i have a gut feeling it is just on its last legs. I know using SSH without skills is a bit like giving a chainsaw to a baby. My fear here isn´t helped by what WD said would happen when I first logged in (request for user to update password) not happening. I logged in and out successfully using PuTTY on Windows 10 but no request to change password. As I don´t know Linux security I have disabled SSH on my WDMC web interface again while I figure it out.

There are some very good instructions by @Tfl on a popular thread re SSH and WDMC, but they are written using a Unix something. There is enough in that thread to tell me I need specific advice on how to do this data extract;

Environment: from a Windows 10 workstation, using Putty and connecting to the WdMyCloud by ip address using Root user information it provided.

Linux WDMyCloud 3.2.26 #1 SMP Thu Jul 9 11:14:15 PDT 2015 wd-2.4-rel armv7l
WDMyCloud:~#

Task: Copy all files (2 password protected user Shares and one Public Share) to a location (either its USB port or my LAN - doesn´t matter which)

Security: This bit isn´t essential, if i can just get the files off I´ll turn off SSH again if necessary. But if someone has the time to tell me how to do the 2 key bit, that would be super appreciated.

Thanks so much for reading, and if you have the same my cloud you should probably do the same - I think they are all set to self destruct on April 15th :cowboy_hat_face: :disguised_face: :ok_hand: :fist:

Heres the steps I´ve taken to try and get the copy to work. …

I´ve checked every single thing I can, running the copy from different workstations, utils on the disk, disk check, cable swaps, etc on the wdmc itself, lan transfer speed check, and everything is gigabit. But any level of file copy is a level of insanity slow. Quite often it just pauses, when it is running it is at sub 100kb, and yet the machine doing the work is showing no stress whatever - cpu loads under 10%, nothing is at anything approaching bottleneck. The one thing i haven´t done is a db rebuild, but tbh i don´t want to. It just feels so fragile, so deathbed, that i feel a midpoint fail could brick it.

2

Using SSH to copy My Cloud Shares to a USB hard drive is a streight forward process. You’ll need to know the Share name (if copying selected Shares) and the mounted USB drive name.

On my first gen v4.x single bay My Cloud I’ve used the following command(s) to copy Shares to a USB hard drive attached to the My Cloud. Here is an example of that command. It is copying the contents of the Public Share to a similar folder on a USB drive (called Hitachi_750GB)

sudo rsync -rtvh --progress /DataVolume/shares/Public/*.* /var/media/Hitachi_750GB/Public/

Another example, this time copying just MP4 video files to a Toshiba Canvio drive using the nohup command.

nohup rsync /DataVolume/shares/Public/*.mp4 /var/media/Toshiba_Canvio_1TB/

Without the nohup command one would have to leave their SSH client (Putty, etc.) running the whole time the files are being copied. By using nohup one can close the SSH client and the copying will continue till finished. One can reconnect using SSH and issue the top command to see if nohup rsync is still running or has finished.

Or just extract the My Cloud hard drive from it’s enclosure, attach it to a computer running Linux or running a Linux boot disc/flash drive and copy the contents that way. If using Windows one will need to use a third party Linux driver to read the contents of the My Cloud hard drive since it’s formatted for Linux. There are inexpensive powered USB to SATA adapters if one doesn’t have a a spare SATA port on their computer.

3

Star @Bennor , thank you so much, this is exactly the level of instruction i was hoping for.

I need to set aside some time for linux commands 101 at some point soon - it is coming up more and more often as a restrictive factor in running my media etc servers, but for now it means i can sleep sound in the knowledge i´ve got all of some very precious data off this drive.

Good work fella, if you´re ever in Ibiza on your hols, look me up for drinks and favours :fist: :ok_hand:

5

Thank you for this information! Prior to os5 Backup to a USB drive worked flawlessly. With OS5 the backup up app fails every time for me and has no easy way to view error messages or logs.
SSH is definitely the way to do this now using rsync.
I was trying to copy 2TB of data onto a USB drive and the backup app would run for 5or6 days and fail.
Using rsync through SSH the copy finished in about 5 hours with no errors.
One question, when I used *.* on the source directory it skipped a bunch of files but when I ran the command with just * it copied everything.

example:

sudo rsync -rtvh --progress /Source/*.* /USB/

skipped some files.

sudo rsync -rtvh --progress /Source/* /USB/

worked.
Whats the difference between *.* and *

6

One can use their favorite internet search engine to learn about using wildcards with rsync.

Generally using wildcard like *.* will tell rsync to copy or sync all files (example: xyz.doc) to the new location. If a file doesn’t have an extension (example .doc) then it likely would not be copied. Or as my example above shows if one wants to only copy one type of file then can, for example, use the wildcard *.mp4 to copy or sync only MP4 files.

7

Do not use *.* if you want to copy all the files in a shared folder, use *. The first will copy files if the file name contains periods, the last will copy everything.