Clone NAS hard drive to SSD

Graeagle · August 7, 2018, 2:58pm

The MyCloud motherboard is out of the case and the wand is quite bright and appears more-or-less blue. However, if I look at it head-on it might very well be showing more reddish than blue so one might conclude from that that the MyCould has not been able to boot.

With the Netgear router, it will display among other things the MAC addresses of all attached devices, both wired and wireless. With my 2nd “production” MyCloud un-plugged, there are no WD MAC addresses showing at all, i.e., “00.90…”.

The MyCloud IP addresses are dynamic.

Graeagle · August 7, 2018, 3:51pm

BTW, I haven’t yet studied your script but two questions come to mind: (for Wednesday)

Are both source and target devices to be SATA connected?
Will the script come “un-glued” when it finds the target disc smaller than the source?

cpt_paranoia · August 7, 2018, 7:43pm

Yes. Connect things exactly as you have been doing up to now. The script runs under your linux environment, in a command shell, and only does what you have been doing manually, only it eliminates the possibilities of manual typos, mis-spellings, missed commands, etc. The only thing you have to get right is identifying the target disk, and copying the tar.gz file to /mnt, which you now seem you have mastered. Bear in mind that it is written to use the original_v04.01.02-417.tar.gz file, not the v03. If you really, really want the v3 firmware (and I can see little reason why you would, given your inexperience with linux), you will have to edit the script to put in the relevant filename. I recommend you don’t fiddle with it.
By using '100% for the 4th partition, it will automatically adapt to whatever size of disk you use as the target. Provided it’s bigger than 8GB…

cpt_paranoia · August 7, 2018, 7:45pm

Here is the log of one of my sessions, using a disk that had previously been cloned, but had all the partitions removed. Some of the warnings are because parted found the remnants of these partitions (it’s clever like that),

root@sysresccd /root % fdisk -l
Disk /dev/loop0: 469.9 MiB, 492683264 bytes, 962272 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sda: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x264023b8

Device     Boot   Start       End   Sectors   Size Id Type
/dev/sda1  *       2048   1255423   1253376   612M  7 HPFS/NTFS/exFAT
/dev/sda2       1255424 976771071 975515648 465.2G  7 HPFS/NTFS/exFAT


Disk /dev/sdb: 1.8 TiB, 2000398934016 bytes, 3907029168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: A92F95D5-AB94-4858-8863-0836F6BC47B7
root@sysresccd /root % chmod 777 clone_wd
root@sysresccd /root % ./clone_wd b
./clone_wd: line 1: #!/bin/bash: No such file or directory
This will create a WD MyCloud clone HDD on /dev/sdb
IT WILL DELETE ALL EXISTING DATA

Are you sure (y/n)? y
Creating clone on /dev/sdb

the drive should start with no partitions
/dev/sdb contains the current partitions:
Model: ATA WDC WD20EFRX-68E (scsi)
Disk /dev/sdb: 1907729MiB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start  End  Size  File system  Name  Flags

Creating new partitions                                                   
Warning: The existing disk label on /dev/sdb will be destroyed and all data on
this disk will be lost. Do you want to continue?
Yes/No? y                                                                 
Information: You may need to update /etc/fstab.

Information: You may need to update /etc/fstab.

Information: You may need to update /etc/fstab.                           

Information: You may need to update /etc/fstab.                           

Information: You may need to update /etc/fstab.                           

Information: You may need to update /etc/fstab.                           

Information: You may need to update /etc/fstab.                           

Information: You may need to update /etc/fstab.                           

Information: You may need to update /etc/fstab.                           

                                                                          
Cconfiguring partitions 1-4
Information: You may need to update /etc/fstab.

Information: You may need to update /etc/fstab.                           

Creating RAID array on clone                                              
mdadm: error opening /dev/md: Is a directory
mdadm: stopped /dev/md0
mdadm: /dev/sdb1 appears to contain an ext2fs file system
       size=1999808K  mtime=Tue Aug  7 12:03:24 2018
mdadm: /dev/sdb1 appears to be part of a raid array:
       level=raid1 devices=2 ctime=Tue Aug  7 11:57:29 2018
mdadm: /dev/sdb2 appears to contain an ext2fs file system
       size=1999808K  mtime=Tue Aug  7 12:03:24 2018
mdadm: /dev/sdb2 appears to be part of a raid array:
       level=raid1 devices=2 ctime=Tue Aug  7 11:57:29 2018
Continue creating array? y
mdadm: /dev/md0 is already in use.
Setting up swap and data partitions
mkswap: /dev/sdb3: warning: wiping old swap signature.
Setting up swapspace version 1, size = 512 MiB (536866816 bytes)
no label, UUID=d48f7b7d-eec5-4f4a-9803-d9a80fb3eedf
mke2fs 1.43.6 (29-Aug-2017)
/dev/sdb4 contains a ext4 file system
	last mounted on /mnt on Tue Aug  7 11:49:16 2018
Proceed anyway? (y,N) y
Creating filesystem with 487142656 4k blocks and 121790464 inodes
Filesystem UUID: 947ad9ea-e3df-4217-9fed-197aeed93200
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 
	102400000, 214990848

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done       



Partition creation completed
partitions are created as follows:

Model: ATA WDC WD20EFRX-68E (scsi)
Disk /dev/sdb: 1907729MiB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start    End         Size        File system     Name     Flags
 3      16.0MiB  528MiB      512MiB      linux-swap(v1)  primary
 1      528MiB   2576MiB     2048MiB     ext3            primary  raid
 2      2576MiB  4624MiB     2048MiB     ext3            primary  raid
 5      4624MiB  4724MiB     100MiB                      primary
 6      4724MiB  4824MiB     100MiB                      primary
 7      4824MiB  4826MiB     2.00MiB                     primary
 8      4826MiB  4828MiB     2.00MiB                     primary
 4      4828MiB  1907729MiB  1902901MiB  ext4            primary



Mounting data partition for working area
/mnt created

Please copy original_v04.01.02-417.tar.gz to /mnt

press 'y' when you have done this y
Extracting images
./
./kernel.img
./rootfs.img
./rootfs.md5
./config.img

Copying images to partitions 5-8
196608+0 records in
196608+0 records out
100663296 bytes (101 MB, 96 MiB) copied, 7.9155 s, 12.7 MB/s
196608+0 records in
196608+0 records out
100663296 bytes (101 MB, 96 MiB) copied, 6.935 s, 14.5 MB/s
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.286197 s, 3.7 MB/s
2048+0 records in
2048+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0569286 s, 18.4 MB/s

Starting RAID array
mdadm: stopped /dev/md0
mdadm: /dev/md0 has been started with 2 drives.
Copying rootfs
this could take a while...
3999616+0 records in
3999616+0 records out
2047803392 bytes (2.0 GB, 1.9 GiB) copied, 169.645 s, 12.1 MB/s

Preparing for first boot
umount: /mnt: target is busy.

Finished!

now connect HDD to MyCloud PCB
connect ethernet and apply power
should get white light, followed by blue light after 5-10 minutes
do a 40-second reset when the LED turns blue
wait for blue LED, then access Dashboard
upgrade to whatever firmware you want
do another 40-second reset when LED turns blue
configure device as required
root@sysresccd /root %

Graeagle · August 8, 2018, 12:44am

I’ll tackle the process in the AM but I wanted to clarify a point. The past week or so has been an exercise in de-bricking a SSD so that it could be placed under the MyCloud motherboard in the MyCloud housing and boot it up. Previously, Wierd_w had been helping with the attempts to “CLONE” a working 2TB WD MC to the SSD using a cloning scenario but simply couldn’t get that to work. I think it was Bennor that suggested that I abandon the cloning approach and simply de-brick the SSD. My understanding of your script is to clone my healthy 2TB WD Hdd with both source and target connected to my SATA PC. Correct?

From debian’s “point-of-view” that will likely be /dev/sdb being cloned to /dev/sdc, correct?

Like this: (Taken from your instructions shown in post 126)
./clone_wd /dev/sdb /dev/sdc ?

cpt_paranoia · August 8, 2018, 11:06am

NO

My instructions and example log file don’t show anything like that. Go and read them again, carefully.

‘clone’, ‘unbrick’, ‘de-brick’, whatever we call it, it’s creating an HDD with a working MyCloud image on it. In the case of my script, cloning from a tar file, downloaded from fox_exe’s repository.

My script should NOT be used to de-brick (restore to operation) an existing MyCloud HDD, because it creates different sized partitions (because it uses parted as originally intended, rather than incorrectly, as WD have clearly been doing for years). This would damage the existing data partition, hence the WARNING that it will do that.

Put the target SSD in your linux machine.
Boot the linux machine with a graphical desktop environment, and log in as root.
Use Firefox to download the original_v04.01.02-417.tar.gz file
Copy the script from the post above, and paste it into a text file, call it clone_wd, and save it in /root
Open a console window, and cd /root
Make the script executable by chmod 777 clone_wd
Use the command fdisk -l to identify which is the linux HDD/SSD, and which is the target SSD. It is likely that your linux boot disk will be /dev/sda, and your target SSD /dev/sdb
If the target is /dev/sdb, use the command ./clone_wd b
Respond to the prompts from the script, including copying the image file to /mnt

Graeagle · August 8, 2018, 3:42pm

Sorry, I know you’re just wanting to help with this caper.

I was just trying to decipher “(e.g…/clone_wd a for /dev/sda)” given a mindset of “copy from here to here”, “move from here to here”, “rename this to that”, “clone device1 to device2”, etc. Simply expecting a function invocation more consistent with most programming languages and didn’t want to screw up the works.

Bill
(PS) My Linux is installed on an old HDD and it always shows up as sda so I’m expecting that I’ll be cloning b → c. There are three (3) physical discs involved when Linux boots up, Linux sda, the MC HDD and the SSD. ALL three are connected to SATA ports on my PC motherboard. Looking at your instructions, it’s not clear how your ./clone_wd is making the distinction among the three.

cpt_paranoia · August 8, 2018, 5:30pm

You don’t need your original MyCloud HDD plugged in. We’re not using it for anything, and it’s safer not to have it plugged in, so you can ruin it by mistake…

My script uses the disk you tell it to use. Again, read the instructions. I can’t see how I can make them any clearer:

or, as stated in the script:

# e.g. for /dev/sda, use 'clone_wd a'

The ‘quotes’ and code formatting are used to distinguish between English and linux command.

Since you’re familiar with programming languages, you will see that I construct the full device path from the first command line parameter, using

# set drive name
drive="/dev/sd"$1

and then construct the partition identifiers in a similar way, e.g. ${drive}3

Graeagle · August 8, 2018, 5:51pm

Linux won’t let me login as root. From the logon screen: “Sorry, that didn’t work. Please try again”. I use the pw established at the time I installed Linux and it is the same as used when I issue the su command when using the terminal console.

BTW, to me “clone” means to make an exact copy of something I already have so I naturally expected the MC HDD to somehow become part of the process.

Anyway, until I can save clone_wd in root I seem to be dead in the water without being able to log into root.

cpt_paranoia · August 8, 2018, 6:07pm

su sets you as root. That will do.

You can save it anywhere. You just need to cd to the directory you saved it in, and then do an su to become root, and have privileges to write wherever you want.

Graeagle · August 8, 2018, 6:14pm

That’s what I had hoped from the beginning but keep bumping my head on trying to copy the script into /root from the file manager. I couldn’t do that even when using su at the terminal console. Anyway, the script is sitting in Documents and I’ll go from there.

Graeagle · August 8, 2018, 6:58pm

Going great up until Linux will not let me move and/or copy the V04 from Downloads to mnt at the prompt. So, I thought I’d try to download the V04 file again directly into mnt via Firefox. For some reason, I can’t get Linux to connect to my wired network connection to redo the download.

I will likely have to reboot my PC. What does that do when restarting clone_wd?

Graeagle · August 8, 2018, 11:11pm

I looked at permission properties of /mnt and with being logged in via console su I still cannot get V04 to store in mnt, either with a copy/paste OR direct download via Firefox.

Screenshot%20from%202018-08-08%2011-33-41

cpt_paranoia · August 9, 2018, 12:36am

su in the console only elevates the console, not the entire desktop environment.

Open another console window whilst the script is waiting for you to copy the file…
Do su.
Do a command line cp <source> /mnt in the console. Make sure you type the source path (<source>) correctly, and remember what the two most obvious error messages mean. ‘Not found’ means “you typed the name wrong”.

I can’t tell you what <source> should be, because I don’t know where you have saved the file. Replace <source> with the full pathname.

Graeagle · August 9, 2018, 12:52am

I wondered about that but since I was unable to log into root from the opening screen I had no way to confirm one way or the other.

I think I left the V04 file in /tmp before I switched my PC back to Windows so the copy should be:
cp: /tmp/original_v04.01.02-417.tar.gz /mnt

cpt_paranoia · August 9, 2018, 12:52am

And with that, I’m out of here.

I only popped back to report my findings on the parted issue, not to get dragged back into proving free product support for WD, since my posting style isn’t appreciated by the mods here.

Good luck.

Graeagle · August 9, 2018, 12:54am

Thanks, I couldn’t have done this without your help. I will post the success for your viewing pleasure even if you decline to respond.
Thanks again,
Bill

Graeagle · August 9, 2018, 12:41pm

Well, “this” looks encouraging:

Screenshot%20from%202018-08-09%2005-40-24

5:45AM Bye for a few hours…

Graeagle · August 9, 2018, 6:16pm

Went through the two 40-second resets but end up with a red LED and a “critical” error 400053 message on the Dashboard “Content Status” Not Mounted. (Dashboard shows green check for the V04 firmware)

I did a reboot of the MC and attempted to re-enter the Dashboard but confronted with a logon pane I can’t satisfy with any known name(s) or PW’s. LED remains RED.

It seems that not all that is required for the SSD is present?