Catastrophic Failure > 14TB GONE > HELP!

I dont know why I still have this thing…  I have tried to be positive through all the issues I have had, but this is unforgivable

I have 4 drives in JBOD totaling 14GB of data, and its all gone…

Does anyone know if there is any Windows based utitility to scan / recover these drives?

I was:

  1. Swapping to a new machine WD sent me for the last set of known problems

  2. I also upgraded it to the latest firmware

  3. and if that wasnt enough I was also upgrading a 2TB drive to a 4TB drive.

When I when in to create a new volume on the new drive, I noticed NOTHING WAS THERE… ON ANY DRIVE…

I have tried:

  1. Pull them out an put them back in the old NAS

  2. Removed and reinserted them numerous times on both machines.

  3. Tired same and different bays.

  4. All drives at once and one at a time. 

  5. Re “fomatted” one to see if it was recognised.  It was.  Moved it to the new machine, and it recognised it.  But no old data.

HELP!  (please) I need recomendations on a windows utility to scan or recover these drives. (Ive opend ANOTHER ticket with WD, but they will probably get back to me in a couple weeks as usuall with some irrelevant explanation and telling me data loss is acceptable on a storage/backup unit…)

Anyone experience this or have any suggestions on what the heck happened!?

Ok, I have somewhat triaged my own problem and determined another fault on the WD side.

The data LOOKS to be there…

Against my better judgement I downloaded a “freeware” app in my frantic panic…  Paragon ExtFS…  (basicly freeware tha alows M$ Window$ to read the EXT4 file system)  (Please leave me some feedback if any of you have experience with this software or if I have corrupted my system with spy/adware.)

Regardless, it sees all the folders and files…

WHAT THE HECK WD!?  Now how do I recover your problem besides the already known and tedious path of purchasing another HD, and copying it all over the stoneage slow interface/connection this thing has @ 24-40 MBs…  

Lets see: $150 for another 4tb drive… and copying 14TB (again) @ ~30Mbs = NEXT YEAR…  (and this procudeure is what caused all the maxing of the CPU & 500 errors, so next year is a kind estimate) (I really wish I bought a Synology)

I’ll say it again WD, UNACCEPTABLE!!! 

sorry to hear about your problem

Paragon isn’t bad… I"m sure there are other products out there.  As a Mac user I purchase Paragon’s software for reading writting NTFS from my Mac.  it’s not bad… but other free ones exist.  If it saved your bacon and perhaps will in the future.  I’d pony up the small amount of money for the paragon software. 

I don’t think ext4 is a bad filesystem obviously companies like Synology can do it right.  But… I still stand behind my position those of use who are serious about NOT losing data pony up for a ZFS based storrage array. 

when your ready to chuck this box, check out IX systems ZFS based NAS.  

JBOD doen’t leave you much room for error…

Update: After about 16hrs into this triage…  And im no where near getting my data back to a sharing state.

16hrs of my time wasted fixing WDs bug. (so far)

I used the above mentioned paragon driver software to “look” at all the drives. Ill be upfront and say I know little to none agout Linux EXT4 partitioning except for what I am being forced to learn on the fly. (As eluded to earlier I have pulled all of these drives into a windows machine dock with the EX4 reading driver because the EX4 sees all of them as a new drive with no volumes)

Each of 4 drives has 3 partitions on it with the largest partition I am assuming is the data because thats where I found some. I have not looked at what the other 2 partitions are for. They are pretty small in comparison to all drives capacity.

3 of the 4 drives have data in the “data” partition. (Still dont know if its viable yet) (and it is still not accessable in the EX4.)  The one drive that doesnt have any data has all the partitions but no “volume meter”.  ie specs show 3xxx gb, but doensnt have the “fill bar” below it in windows, and when I click on reads, it needs to be formatted.

Coincidentily the drive that has no data was drive 1 / Volume 1. So it looks like Drive 1 “lost” its volume & partition information, but why would that corrupt all the others?  To sgallant04 comments about little room in JBOD, I agree, but again, why across all 4 drives?  Loosing 1 drive for a standard drive failue is the risk I am willing to accept for running JBOD, but again, not across all 4!  They should not be joined / spanned for any reason which is why im running it that way.  My data is not totally critical to loose warranting any type of such redundancy nor expense. (but then again looking at my financial analysis below, it may pay for itself…)  Again my point and question to WD is why all drives, and how much it costs comsumers to troubleshoot thier product faults.

I am currently running diags on drive 1. > Keep you posted with results. (and so far the drive is passing diags.)

Paragon has another software I am debating purchasing because it support EXT4 and to look at drive 1 in more detail:
Hard Disk Manager 14 Suite
I have NO idea if this is any good, or ad/spyware…  (glad sgallant04 has left a positive note, thanks.) (any others?)

FINANCIAL IMPACT ANALYSIS of owning this WD EX4: (and this is just for this issue, not the previous5 I have had)
16hrs (so far) @ $60hr (average computer tech repair rate) = $960

An additional 4TB “temp/swap” drive to swap data = $150

Another estimated 130 hrs to copy data @ 30MBs = say at $30per hr = $3900
Probably pickup this Paragon Hard Disk Manager 14 Suite @ $49.95 (supports EXT4)
GRAND TOTAL (so far) > $5059.95… (and climbing) THANKS WD for such a wonderful product!

I’m guessing how this small NAS does JBOD in linux software is LVM to allow for future expansion. 

Looks like this supoorts ext* fs & LVM as well…

http://sourceforge.net/projects/ext2read/

sgallant04 wrote:

Looks like this supoorts ext* fs & LVM as well…

Thats the kicker and the main point of my query, I didnt use LVM or span any disks.  

Each disk had its own volume, and a single share.  I wanted to keep it as the simpelist function, because I had already experienced this device maxing is CPU in doing the most basic function.  I didnt want to complicate its limited functionality…

I havent plunked down the coin yet for any recover software.  Still hoping to get additional feedback on Paragon or something else.  Im also hoping WD will contact me from my service ticket I created and front the expense for this comtuined buggy machine  (nto to mention how many of us they solf on NFS, and then took it away since thier dev team must be an internship program or something newb…

I dont have the coin for another drive either, so im forced to move files areound for the last 8 hours to make space…  (add that to my total expense…)

There has to be a way to rebuild the volumes!

I really wish this thing just supported NTFS…  (but then again they cant even support what is advertized…  (I still choke every time I see the " High Performance, Ultimate Reliability" emblazened across this devices web page)  HAHAHAHA… ( there is the joke of the century )

"I was:

  1. Swapping to a new machine WD sent me for the last set of known problems

  2. I also upgraded it to the latest firmware

  3. and if that wasnt enough I was also upgrading a 2TB drive to a 4TB drive."

If you in fact did all that at once, then that is a recipe for disaster.

Push comes to shove, load up Linux in a VM and mount directly via a SATA connection.

And always, ALWAYS backup a NAS.

skiwi wrote:

If you in fact did all that at once, then that is a recipe for disaster.

Push comes to shove, load up Linux in a VM and mount directly via a SATA connection.

And always, ALWAYS backup a NAS.

Not quite a recipe, but I agree, and admit… A lot going on there.  I think your idea is simmilar to what I have done below with the windows Ext4 driver.  Would the VM option have some additional recovery options?

Reality, they should all be able to be done, and of coarse they were done in sequence, not “all at once”.  (not really possible either)  My point was I am not sure where in the process data loss occured.

Regaurdless, and again, bottom line you should be able to do these activities without data loss.

Todays Update :  So here I am on day 3 and about 24hrs of labor on this… (probably more as I am trying to be objective)

WD support ticket has responded to call me back on Wednesday. (which makes Day 5 ) (Not too bad) (depending on “who” calls me then…)

During the past 3 days, here is what I have discovered using the Paragon Ext4 reader driver for a windows machine on these drives pulled out of the EX4 device:

  1. Somewhere during the WD EX4s upgrade process disk/volume 1 got corrupted. (most likely canidate of processes)
    …a. Disk 1 has been on WDs DLG diag running a quick, and extended scan and passed. (so disc is ok)
    …b. Ext4 partitions are there, but no data is shown.
    …c. Doing a hex view in WD Edition Acronis shows data is still there.
  2. The other 3 drives I can read data directly using the Ext4 reader. (not like drive1)
    …a. I don’t know if the data is valid, but I can at least see it.
    …b. When I try copying it, I get errors at the end, but initially it looks good. (still testing)

Why would all volume data for all 4 discs be on drive/volume 1?

Any way to recover Ext4 partion

Any way to rebuild other 3 Ext4 volumes

Still looking for a software recommendation if any of you have experience/preference.

Day 3 EOD update :

Nothing really new to report having to perform my day job for most of the day BUT…

I was pleasantly surprised to hear from WD L3 support today.  (in addition to the email I noted earlier in which “someone” would contact me on Wednesday.)

We are working through arrangements to troubleshoot and hopefully resolve the issue.  The personal outreach from WD was truely an unexpected courtesy.

(Now if they could just give back NFS!) ;]

@ SGALLANT04

Sidebar question…

How is your netgear & LACP performing now that you have had it running for a while.

I am considering purchasing if I have to recover 14TB of data… (Kind of blows that I have to drop $79 to make this thing perform at speed it should already.)  But if it cuts my recovery time in half from an estimated 130hrs of transfer, I may bite the bullet.

1. What kind of throughput are you getting?

  1. Dare I ask this device to perform double duty already trying to recover data?

No it’s still frustratingly slow.

It’s someother limitation i’m not sure what.  Other NAS products perform much speedier read / write on Gb then this one.  Using BlackmagicdesignDiskSpeedtest  I’m getting about 30 MB/s write and 80 MB/s read in RAID 5 using WD RED drives. 

As an side note, I have switched primarily over to DAS USB 3.0 RAID 0 stripe for local speed and then run a automated job to rsync the DAS back to my NAS.  My DAS USB 3.0 RAID  stripe tests at about 223 MB/s read and 226 MB/s write. 

Unable to work on this yesterday, but expecting a call this evening from WD.

Side note/Anomoly : (may warrant another thread, or search on my part)

Unit will no longer perform a reboot.  50/50 chance this unit now hangs at “Welcome to WD My Cloud EX4” with no activity lights.  This is also new in my whole scenario.  I use to be able to do shut downs and reboots without any issues before this. (moving from 1.02.36)  Often takes multiple times to unplug and even holding the reset button whil doing.  Very hit or miss.

Performing 1.3.41 upgrade now to see if will resolve.

Welcome to WD My Cloud EX4

The life expectancy of an electronic device is 100% death.  Storing 14TB of data with no backup or fault tolerance… losing your critical data is inevitable at some point.  Sorry to hear you lost your movies but you should be running RAID if you want to protect your media. 

I’m on my 3rd EX4 at this point and although WD assured me I could easily swap the drives to the new device, I still had to backup the data and rebuild the RAID array each time.  Moving drives to another unit is not as straightforward as they convinced me it was when I did research before purchasing this device.

If all you’re doing is running JBOD why not put them into a small pc or server and not deal with WD issues?  Theres much better solutions out there for simple JBOD.

Jumpman wrote:

Storing 14TB of data with no backup or fault tolerance… losing your critical data is inevitable at some point.  Sorry to hear you lost your movies but you should be running RAID if you want to protect your media. 

 

I’m on my 3rd EX4 at this point and although WD assured me I could easily swap the drives to the new device, I still had to backup the data and rebuild the RAID array each time.  Moving drives to another unit is not as straightforward as they convinced me it was when I did research before purchasing this device.

 

If all you’re doing is running JBOD why not put them into a small pc or server and not deal with WD issues?  Theres much better solutions out there for simple JBOD.

Aside from the sermon that too could be debatable about RAIDs protection algorithms in your case demonstrated as proof, and the fact that you didnt read/get the point of my request for assistance.  If all 4 drives were RAID and all 4 “lost” or got corrupted, RAID would not help either.  The point is somehing, ie this device corrupted all 4 drives simultaneously.   Yes, I have made a conscience decison to use JBOD for a different set or reasons that are not in debate here wheather you agree with them or not.   Your final query is almost self answered.  I was hoping for a “no maintanaince” simple solution that I didnt have to spend hours “working on”.  (So much for that idea) ;] And I do agree with your “failure/death is inevitable” philosiphy. ;]  (and we are all heading there)

Im still looking for additional feedback on experience/recommendations on:

  1. Ext4 File System reading/recovery software for a windows machine & dock.  The Paragon software has proven to be a little “glitchy”, so I am having reservations of using “freeware”.
  2. Feedback / public reviews on thier software also have comments on recent quality issues.  (of course nothing is perfect)

Update from today with WD L3 techs :

No new love.  We spend about 2hrs tinkering with the box to ultimately determine…  It was not behaving normally and needs to be replaced.  The volumes and shares were not recoverable on the box itself.

I will keep the community posted as this progresses.  I would still appreciate any insight/suggestion…  (but the peanut gallery can rest) ;]

I dont mean to be the jerk who says it, and it might have already be said but this is why you should always have EVERYTHING saved on an external service. I had the original MyCloud and was uploading a movie when I ended up somehow deleting EVERY movie I had (about 2TB worth of files). Now that I upgraded to the EX4 I transfer everything to a separate external harddrive in case something like this happens. 

I am sorry to hear this happened! I know first hand how much this **bleep** and it is indeed a problem with the MyCloud series

P.S. while you might not want to stick with WD after this they do have a decent deal for recertified 3TB HD that come with a warranty if you consider the dual redundancy. Personally, I wouldn’t rely on any of the MyCloud services to have saved your files which is why I do it manually every time I transfer something.

EX4Shot,

Hopefully this will all resolve in a positive outcome.

EX4Shot wrote:

FINANCIAL IMPACT ANALYSIS of this defect on the WD EX4: (Tally continues)

16hrs inital triage @ $60hr (average computer tech repair rate) = $960

2hrs on the line with WD L3 support @ $60hr (average computer tech repair rate) =  $120

4hrs setting up “recovery rigs/hardware” @ $60hr (average computer tech repair rate) =  $240

16 hours into recoveing one drive (the lease full) = say at $30per hr =  $480

Still looking to pickup Paragon HardDisk Manager 14 Suite @ $49.95 (supports EXT4) (still looking for sugestions)

GRAND TOTAL (so far) > $1849.95… (and climbing)

So I am SLOWLY using just the Paragon Driver to recover data to a test rig.

I have recovered 1 of the 4 crashed volumes.  Data continues to “look viable”.

I have 2 more to go, and 1 cannot read at all so ill have to figure out what to do with that one.

As a side note I decided to add to my recovery rig. (but didnt add the hours) I added a Netgear GS108T (obtained locally) to setup LA, and see if there was any gains in recovery/copy time.  As some other experts here on the forums have advised, there was none.  Which directly confirms and contradicts what a WD support rep told me months back.

So that makes I think the 3rd blatent mis information I have recieved from WD support:

  1. Using a drive not on the white list will void my waranty. ( confimed false )

  2. Using LACP will net me double of my transmission speeds to get me around 80MBs. ( confirmed false )

  3. Round Robin LA will also “improve my throughput” ( unconfirmed )

  4. Think there was one more, but its late, and I dont recall.  (ill edit and add when I remember)

The only question I had back to the LACP experts (since I tired to ask back in that original thread, and it appears to be locked):  What are the best / any recommended “optimal” settings I should set to try to squeeze a little more performance out of this process?

Some other experiments I did on my test rigs > tested real world transfer rates across an EX4 running 4 WD reds:

RAID 5 copy TO EX4 @ ~31MBs > default config

RAID 0 copy TO EX4 @ ~36MBs > really?

JBOD copy TO EX4 @ ~39MBs > interesting… JBOD is fastest…

Looking for community feedback on:

  1. Optimal LACP settings

  2. Anyone use a USB 3.0 drive connected directly to this device with any better throughput above the 35MBs?  (I havent even looked at the theoretical yet to see if it possible)

  3. Ext4 File System Recovery Software.

Ill keep copying tomorrow… (but think ill disable LACP since its not doing anything for me)

i lost partition too

please try File Scavenger  4.2

if you need setup PM me

you have better luck them me … i was using spanned volume 

8tb lost after firmware upgrade

some i could recover but its unforgivable… i only did a system restore and all disappeared

data is stil on the HDDS yes!!!

yes it would be a dream if they used NTFS…

im still waiting for WD support to answer my ticket

http://forum.hddguru.com/viewtopic.php?f=1&t=28825

EDIT : contacted by WD support

they are helping me now…

Glad this company stays behind his users