Attached drive with existing data - issues

I purchased a 4TB EX2 mainly for file storage and archive purposes - in particular to archive the data from a lap top which I have revently replaced. The new machine has a smaller SSD disk and I need access to historical data which will not fit on it. There is @ 750Gb of data on a WD ‘My Passport’ USB 3.0 drive.

When I first connected the EX2 it was visible as a network device and I began to transfer this data with Windows explorer over a WiFi connection on the same router. Transfer speed started at @ 11Mbs with an estimated task time pf 20 hours. I left it running all night. by morning (some 10 hours later), only 15Gb had transfered and the tranfer speed was showing at just 320Kbs - that’s K, not M or G. I cancelled the job for a re-think.

However, the EX2 was then no longer visible as a network device. I could still access the dashboard with a browser and WDMyCloud but could not get regular networking to function again. So I thought I might achieve the transfer by connecting the drive directly to the EX2. Big Mistake!

I have now been watching continuous disk activity on both devices for some 5 hours and no longer have access to to the dashbord via Browser|IP address or WD MyCloud. It is reported as ‘Not found’ or ‘unable to connect’

With all disks thrashing away like mad, I am reluctant to simply pull the plug so-to-speak, for fear of corrupting the disks.

Can anyone suggest a way forward and also explain what is going on here?

BTW the EX2 was left at its default Raid 1 setting so I am assuming that the attched ‘MyPassport’ data is being integrated somehow across the EX2’s 2 drives. My frustration is espacially in not being able to gracefully interrupt whatever is happening and with no indication as to whether or not the device is in some kind of endless loop. How long SHOULD I waith for?

Try asking on the EX2 Forum:

http://community.wd.com/t5/WD-My-Cloud-EX2/bd-p/mycloud_ex2

sabretache wrote:

I purchased a 4TB EX2 mainly for file storage and archive purposes - in particular to archive the data from a lap top which I have revently replaced. The new machine has a smaller SSD disk and I need access to historical data which will not fit on it. There is @ 750Gb of data on a WD ‘My Passport’ USB 3.0 drive.

 

When I first connected the EX2 it was visible as a network device and I began to transfer this data with Windows explorer over a WiFi connection on the same router. Transfer speed started at @ 11Mbs with an estimated task time pf 20 hours. I left it running all night. by morning (some 10 hours later), only 15Gb had transfered and the tranfer speed was showing at just 320Kbs - that’s K, not M or G. I cancelled the job for a re-think.

 

However, the EX2 was then no longer visible as a network device. I could still access the dashboard with a browser and WDMyCloud but could not get regular networking to function again. So I thought I might achieve the transfer by connecting the drive directly to the EX2. Big Mistake!

 

I have now been watching continuous disk activity on both devices for some 5 hours and no longer have access to to the dashbord via Browser|IP address or WD MyCloud. It is reported as ‘Not found’ or ‘unable to connect’

 

With all disks thrashing away like mad, I am reluctant to simply pull the plug so-to-speak, for fear of corrupting the disks.

 

Can anyone suggest a way forward and also explain what is going on here?

 

BTW the EX2 was left at its default Raid 1 setting so I am assuming that the attched ‘MyPassport’ data is being integrated somehow across the EX2’s 2 drives. My frustration is espacially in not being able to gracefully interrupt whatever is happening and with no indication as to whether or not the device is in some kind of endless loop. How long SHOULD I waith for?

It’s not clear from your description if you were trying to copy the 750 gigs of data from your Passport drive, but I am assuming you were. You also haven’t specified what kind of wifi you are using…there’s N then there’s the more recent and zippier ac wifi network types, each of which has several flavors with different speeds. The more specifics you provide, the easier it is to understand your scenario…without them, it can be very difficult to identify a solution for your issues. But in this case the wifi type/speed is kinda irrelevant.

Anyway I do not know why you were even attempting to copy 750 gigs over wifi. That’s a H-U-G-E amount of data to copy over wifi. I never even think of using wifi for data that is over 1-2 GB. Then there is the issue of the number of filesinvolved - 750 gigs of mostly small files like documents, pictures, mp3s, etc. is very different from 750 gigs of large files like bluray ISOs. The more the number of files, the more the underpowered EX2’s CPU will feel overwhelmed.

Here are a few general tips for quicker copying and not losing your EX2 from the network:

  • Before initiating ANY file copy, you should do this one-time step - log on to your router’s web interface (not EX2’s dashboard) and assign a static IP to your EX2…exact steps will vary by brand and model and on many routers it is also known as DHCP address reservation or something similar. Once you’ve assigned the IP, note it and use it to access the dashboard from a browser of any computer on the network.

  • Connect the WD passport drive to USB port on EX2’s back. Then one of the simplest ways but not necessarily fastest way to copy the data is to login to the dashboard from a browser, go to apps, click web file viewer and then use that utility to copy files between shares (your entire passport drive will automatically havebecome one share). The nice thing about this way of transferring is that even if your browser times out (and it will for 750 gigs of data copy), it will continue copying in the background. The drawback of course is you won’t know when the copying is done. But this will result in one of the quicker ways, since you are not transferring the data over the network to the EX2, but instead executing the copy command on the EX2 server. If you know Linux a bit and can work your way around an SSH session, then that’ll probably give you a little faster result.

There are various other ways of copying this data over but if you execute the copy command on your laptop’s Windows explorer, you will essentially be making the data make a round-trip over the network (from EX2 to laptop and then back to EX2), even if your passport is plugged into the EX2, because you are executing the copy command on your local computer (laptop) instead of doing all the operations on the server. This round-trip alone is enough to bring the transfer to a grindingly slow speed.

In addition, if these files are media files, EX2 has a background process that indexes all media files (pictures, audio and video files) and that indexing process can really bring the copying to a grinding halt, especially if there are lots of files  -IN ADDITION to the other issues I mentioned above (round-trip copying, number of files, using wifi instead of wired ethernet, etc.)

sabretache wrote:

 

BTW the EX2 was left at its default Raid 1 setting so I am assuming that the attched ‘MyPassport’ data is being integrated somehow across the EX2’s 2 drives.

No, the external USB drives are never included in the RAID 1 setup. They remain standalone. RAID 1 is only for the internal drives.

Thanks a bunch for all that  Cybernut1 - v helpful.

I’ve had near 2 days of internet outage and used it to to arrive a slightly different solution which has created a further problem - no doubt solvable. It goes like this:

I used 7zp to zip up the required content into about half a dozen big files - according to various categories.  I then transfered them using explorer with sustained transfer speeds of 30mb/sec +. Took a while but all done now. However, I take your point about the round-trips and consequent speed/processing hits involved. The new issue is how best to unpack them without the round-trip problem. I’m reasonably OK with the Linux shell. Can I install 7zp on the EX2 and execute the unpacks locally?

The files are mainly web-site scrapes and Firefox/scrapbook page saves, often with several dozen small image files per html page - very few over a couple of K in size. There’s probably a couple of million separate files in total so lots of diskread/writes and associated processor load.

Good to hear that usb-attached devices are simply allocated separate shares and not messed about with by the RAID set up too. I have a couple of big disks I can probably put to good use.

Oops. A bit quick off the mark with my last reply. An unzip utility is already installed and seems to unpack 7zp files OK. 

Using ‘unzip’ from the right-click|unzip option of the web file viewer fails on large zip files. I have tried it on several and the biggest I succeeded with was just 4Gb. I’ve had a brief look at some log files using ssh shell but haven’t found out exactly why yet. However, using ‘unzip’ directly from the ssh command line works just fine. I have just completed unzipping a 26Gb file containing a vast number (maybe a million or so) small files successfully. It took just over an hour and would probably be quicker if the ‘q’ switch were used.

Can anyone suggest if installing 7zip on the EX2 is a viable proposition? I have used it to make a number of large 7z files and have already uploaded them to the EXT successfully. Problem is that the native unzip utility won.t handle them (I mistakenly thought it did and had done in the above post - sorry) . I could zip and upload again but would rather use pzip because it does produce smaller compressed files too. I see that it is a current project for core Busybox but no timeframe on completing it.