Firmware 1.05.21 - File copy fails due to long .wdmc paths

My Cloud OS 3 Personal & Network Attached Storage My Cloud EX2
1

The 1.05.21 firmware update creates serious file copying issues.

Unwanted folders called .wdmc are created “everywhere”.

It contains subfolders and files with very long paths.

When copying files theses paths are too long, and the copy fails.

Needless to say this is extremely annoying.

Why does WD take the liberty to contaminate the filesystem like this?

What can be done to resolve this problem.

This is a serious show stopper.

Backups can no longer be done.

Thanks for all suggestions,

Anders J

2

Hi AndersJ, welcome to the Community. We have passed this along to support.

3

I think I might have figured out what is happening.

IF Cloud Access is turned ON,

AND whenever it is changed from OFF to ON,

folders that contain *.jpg files and possibly other “media” files

will have .wdmc folders added to them.

As if this is not bad enough,

the wdmc folder will contain a folder “transcoded_files” in its turn

containing another folder with the above mentioned “media” files

PLUS a long hexadecimal string in its file name.

The end result is a number of files with extremely long file/path names,

that if long enough, prevents file copy, delete, move etc.

And in addition to that, which is even more serious,

the wdmc folder, and its contents cannot be deleted.

AND FINALLY,

to be able to see all this, you must set up your system to display

system and hideen files, which are often not the case,

making all this even more difficult since the wdmc folder and its content is invisible.

!!! THIS IS A MAJOR BUG !!!

that makes the EX2 device practically useless for anything but the most

trivial storage applications.

This needs to be fixed immediately.

Anders J

4

WD support seems to not understand what I describe here.
Can anyone confirm what I am seeing?

Andes J

5

I get the latest firmware v1.05.21 and my The EX2 is now permanently in Safe Mode, so I would have to wait for a new one.

6

Looks like everyone is having the same complaints.  My complaint is that when I delete something with the recycling bin enabled it leaves the folder structures in place with the .wdmc folder as well.  I have to go to the recycling bin to completely remove the folder from where it was.  Seriously, was this by design? Please make that go away, I shouldn’t be getting told “you require permission from unix user\root to make changes to this file”

7

Seriously, I don’t know why my topic was deleted when it specifically said to create a new thread for issues, but this issue isn’t getting better for me. Seems I can’t even delete some of the folder/files even after purging the recycling bin.  I really hope the issues get addressed soon

8

I manged to remove the .wdmc folder by doing this:

  • Rename the folder(s) from Windows explorer. I called them “Remove”.

  • Start the WD My Cloud program supplied by WD.

  • Navigate to the folder(s) containing “Remove”.

  • Right click “Remove” folder and delete it.

Doable for a few cases.

Not practical if many.

If many cases quickest way is to go in via SSH and use UNix commands to traverse

the file system, locate .wdmc and delete.

Efficient, but dangerous.

Needless to say these are only temporary workarounds for those

that have not yet thrown the EX2 out the door.

I described this in detail two weeks ago,

in this forum and in direkt email contact with WD support.

WD support has however not acknowledged the bug, and AFAIK not fixed it.

This needs to be fixed now.

I believe WD is loosing customers every day due to this issue,

and the slow performance when handling many smaller files.

The speed issue has not been acknowledged either.

9

I’d really just like to be able to delete my files and folders with out getting file access denied.  I shouldn’t need unix user\root to make changes on my shares.  

10

WESTERN DIGITAL:

* Why do you not comment on this?

* Why does your support not understand the issue?

* When are you going to fix this problem?

//Anders J

11

Yes this is a problem. It’s not a bug per se, but as John Madden would say, '‘Ill conceived and poorly executed!’ The cloud service is creating hidden thumbnails in the .wdmc directories in order to supply low-res images to mobile apps without consumming a lot of bandwidth (and data cost) on mobile networks. However, because the .wdmc files are created by the Unix\root account you do not own them and cannot change them in any way. Any folder you attempt to delete or copy that contains a hidden .wdmc folder will fail.

If the .wdmc directories had been created using open permissions, then it’s likely we’d never have even noticed them and only blissfully enjoyed the data conservation and quick load times from mobile access. Instead it’s a major pain.

The implememtation has two major flaws.

  1. the permissions prevent deleting or copying our own directories, because that would in turn also modify the hidden .wdmc subdirectory. This is the major flaw.

  2. The thumbnail images are created immediately upon saving a JPG to the drive. When copying large numbers of JPG files, the data transfer slows to a crawl because the drive is trying to thumbnail everything as it arrives. Much better to wait unitl idle times to generate thumbnails.

I’m sure WD is aware of the issue and is fixing it, even if customer support doesn’t have any information. It’s as obvious a problem as they come. However, I can’t wait months for the next firmware update to correct it. I’ll try to find a workaround that corrects the issue without fundimentaly breaking the intention of the feature and I’ll post what I find.

Ben

12

Okay, here is the solution.

First, you’ll need to enable SSH access for the device. Open the device webpage and go to Settings | Network and turn on SSH. It will ask you to create a password. Do so.

Ok, now you’ll need an SSH client since Windows doesn’t come with one (don’t know about Mac).

Download PuTTY from here. I recommend just putting the putty.exe file into C:\Windows. Then you can access it anytime by running “putty” command.

Assuming you’ve put it where I said, go to Start | Run…

Enter putty in the run command, then enter.

Now you have the PuTTY Configuration dialog open. In the Host Name (or IP Address) field enter the name of your device. For instance WDMyCloudEX2. Or you can just put the IP address if you know it. Then press the Open button.

Now you’ll have an SSH shell open. It will ask you to logon. Use sshd for the username. Use the password you setup earlier.

Now you have a command prompt that looks like:

~ #

Type the following command and press enter:

find /mnt/HD -type d -name .wdmc -printf “"%p"\n” | xargs -n 1 chmod o+w -R

The drive will churn away for awhile and then you’ll have the command prompt again.

What has happened is all .wdmc directories, and their subdirectories, now have write permission added for all users. By default, these directories only have write permission for the owner and the owner group.

To double check the command was executed properly, type the following command and press enter:

find /mnt/HD -type d -name transcoded_files -printf “"%p"\n” | xargs -n 1 ls -ld

You should see a listing of all the transcoded_files directories. This is the directory just under .wdmc. We are listing these directories to ensure the permission wasn’t only changed on the .wdmc directory, but also it’s subdirectories.

The list should have entries like this:

drwxrwxrwx    3 root     share         4096 Nov 25 10:18 /mnt/HD/HD_a2/Public/Shared Music/Mercyful Fate/Melissa/.wdmc/transcoded_files

What we are looking for is the drwxrwxrwx portion. If it looks like that then success. If it looks like drwxrwxr-x then failure. Check the original command was type exactly as shown and try again.

Once successful close putty. Now you should be able to manipulate your folders without the root permission issue getting in the way. Unfortunately, if you add a new folder with JPG files to the drive, it will again create .wdmc subdirectories with limited permissions. So in that case run the command again if the folder needs to be manipulated.

Regarding the second issue of the thumbnail generation slowing down transfers, I suspect turning off the cloud access service before moving a large number of JPGs to the drive will solve it. However, I haven’t tested the theory yet. (Settings | General | Cloud Access = off)

A few additional notes for those who are interested and may want to take this further. I had hoped the user could simply be added to the owner group and that would fix it for all current folders and future adds.

The .wdmc directories are owned by root and the owner group is share. The directories are created with rwx permissions for both the owner and owner group, so it should be a matter of ensuring the user is a member of share and it should work fine.

However, I see the nobody user and the created user (ben in my case) are both members of share. So if windows explorer was logging into the network share using either of those users, then the .wdmc directory should be freely modified. I think that is the intention, but it isn’t working. Likely explorer is connecting to the share using some other credentials, a guest account or something. Anyway, if that was working properly then the whole issue goes away. But I don’t know enough about Unix to figure it out quickly and I need to get to work. So the above workaround will work for anyone who needs it. If someone finds a solution that works permanently please post. Otherwise, I’m sure a future firmware update will get this sorted out permanently.

13

I have almost done what you describe.

I turned Cloud Access OFF.

I opened a SSH shell.

I deleted all -wdmc folders and their contents.

like this: find /mnt/HD/HD_a2 -name .wdmc -exec rm -rf {} ;

Unfortunately something seems to turn Cloud Access back ON, now and then.

I have needed to redo the above a couple of times in the last few weeks.

I still think this as a bug.

A NAS is a Networked Attached Storage.

It should store files where I tell it to store them, nothing else.

It should NOT create anything by itself, without being specifically asked to do so.

14

Oh I don’t know, this device is marketed as cloud-like storage, not a simple barebones NAS. To support all the features it has it will need to do a bunch of things under-the-hood to make those features work. So, I think it should do whatever it’s designed to do, but do it well. That’s just how I feel about it, probably because I’m a software engineer myself. Design a feature well and users love it. Design a feature poorly and its worse than not having it at all. Maybe a better method would have been to put the thumbnails in a database or mirrored directory tree, instead of as subfolders, so that there was no chance the user would bump into them. 

Anyway, back to the point at hand, since you’d like to actually stop the files from being created at all, you’ll need to permanently stop the services that make them. Give this a shot:

From ssh execute these commands

/etc/init.d/wdmcserverd stop

/etc/init.d/wdphotodbmergerd stop

update-rc.d wdphotodbmergerd disable

update-rc.d wdmcserverd disable

The first two will stop the service that makes the files. And the second two will disable them from restarting after a reboot.

Let us know how you make out.

15

Does “cloud like storage” automatically imply that it it is allowed

to contaminate my file structure with unwanted entries?

I believe I read in another post that the update command is not available.

Can’t recall if I tried it myself, but I know I gave up on it.

Perhaps I’m wrong about that,

and just settled for disabling Cloud Access, and removing the unwanted folders.

16

You’re right about update-rc.d not being available. Apparently it was removed in more recent firmware versions.

I copied that suggestion from this post but didn’t test it because I’m not interested in killing the cloud service.

However, later in the thread is another suggestion that should work just as well. Simply remove execute permission from those services. Like this:

chmod -x /etc/init.d/wdmcserverd

chmod -x /etc/init.d/wdphotodbmergerd

That should prevent those services from being able to start. I don’t plan to test it, but if you do please report what you find.

17

thanks for the tips bdpratt

used for ex4 lets see if gets better performance

18

bdpratt wrote:

…

So if windows explorer was logging into the network share using either of those users, then the .wdmc directory should be freely modified. I think that is the intention, but it isn’t working. 

The .wdmc directory is just not visible when you pull up a share in Windows because directory and filenames beginning with . are hidden by Windows but you can see the .wdmc directories from a Mac (since a Mac OS has a linux kernel).

19

Thanks for contributing Cybernut. The issue isn’t about visibility, per se. The issue is that the .wdmc directory is a subdirectory of the users directory, but does not have the same permissions as its parent. So when the user attempts to modify his own visible directory, say for instance deleting it, the file system naturally also attempts to delete the child .wdmc subdirectory. That operation fails because the user doesn’t have permission to delete .wdmc. The user isn’t explicitly trying to delete .wdmc…in most cases the user doesn’t even know it exists, because as you say, it’s hidden. But modification to the users own directory, which should be freely allowed, fails because the hidden subdirectories reject the whole operation.

20

To see .wdmc and its contents i Windows,

you need to turn on visibility of hidden files and folders.

But,

as pointed out by bdpratt, visibility is NOT the issue here,

but visibility helps in understanding the issue.