Firmware 1.05.21 - File copy fails due to long .wdmc paths

Agree!  I have been waiting for a fix to this for MONTHA.  Support level 2 acted like it was normal since they were ‘system’ files

AGREE!!!   Same annoying issue.  I have all kind of files I created that can not delete!!!  

Because, detailed instructions have been posted previously for getting rid of the pesky autocreated .wdmc files.

 /etc/init.d/wdmcserverd stop
/etc/init.d/wdphotodbmergerd stop

chmod -x /etc/init.d/wdmcserverd
chmod -x /etc/init.d/wdphotodbmergerd

Then what happens if I hard power cycle it?  I bet these changes dont persist.

WD: Provide a PERMENANT solution to this nonsense.  You should _NEVER_ alter my file structure unless I tell you to.

I can confirm that these changes do not persist.  Again I had to setup a snippet in JuiceSSH on my phone so I can quickly disable these useless ‘services’ every time I reboot the device.  

Discovered that the creation of all the wdmc files was one of the main reasons I was having issues with webDAV shares and btsync.  Both the processes that create the .wdmc folders were fighting for CPU and RAM with everything else - vast majority of the 4TBs on my EX2 are photography backups, then all my music and video files have albums and box covers in the folders so combined with the general slowness of the device it was taking forever to work through them all.  Additionally, btsync uses 500 bytes of RAM per file being shared so the EX2 proliferating my photography folders with literally thousands of unnecessary files was what was causing btsync to intermittantly work one minute and stop working the next - as the EX2 no longer had enough RAM for it to run properly.  After deleting all the unnecessary files btsync now works completely fine with no issues.  

On seperate note - is there a list anywhere of all the processes that are safe to kill?  On reboot I have roughly 58 processes running and I’m sure like wdmcserverd and wdphotodmergerd a few of them are unneeded and just hogging up resources.

bdpratt wrote:

Okay, here is the solution.

 

First, you’ll need to enable SSH access for the device. Open the device webpage and go to Settings | Network and turn on SSH. It will ask you to create a password. Do so.

 

Ok, now you’ll need an SSH client since Windows doesn’t come with one (don’t know about Mac).

Download PuTTY from here. I recommend just putting the putty.exe file into C:\Windows. Then you can access it anytime by running “putty” command.

 

Assuming you’ve put it where I said, go to Start | Run…

Enter putty in the run command, then enter.

 

Now you have the PuTTY Configuration dialog open. In the Host Name (or IP Address) field enter the name of your device. For instance WDMyCloudEX2. Or you can just put the IP address if you know it. Then press the Open button.

 

Now you’ll have an SSH shell open. It will ask you to logon. Use sshd for the username. Use the password you setup earlier.

 

Now you have a command prompt that looks like:

~ # []

 

Type the following command and press enter:

 

find /mnt/HD -type d -name .wdmc -printf “”%p"\n" | xargs -n 1 chmod o+w -R

 

The drive will churn away for awhile and then you’ll have the command prompt again.

 

What has happened is all .wdmc directories, and their subdirectories, now have write permission added for all users. By default, these directories only have write permission for the owner and the owner group.

 

To double check the command was executed properly, type the following command and press enter:

 

find /mnt/HD -type d -name transcoded_files -printf “”%p"\n" | xargs -n 1 ls -ld

 

You should see a listing of all the transcoded_files directories. This is the directory just under .wdmc. We are listing these directories to ensure the permission wasn’t only changed on the .wdmc directory, but also it’s subdirectories.

 

The list should have entries like this:

 

drwxrwxrwx    3 root     share         4096 Nov 25 10:18 /mnt/HD/HD_a2/Public/Shared Music/Mercyful Fate/Melissa/.wdmc/transcoded_files

 

What we are looking for is the drwxrwxrwx portion. If it looks like that then success. If it looks like drwxrwxr-x then failure. Check the original command was type exactly as shown and try again.

 

Once successful close putty. Now you should be able to manipulate your folders without the root permission issue getting in the way. Unfortunately, if you add a new folder with JPG files to the drive, it will again create .wdmc subdirectories with limited permissions. So in that case run the command again if the folder needs to be manipulated.

 

Regarding the second issue of the thumbnail generation slowing down transfers, I suspect turning off the cloud access service before moving a large number of JPGs to the drive will solve it. However, I haven’t tested the theory yet. (Settings | General | Cloud Access = off)

 


 

A few additional notes for those who are interested and may want to take this further. I had hoped the user could simply be added to the owner group and that would fix it for all current folders and future adds.

 

The .wdmc directories are owned by root and the owner group is share. The directories are created with rwx permissions for both the owner and owner group, so it should be a matter of ensuring the user is a member of share and it should work fine.

 

However, I see the nobody user and the created user (ben in my case) are both members of share. So if windows explorer was logging into the network share using either of those users, then the .wdmc directory should be freely modified. I think that is the intention, but it isn’t working. Likely explorer is connecting to the share using some other credentials, a guest account or something. Anyway, if that was working properly then the whole issue goes away. But I don’t know enough about Unix to figure it out quickly and I need to get to work. So the above workaround will work for anyone who needs it. If someone finds a solution that works permanently please post. Otherwise, I’m sure a future firmware update will get this sorted out permanently.

this post is GOLD.

I have a ffew questions for WD, since they state that .wdmc folder is normal as it is part of the system:

  • If i have to backup all my 30k pictures from over 150 folders to another location, let’s say a windows server,  why do i have to also copy the .wdmc and triple the amount of pictures?

  • if i have to give access to someone to take all the picture folders via FTP or VPN, why does he have to copy also the .wdmc folder since he can’t otherwise ?

  • if i have to copy the picture folders on another NAS, why do i have to copy the .wdmc since the other NAS doesn’t know what that folder is ?

Wow that last post was a lot of stuff.

Lets try to simplifiy.  Anyone doing this stuff already knows about SSH.

Remove all WDMC .dirs:

find /mnt/HD/HD_a2 -name .wdmc -exec rm -rf {} ;

Stop the service:

killall wdmcserverd ; chmod -x /etc/init.d/wdmcserverd
killall wdphotodbmergerd ; chmod -x /etc/init.d/wdphotodbmergerd

Must be done again after hard power cycle.

After many iterations with WD support
they claimed that this is how the EX2 is designed to work.

That is when I ran out of patience,
and replaced the EX2 with a Synology DS713+.

All issues are gone now.

//AndersJ

I wanted to verify; I must do all these steps because the EX2 has taken all my files and placed them in a .wdmc hidden folder for which I can see, because I have hidden folders visible and within these .wdmc hidden folders is another folder called transcoded and within that folder are folders with as mentioned in a previous post a long file name consisting of letters and numbers, roughly over 35 characters long ?

[Original Folder Name] > [.wdmc] > [transcoded folder] > [35+ character folder] 

Your description is almost correct.
You put your files where you want them.
After that,
the EX2 creates the unwanted folders and files in the folders where you put your files.

Anders J

AndersJ wrote:
After that,
the EX2 creates the unwanted folders and files in the folders where you put your files.

Anders J

That description almosts sound like a tongue twister.  :smiley:

I read that it can be dangerous to do what has been mentioned to do in previous posts in this thread, I don’t want data loss.  Why does the EX2 do this ?

bdpratt wrote:

Okay, here is the solution.

 

First, you’ll need to enable SSH access for the device. Open the device webpage and go to Settings | Network and turn on SSH. It will ask you to create a password. Do so.

 

Ok, now you’ll need an SSH client since Windows doesn’t come with one (don’t know about Mac).

Download PuTTY from here. I recommend just putting the putty.exe file into C:\Windows. Then you can access it anytime by running “putty” command.

 

Assuming you’ve put it where I said, go to Start | Run…

Enter putty in the run command, then enter.

 

Now you have the PuTTY Configuration dialog open. In the Host Name (or IP Address) field enter the name of your device. For instance WDMyCloudEX2. Or you can just put the IP address if you know it. Then press the Open button.

 

Now you’ll have an SSH shell open. It will ask you to logon. Use sshd for the username. Use the password you setup earlier.

 

Now you have a command prompt that looks like:

~ # []

 

Type the following command and press enter:

 

find /mnt/HD -type d -name .wdmc -printf “”%p"\n" | xargs -n 1 chmod o+w -R

 

The drive will churn away for awhile and then you’ll have the command prompt again.

 

What has happened is all .wdmc directories, and their subdirectories, now have write permission added for all users. By default, these directories only have write permission for the owner and the owner group.

 

To double check the command was executed properly, type the following command and press enter:

 

find /mnt/HD -type d -name transcoded_files -printf “”%p"\n" | xargs -n 1 ls -ld

 

You should see a listing of all the transcoded_files directories. This is the directory just under .wdmc. We are listing these directories to ensure the permission wasn’t only changed on the .wdmc directory, but also it’s subdirectories.

 

The list should have entries like this:

 

drwxrwxrwx    3 root     share         4096 Nov 25 10:18 /mnt/HD/HD_a2/Public/Shared Music/Mercyful Fate/Melissa/.wdmc/transcoded_files

 

What we are looking for is the drwxrwxrwx portion. If it looks like that then success. If it looks like drwxrwxr-x then failure. Check the original command was type exactly as shown and try again.

 

Once successful close putty. Now you should be able to manipulate your folders without the root permission issue getting in the way. Unfortunately, if you add a new folder with JPG files to the drive, it will again create .wdmc subdirectories with limited permissions. So in that case run the command again if the folder needs to be manipulated.

 

Regarding the second issue of the thumbnail generation slowing down transfers, I suspect turning off the cloud access service before moving a large number of JPGs to the drive will solve it. However, I haven’t tested the theory yet. (Settings | General | Cloud Access = off)

 


 

A few additional notes for those who are interested and may want to take this further. I had hoped the user could simply be added to the owner group and that would fix it for all current folders and future adds.

 

The .wdmc directories are owned by root and the owner group is share. The directories are created with rwx permissions for both the owner and owner group, so it should be a matter of ensuring the user is a member of share and it should work fine.

 

However, I see the nobody user and the created user (ben in my case) are both members of share. So if windows explorer was logging into the network share using either of those users, then the .wdmc directory should be freely modified. I think that is the intention, but it isn’t working. Likely explorer is connecting to the share using some other credentials, a guest account or something. Anyway, if that was working properly then the whole issue goes away. But I don’t know enough about Unix to figure it out quickly and I need to get to work. So the above workaround will work for anyone who needs it. If someone finds a solution that works permanently please post. Otherwise, I’m sure a future firmware update will get this sorted out permanently.

The method you posted, deletes the .wdmc folder and all transcoded files etc sub-folders, but my main concern is, will data be deleted or there is a risk, can anyone verify ?

As well, what about file names with the alphanumeric long tacked on file name, are they automatically removed from all file names after applying this so called fix ?

Why don’t you try the much simpler method outlined by tgwaste couple posts below this long-ish method? That’s more simpler and he mentions how to stop these folders/files being generated as well, until the next EX2 restart that is.

http://community.wd.com/t5/WD-My-Cloud-EX2/Firmware-1-05-21-File-copy-fails-due-to-long-wdmc-paths/m-p/878794#M3779

The .wdmc directories are gone, although this command doesn’t work, I get an error; no process killed

killall wdmcserverd ; chmod -x /etc/init.d/wdmcserverd

How come I can’t see the files for the folders that had the .wdmc folders, are they deleted ?

Yes, all .wdmc folders have been deleted, which is what you wanted. Your data folders and your data inside them are untouched however. The only thing to keep in mind is this basically affects the mobile WD photo app (since you basically deleted all thumbnails to your pics and vids by deleting the .wdmc folders). So I hope you knew that.

As for the error, not sure if the error occurred because the wdmcserverd process (program) wasn’t running or something else was the reason. What you can do is simply split the two commands separated by the semicolon and execute them individually…so first run ‘killall wdmcserverd’ (without the quotes of course) and then the chmod one separately. After that you can execute the next pair of commands separately as well.

If you still get the no process killed error after the killall command, then that would indicate they weren’t running anyway. Don’t know why or how that’d be the case, IF that is the case, but it doesn’t matter. Do continue on with the chmod command in both cases.

If my post helps you, please click the star on the left under my username to give me a kudo.

Cybernut1 wrote:
Yes, all .wdmc folders have been deleted, which is what you wanted. Your data folders and your data inside them are untouched however.

Are you sure, because I don’t see the files ?

Running this command as is;

killall wdmcserverd ;

produces a no progress killed reply ?

What files are you talking about? Your data files?? They should all be there.

If you don’t see the files, are you looking at the directory listing from the shell or from Windows Explorer? Please paste a screenshot of one such sample directory where you do not see any files when you expected there to have been files.

As for the command…if you followed my instructions in my previous post, you’d have noticed that I did not state ANY space or semicolon following the command. Why are you adding them?? Please look carefully at my previous instructions on this. The space following the command in the original post seems like a typo and the semicolon is there for executing two separate commands from same line. But I suggested you split them - so space and semicolon were left out of my previous instructions. Seems like you are very new to this. These instructions are meant for people who are experienced with Linux/Unix command line. You should not have been doing this if you were not comfy with basic command line. One small mistake can really mess you up in command line…and can potentially have dangerous consequences.

What’s really sad is that we have waited going on a quarter of a year for a software release and we get this release with a single bug fix that is apparently causing other problems … pathetic.

Cybernut1 wrote:

 

 As for the command…if you followed my instructions in my previous post, you’d have noticed that I did not state ANY space or semicolon following the command. Why are you adding them?? Please look carefully at my previous instructions on this. The space following the command in the original post seems like a typo and the semicolon is there for executing two separate commands from same line. But I suggested you split them - so space and semicolon were left out of my previous instructions. Seems like you are very new to this. These instructions are meant for people who are experienced with Linux/Unix command line. You should not have been doing this if you were not comfy with basic command line. One small mistake can really mess you up in command line…and can potentially have dangerous consequences.

I never entered in a space, that was a typo :neutral_face:

Regardless this command; killall wdmcserverd;chmod -x /etc/init.d/wdmcserverd or this command  killall wdmcserverd;chmod-x /etc/init.d/wdmcserverd or this command  wdmcserverd;chmod-x/etc/init.d/wdmcserverd  causes the same error as I already stated previously; no process killed?

My data files are gone, the folder to which I had a few hundred megabytes now says 0 bytes, why post a screen shot, it’s empty, nada, zipo !

CookieMonster7 wrote:

I never entered in a space, that was a typo :neutral_face:

Regardless this command; killall wdmcserverd;chmod -x /etc/init.d/wdmcserverd or this command  killall wdmcserverd;chmod-x /etc/init.d/wdmcserverd or this command  wdmcserverd;chmod-x/etc/init.d/wdmcserverd  causes the same error as I already stated previously; no process killed?

 

My data files are gone, the folder to which I had a few hundred megabytes now says 0 bytes, why post a screen shot, it’s empty, nada, zipo !

I asked you to look at  MY  prior comment where I specified the commands…this comment here ->  http://community.wd.com/t5/WD-My-Cloud-EX2/Firmware-1-05-21-File-copy-fails-due-to-long-wdmc-paths/m-p/889428#M4009

NOT  the original comment where you got the commands from. There should be NO semicolon in the commands, nor should they be combined. I said in there: so first run ‘killall wdmcserverd’ (without the quotes of course) and then the chmod one separately. After that you can execute the next pair of commands separately as well.

And I still do not know despite asking where you are seeing that your data files are gone from…whether from the shell or in Windows explorer. In any event, if they are gone…well, all I can say is that I just ran the same command ( find /mnt/HD/HD_a2 -name .wdmc -exec rm -rf {} ; ) and it  ONLY removed the .wdmc folders on my EX2. Every other folder and file has been left intact. See the following two screenshots…first one is the output from running the first command (the one that deletes the .wdmc folders) and the second is the Windows explorer screenshot of one of my data directories. All files and subdirectories have been verified to be where they were. As I said before, even the smallest typo in a command can have dangerous consequences…If you run a shell command, you better understand what it does. So if your files are truly gone, I can tell you that it would not have been from issuing the command exactly as they are given. but it can occur if there were other mistakes made.

http://i.imgur.com/3SZ2NM0.jpg

http://i.imgur.com/5NzxTgP.jpg