Runaway mediacrawler: The Ultimate Solution!

Prologue:

I purchased a 3TB My Book Live and a 6 TB My Book Live DUO about 6 months ago. The 3TB has worked fine but the 6TB has always had this problem: the disk drives never went to idle, the case was hot (I sometimes got thermal warnings), the status light would blink green even when the Ethernet cable was disconnected. I intended the 6TB (using it in RAID configuration) to be my main archival drive, and spinning the disks 24/7 is a good way to burn it out quickly. Between then and until the last few days I’ve been occupied with higher priority tasks and never had the time to track down the problem.

A few days ago I started Googling and discovered that countless other My Book Live owners face the same problem. Even worse, most of the discussion was on forums and most topics ended with no satisfactory solution. (I never did find a satisfactory solution that worked for me.) I found countless requests for WD to fix he problem, but I updated to the latest firmware and it isn’t fixed yet.

I’m a retired hardware/software engineer and my job was to design gear and to fix designs that weren’t right. I decided that finding the final solution for the constantantly spinning My Book Live was just the right problem for me, particularly since my DUO was on a course that would lead to its destruction in a mere few years rather than the decade or more I expect to get out of it.

So here is my solution:

1.) First you must enable SSH access to your My Book Live (or My Book Live DUO)

1a.) Enable SSH access (easy method): First log in to your control panel and look at the URL in your browser’s address window. Since many of us have changed the default name of our device this URL will vary. For purposes of example let’s say you’re on the main login page and your URL is http://my-book-live/UI/#. Change the “#” to “ssh” and access that address. You will get a page titled Secure Shell (SSH) Access. You will see a box titled SSH Access: check it and save the setting. Note the comment that the default user name is “root” and the default password is “welc0me” (that’s a zero character.) Save the setting and your SSH is enabled. (You should probably reboot.)

1b.) Eable SSH access (alternate method if the first method doesn’t work): Log in to your control panel and go to Settings -> Utilities -> Import/Export Current Configuration. Click “Export: Save Config File” and follow the dialog to save the file somewhere on your HDD. Use a good text editor (not Notepad) to make the following change. (Hint: Use Notepad++ available at http://notepad-plus-plus.org/)..)

Before proceeding, save a copy of this configuration file. It never hurts to be cautious. Now change the line that says ssh_enable=“disabled”, replace disabled with enabled, then save the file. Referring to the previous paragraph, click “Import: Choose File” and import the file you just edited. Reboot your device (Settings -> Utilities -> Reboot/Shutdown). If all has gone well when your device has rebooted you will now have SSH access enabled.

2.) Get an SSH client program:

Get an SSH client that runs on your computer. Many computer people recommend the freeware SSH client PuTTY: https://en.wikipedia.org/wiki/PuTTY which supports Windows and various other operating systems. Written and maintained by Simon Tatham, you can get the free download from the PuTTY official website: http://www.chiark.greenend.org.uk/~sgtatham/putty/

It does not require installation and executes from a command line. Jut type the program name and IP address of your device. Example: “putty 192.168.1.123” You’ll get a typical login sequence: “login as:” – enter “root”, then you’ll see "root@192.168.1.123’s password:" – enter “welc0me” (0 is the number zero, not the letter).

If successful you should see a prompt looking something like this: MY-BOOK-LIVE:~#

3.) Optional step - change your root password:

Root and welc0me is the default for all the WD My Book Live products. The Unix command to change your password is “passwd” and its use is obvious. You should consider doing this for security reasons.

4.) Optional step - let’s get familiar with our system:

Type the command “top” (which I believe means "table of processes). It’s very similar to the Windows Task Manager. It updates in real time and you will probably see that nasty mediacrawler program bouncing around right at the top of the list (the most active programs) hogging resources.

All the information on the Internet says if you just let it do its thing it will finish indexing after several hours and will no longer be a resource hog. I let mine go for 6 months and it never did quit hogging resources. I’ve got both the 3TB MBL and the 6TB MBL DUO. The 3TB worked fine but the DUO never got over mediacrawler hogging resources and slowing down my accesses. During that time my DUO’s activity light never quit blinking and the disks never went to idle. This device was on a short path to self destruction and something had to be done.

Side detour: There is an interesting article at http://community.wd.com/t5/My-Book-Live/Guide-Free-up-some-resources-on-hacked-MBL/m-p/561820/highlight/true#M21353 on starting and stopping mediacrawler. This is similar to starting and stopping Windows services. BUT… If you reboot after stopping mediacrawler it will automatically restart. Not a permanent solution!

5.) How I developed my solution (optional reading):

This may be a bit complex to many of you but don’t worry, when you reach the chase scene the fix is easy. I’ll even supply an “unfix” if you want to restore normal operation.

The first problem, where is this pesky mediacrawler program located? When you login as root you end up in your home directory, so first we go to the top level of the entire disk structure using the command “cd /” (change directory, about the same as DOS). Then we do a directory search of the entire HDD including subdirectories and filter that to show only files with the word “mediacrawler.” You can Google up some Unix lessons but here’s the command that I used:

ls -a -R | grep mediacrawler

It will take a while to scan your entire disk. I’ll admit I’m no Unix expert so when expertise fails intuition is a good place to start. I saw a lot of mediacrawler files in ./usr/local/mediacrawler so that looked like a good place to start. I changed directories and did a long form directory listing:

cd /usr/local/mediacrawler
ls -all

Yep, it looks like the right neighborhood but I don’t see any mediacrawler program. The “drwxr-xr-x” stuff is the permissions followed by file owner, size, date, name… I’ll get into permissions in a few paragraphs. One important thing is that permissions starting out with “d” are directories, those starting out with “-” are files. Note also the file names are color coded. Directories are blue, executable files are green. I see a executable named “mediacrawlerd” but I was pretty sure that wasn’t the right file because Unix executables ending with the letter “d” are usually demons (you’ll have to Google that, but it’s not what we are looking for).

I know our pesky file is probably hiding nearby so I changed to the bin (binary = executables) directory and did a long form listing:

cd bin
ls -all

And there it was!!!

-rwxr-xr-x 1 root root 405711 Mar 8 14:47 mediacrawler

6.) Unix file permissions (optional reading):

The file permissions are the first 10 characters in the long listing. As I said earlier, “d” is directory, “-” is file. The remainder can be broken up into three triads. They are the owner’s permissions, the group’s permissions and “all users” permissions. They follow the form “rwx” which stands for read, write, execute. If the letter is there the access is granted, if not granted there is a dash as a place holder.

Let’s decode what our media crawler’s permissions are:

  • = it’s a file

rwx = owner permissions (that’s us, root) can read, write or execute the file

r-x = group permissions can read and execute, but cannot write the file

r-x = “all users” permissions can read and execute, but cannot write the file

Important note: These permissions are often written as octal numbers. For example rwx = 111 = 7, r-x = 101 = 5, etc. Expressed in this termininology the permissions of this file are 755.

7.) Now the plan:

We want to change this file so that nobody can execute it! We want to turn those x’s into dashes.

current permissions: rwx r-x r-x = 111 101 101 = 755
desired permissions: rw- r-- r-- = 110 100 100 = 644

For this we will use the Unix chmod command. http://www.computerhope.com/unix/uchmod.htm

8.) Cutting to the chase scene:

The command you want to turn off execute permissions for mediacrawler to type this in mediacrawler’s directory:

chmod 644 mediacrawler

Reboot your device and execute the “top” command and you will see that mediacrawler is gone from the process list. Log out and in a few minutes your NAS status LED will turn a nice pretty, solid blue.

10.) Summary:

To disable mediacrawler:

  1. SSH into your device
  2. cd /usr/local/mediacrawler/bin
  3. chmod 644 mediacrawler
  4. reboot

To re-enable mediacrawler:

  1. SSH into your device
  2. cd /usr/local/mediacrawler/bin
  3. chmod 755 mediacrawler
  4. reboot

11.) Even shorter summary:

To disable: SSH, type “chmod 644 /usr/local/mediacrawler/bin/mediacrawler”, reboot
To enable: SSH, type “chmod 755 /usr/local/mediacrawler/bin/mediacrawler”, reboot

Epilogue:

With mediacrawler disabled my HDDs no longer spin 24/7, the blinking green LED has been replaced by a nice blue LED when I’m not using my unit, and my drive data transfer speed has increased by a noticeable amount.

7 Likes

Thanks for the nice writeup!

1 Like

This work for me.

Thank you very much!!! :laughing:

Everything is fine in the morning. The light is blue.

Nice explanation will try today

I tried ur solution . I was able to stop the media crawler running but the miocrawler is still working. Any way to stop the mio crawler `?

Oct  4 21:57:39 SudheersNAS [bootscript_095_admin-rest-api] 10/04/13 20:57:39: ( boot-system  ) begin

Oct  4 21:57:39 SudheersNAS apache2: 169.254.151.230 WebUI::session_start()

Oct  4 21:57:40 SudheersNAS apache2: 169.254.151.230 WebUI::session_start()

Oct  4 21:57:44 SudheersNAS apache2: 169.254.151.230 WebUI::session_start()

Oct  4 21:57:45 SudheersNAS apache2: 169.254.151.230 WebUI::session_start()

Oct  4 21:57:46 SudheersNAS [bootscript_095_admin-rest-api] 10/04/13 20:57:46: done.

Oct  4 21:57:49 SudheersNAS logger: /usr/local/sbin/monitorio.sh: waiting for system to become ready…

Oct  4 21:57:55 SudheersNAS logger: /usr/local/sbin/monitorio.sh: waiting for system to become ready…

Oct  4 21:58:03 SudheersNAS logger: /etc/rc2.d/S86orion: waiting for system to become ready…

Oct  4 21:58:04 SudheersNAS : System ready

Oct  4 21:58:08 SudheersNAS logger: Starting orion services: miocrawlerd, mediacrawlerd, communicationmanagerd

Oct  4 21:58:46 SudheersNAS logger: WD NAS: Email alerts REST API failed to return Success

Oct  4 21:58:46 SudheersNAS : Check if new firmware is available

I don’t have miocrawler on either my MyBookLive or my MyBookLive Duo. Perhaps you should try a firmware update.

Nor do I know what it does or the effect of terminating it.

SSH into your MBL and run the “top” command and see if miocrawler is running. You can kill it and see if that helps. (Google “how to kill a Linux process.” If the kill works you can find the file and change its permissions as I described above for Media Crawler.

Hi,

mediacrawler is an indexer of your images. It creates the image database and thumbnails, all stored in the /CacheVolume folder. It’s part of the “orion” service, together with miocrawler and communicationmanager. I guess all this fcionality is used when accessed thru mobile app such as WD Phtos for iPhone.

MBL is running pretty much standard Debian distro, so:

  • to stop it immediately, you can just run /etc/init.d/orion stop , but if you reboot your MBL, it will be running again
  • if you are familiar with bash, you can edit the /etc/init.d/orion so it won’t start
  • or, you can simply move this script from this location somewhere else, e.g. to /etc/init.d/backup or so. if your img database needs refresh (e.g. your mobile app doesn’t show all images that it should), just simply start the orion services (incl. mediacrawler/miocrawler) with  /etc/init.d/backup/orion start command and let it re-index your images and re-create thumbnails.
  • or there are million ways to disable this, just pick one you prefer…
2 Likes

You can also leave everything intact, but remove the “orion” piece from the auto-start initialization process to avoid mediacrawler from starting everytime the drive starts up.

update-rc.d -f orion remove

You should see the following message

Removing any system startup links for /etc/init.d/orion ...
   /etc/rc0.d/K05orion
   /etc/rc2.d/S86orion
   /etc/rc3.d/K05orion
   /etc/rc4.d/K05orion
   /etc/rc6.d/K05orion

 To kill the existing processes (without needing to restart)

/etc/init.d/orion stop

 If you wanna put it back some other time, just add it back using the update-rc.d command

update-rc.d orion defaults; /etc/init.d/orion start
2 Likes

Twonky seems to be at the top of my list. Still get reasonable HDD run time. Yet to see a Blue light unless i disable twonky which is why i bought it to serve media.

Remote access etc all disabled except twonky and transmission

Firmware is MyBookLive 02.11.09-053 : Core F/W & Twonky 5.1.9 & Transmission 2.77

Thoughts?

PID USER PR NI VIRT RES SHR S %CPU %MEM      TIME+ COMMAND
2968 root 21 1 22848 5696 2432 S 10.9 2.2                810:36.29 twonkymediaserv
30926 www-data 4 -16 66752 16m 9216 D 6.6 6.6     0:04.37 apache2
1612 root 39 19 4288 2560 1536 R 2.0 1.0                  0:03.23 du
153 root 20 0 0 0 0 S 0.7 0.0                                          40:10.38 kswapd0
1245 root 20 0 5056 3456 2368 R 0.7 1.4                     0:00.90 top
1212 root 20 0 31616 3840 2048 S 0.3 1.5                   1:38.95 rsyslogd
2446 root 20 0 53568 5248 2432 S 0.3 2.1                 11:12.51 transmission-da
2899 root 20 0 95616 3264 1984 S 0.3 1.3                 12:14.28 upnp_nas_device
1 root 20 0 4352 1664 1344 S 0.0 0.7                            0:01.61 init

I think I broke my brain writing the OP. As far as moving it along I gave it my all at the beginning of the topic.

I have multiple My Book Live drives so I use my 3TB for serving media, and my 6TB RAID and 8TB RAID as ordinary NAS.

Funny thing, my smart TV can play media files from my 6TB DUO, probably because the TV is smart enough to access a network share.

Brilliant solution, Banned! The same 24x7 disk thrashing and never sleeping was happening with the WD My Cloud NAS. Some minor changes based on your solution for the My Book Live are posted here:

How to cure endless disk thrashing and never sleeping on WD My Cloud

Thank you for the guidance. Much appreciated.

You’re welcome. It took me about a week spread out here and there over months to figure out why my NAS wouldn’t shut down. I figured it would have burned out in a year or two spinning the discs 24/7.

Lucky I used to have a Sun SPARCStation 25 on my desk a few decades ago and remembered some of my commands.

Hey Banned…Many thanks many times over!:stuck_out_tongue: I was beginning to think this baby was gonna blow up… she wouldn’t stop grinding away, and it was pretty annoying. 24 hours into the “fix”, all seems to be well… she’s a tired NAS, and now she’s sleeping well and connectivity and performance have dramatically improved. Let’s hope there’s a firmware fix for this in days/months to come. Thanks again and Kind Regards. ssprod (WDMyCloud 4TB-purchased February 2014)

Excellent solution!  Disabling orion (and a hardware reboot) have made MyBookLive’s response time almost decent.

UPDATE:

I’ve been having horrible response times from my 6TB MY BOOK LIVE DUO and finally SSH’d in to see what was wrong. I discovered that MediaCrawler had re-enabled itself. I don’t recall any firmware updates so maybe the MBL automatically updated itself and turned Mediacrawler back on. AFAIK I have automatic updates turned off.

So I repeated my solution in the OP and now Mediacrawler is dead again.

Unfortunately the problem was transfer speeds of about 50 KB/second and it’s still about 50 KB/second.

I’m beginning to get really sorry I purchased these three WD MyBookLive NAS devices. At 50 KB/s it’s practically unusable.

Thank You Banned!

Since I got the MyBookLive 3TB almost a year ago it has not stopped spinning. I shared on another thread how the drive asked me to re-back up my files and how the back up took six days! 

Most people don’t know the first thing about SSH. They could read your directions over and over and it may as well be heiroglyphs. For the regular Joe Shmoe this drive is just too complicated. It’s not plug and play. Because if it was it wouldn’t take me almost a year to figure out that a constanly spinning drive is not normal. This is suppose to be easy, you buy the drive, you plug it in, you start your web browser, you set it up, you forget about it.

Really disappointed…

Thanks again! I now have a blue LED where, for the last year, there has been a blinking green LED.

You are blaming me because the solution is too complicated.

Fine. Get WD to fix it. They are the ones to blame for the problem in the first place. They could issue a firmware update if they wanted to fix their faulty design.

My three WD drives still experience problems. My advice is to shut them down when you are not using them.