Rant! The perpetual Scans forever and ever and ever - frustrated with road rage type anger

Yes I have an EX2 Ultra now so why am I posting this back in my old stomping grounds? because there are about 3 users in the EX2 forums; including myself so I get about 3 reads in 6 hours.

The EX2 ultra is identical to the Single Bay My Clouds anyways so any complaint about the EX2 is a complaint about the My Cloud.

So with a little bit more time on my hands these days I decided to let the drive run as-is with indexing and building up the thumbnails so that you get this tiny little photo, the size of a pea, on the left side of your photo of which you have to click on it in order to see what you are looking at. There is absolutely no way to identify the photo by the thumbnail. A total waste of time for WD to build up these thumbnails that you can’t even see.

The building up of these thumbnails drove me crazy as it took over a week (4 x 24 hour days actually) to scan through my whole 9TB of data with 3TB of photos including NEF, Panoramic, weddings, headshots and so forth but the unit rattled through the night, through the day, through the night, chattering away, building up speed at certain times and churning at my panoramic. The fan on the EX2 also started ramping up to over 10,000 RPMS whining away like a turbo jet and whines through the night.

At times I would SSH into the device to see what it is doing… and there it was… converting my photo into a thumbnail…

I waited patiently because after almost a decade of using WD live/cloud products I want to see and hear if after scanning all the files, “is it true?” does the My Cloud actually sleep?

Many times, I was tempted to kill the services and just do what I have been doing for all these years. Killing the services to make the “my Cloud” work like it should have in the first place.

The fan on the EX2 was the biggest issue because for some reason the fan started ramping up in noise up to 10,000 rpm even though the drives were only reporting 51 degrees. I had even modified the fan table so it would stay off until 60 degrees. For some reason that fan table doesn’t work, thus all night long the EX2 acted like a very noisyvheater.

Then suddenly one day the device was quiet… It was sleeping. So it is true that once all your data is scanned and thumbnailed, it does go to sleep; albeit not completely as the cron and other linux activities seems to wake up the drive for a few minutes on the hour.

If however, deleting the time machine backup of 100GB, the scans starts all over again and this time I didn’t see any conversions but scanning through all my files took a couple of days.

I hate it. Any change on the drive causes the scans to erupt into a flurry of activity.
I hate it. Click-a-Click-a-click-a-click… and on and on and on… like somebody from outside is hacking my device… and it does this every firetruck time I add or delete a bunch of files.

Yes I know that turning off “Cloud” access will turn off the media scans, but for goodness sakes

"PLEASE give us a switch to turn off the scans" even with "Cloud ON"

I want “Cloud” access and it works perfectly fine without thumbnails.

This is the one unnecessary feature that has driven all your customers away. Even users on the EX2 forum has asked about the noisy fan that runs all night. Ramping down the fan isn’t the problem, dialing down the scans so they don’t affect your normal usage isn’t the problem.

The problem is that your (WD) is the only company that has unauthorized scans on a NAS device and you re-start the scans whenever the drive changes. You re-build the thumbnails whenever your firmware changes. Not only do you scan the EX2, you then scan the attached USB drive if the option of “cloud” access is turned on for the USB drive.

**THIS IS UNWARRANTED!! and UNNECESSARY SCANNING!! **

You have LOST-A-LOT of sales because of this one fact
You have RMA almost every 2 drives that you sold because of this fact
You rewrote the whole forum software to keep out unhappy people because of this fact

Recently I’ve bought a couple of drives from eBay at a very low price and I’m getting these brand new un-opened drives at sell-out prices and I’m suspecting that you have a lot of unhappy customers which meant that you have a lot of inventory because you think that everyone wants to use the cloud now but not yours.

I’ll repeat it again that because of this one fact that your devices basically scans for a whole week, non-stop, with full fans blasting (on the EX2) that your customers goes “WTH???” and even your seasoned Customer like myself after a few days of it was ready to throw his device out the window; I kid you not because it is incessant noise.

You know what really gets my sheep… is that after knowing all this and half a decade later, you still don’t change it.

All you have done over the last half decade is to make the scans a lower priority.

That isn’t the solution.

The solution is "DO NOT SCAN THE DRIVES for thumbnails.

Also you do not build indexes on the drive for quick searches. Yes I know Apple does this, but it is on a much much much smaller scale of 100GBs on most Macs and they leave well enough alone for the rest of the data.

@Bill_S, @Great_Scottt
Pass it on please… I think it is time for WD to have a Thumbnail Switch OFF, Media Scan Switch OFF, Indexing Switch OFF. CRON OFF.

This was in the idea box for years and your answers were the Cloud Switch. That isn’t good enough. Really. There is absolutely no reasons for a Cloud device to scan willy nilly all through the night for a whole week with the fans at full blast. If you have doubts, go check on the number of returns you had for the last half decade.

It is time to include a switch.

Give it some great labels like.

Thumbnails ON- for instantaneous iPhone browsing
Indexer ON- for great file searches
Movies Thumbnails ON- Plex like covers
Heater ON - for cold nights

Thankfully, I am now using my script to turn everything off so it is now sitting quietly, finally.

Wait I stand corrected that not only do you create a thumbnail but you resize the photo so that it downloads quickly on iPhone devices. I totally understand the reason for it. However you still cannot do it for every file on the hard drive because that isn’t YOUR choice as a device.

ON/OFF and perhaps a box to add paths to photos and/or media that you can scan.

Well, as I told you in EX forum, my DL2100 NAS is mostly deadly silent with Plex app turned off. So. as I decided to use Plex more often, I kept the app running and my NAS got noisy. It would click the heads every second – day and night. I turned off Plex app and all became quiet again. So, If I want to use Plex I can easily turn on app in Dashboard, and likewise I can turn it off when finished with Plex.

I think a possible fix is if you can ssh into the device. Change the priority of the scanning software to the possible value. This way it will only run when nothing else needs to run.

Totally agreed on the Plex noisiness as I feel like they are streaming my stuff out to other Plex subscribers :stuck_out_tongue:

As much as I like the concept of Plex, the toggling off of Plex which includes turning off the port allowing the EX2 to sleep, is too much effort mainly because I rarely watch any movies/TV shows these days.

I have subscribed to one year of infuse for $12 and it is good enough for my use.

However the scans that I’m referring to here on the My Clouds forum is really an old complaint and that is
the two services that runs just like Plex; creating thumbnails and resizing photos down to 1024x1024 so they download quicker when accessed on your mobile phone.

wdphotodbmergerd
wdmcserverd

The problem with these two services is the incessant chattering they make if your hard drive is full of photos, movies, ebooks and music. The chattering of your drives not only occurs when creating thumbnails and resizing photos, but because they traverse every single directory on your hard drives they do this on every change of your hard drive.

As you mention, we don’t need plex to be running 24/7 and neither do we need these services running 24/7. We do not need thumbnails, nor do we need photo resizing because turning on Cloud access doesn’t mean that I’ll be viewing photos.

WD has forced these services on us for well over half a decade starting with the My Book Live of which I could hear the drive chattering away in the closet. That was the time that I discover these forums and how to turn off the scans.

Although I can turn off the scans via SSH, I am tired of doing so. Every time I boot up my EX2, I have to SSH into the device and stop the scans.

Even with the scans off, I still hear a click, click, click every second which is a routine that WD has in their startup init that traverse every directory on the hard drive and sends jpgs and other files to their converter program. I can stop the converter programs but I cannot stop the traversing of my drive because it is part of the startup.

So for years now, rather than a chattering, I get a click… pause 1 second, click… and so on.

Normally I don’t pay it much attention as it is what it is… a My Cloud Cheap NAS… but once in a while I am infuriated with the nonsense of it all; a NAS drive that scans my drive for photos just so it can resize it for cloud use? like why are you doing this?

That is what WD did to fix it and frankly it does work as shortly after they changed the priority to the lowest possible value allowing the drive to operate normally while the device is almost quietly scanning your photos, the number of posts declined to almost zero with exception of a few.

There are still a few EX2 users that have been complaining about the fan being on all night and in my case several days of continuous fan noises. This is because the priority goes back to high when nobody is using the device and the chatter is really noisy, I kid you not. The noise is because two drives on the EX2 is being written to when a thumbnail and a 1024x1024 photo is generated. So the heat starts to ramp up, then the fan starts to hit 10,000 RPM.

There is absolutely no reason for photo resizing or thumbnail generation no matter what the priority is. This should be a switch for those who wants and needs thumbnail photos and smaller photos for quick access and so on.

It shouldn’t be forced on us like Windows Defender that scans all your files every night whether or not new files have been added. It simply makes no sense.

The my Cloud isn’t a photo album.

Actually the priority does not change. But since the system has nothing else to do. It runs the scan constantly. But it shouldn’t scan the files if no new files have been added. There is a function called inotify used by minidlna. Telling it what file and where the file is located. When a file is written it notifies minidlna.

What? minidlna? The My Clouds run Twonky. Did you mean to say Twonky?

You are right rac8006 that the priority doesn’t change…

sometimes my imagination runs away with me as I hear the gurgling of data being swept around my hard drive and the gurgling grows in volume into a Niagara fall of data as photos, movies and music files gets sucked into a vortex of …
and the fan gets louder and louder and louder…

minidlna is only part of the DLNA scanner and not part of the photodbmerger scanner. I don’t turn on DLNA on my main my cloud.

I love your enthusiasm but unfortunately photodbmerger scans seems to be incessant with its scans. After a week of scanning, I copied a couple of non-photo files onto the drive… and the drive started scanning again. Even a hibernate and restart, the scans are in full force and it is driving me nuts.

Can you imagine your USB drive doing this? this is nuts I tell ya…

absolutely true, that is why we also have “engine check lights” telling you that it is time to trade in your car :stuck_out_tongue:

@dswx42

How would you explain why I don’t have the issues as Raphael has? My DL2100 is very quiet unless, and only when Plex is running.

I am a photographer and I have about 2TB of photos ranging from simple weddings, headshots, baby photos to huge panoramas photos that takes up 400GB of space for one photo and I have hundreds of these huge GB photos along with thousands of RAW NEF files.

I also have about 3TB of movies, 2TB of ebooks about 200,000 epubs of which the directory doesn’t show up for 30 minutes if I use SMB, but I can access the directory with AFP.

I also have about a TB of midi files which strangely is like my epubs; I cannot pull up the directory if I connect via SMB.

I have a lot of files and even “Beyond compare” which compares two hard drives for changes just by file dates and sizes, takes about 8 hours (well maybe 4 to 6 hours as I just set it up on my remote mac mini) to compare everything and I have to connect via AFP otherwise it would hang forever via SMB. This is just the file/date compare and not the actual file copying and syncing which takes another couple of days sometimes.

Also the reason that everyone have been putting up with this for years is the fact that every one doesn’t have file changes everyday. If you have only a few TB of files and mainly large files, the scans are pretty decent. In fact my test drives with only a couple of TB of movie files operate decently and it actually sleeps without any SSH’ing into them.

For me, it is a constant file changes with new photos, photo editing, new downloads and so on. As an alternative I’ve been contemplating on using just a mirrored USB drive or ultimately just go with a real NAS drive that doesn’t scan my photos, but then again… if I just SSH into them and turn them off… I’m good… but once a year I just get really frustrated with WD and their nonsense.

edit: the other times you will notice this is when WD does a firmware update. They might not force a index rebuild these days but at one time they did on every new firmware because they changed the scanning methods.

yes, I’m am currently at the insane level…

Every time I buy a new device like the 8TB single bay My Cloud two years ago and recently the EX2 Ultras with 20TB of hard disk, I would experiment with them by letting them do whatever they are suppose to do, like right now I just came back up from a power outage and my EX2 is thrashing my device, going through all my directories.

Remember that this isn’t building any thumbnails nor resizing photos, it is just … wait a moment…

It is the wdmcserver that is causing all the ruckus!! just went in to kill the service and it is now silent. So I think wdmcserver is the scanner. Rebooting to let it complete the scans to see how long it would take.

The thing is if I SSH into the device, the scanning stops. Any activity on the device also stops the scanning and the traversing of the directories isn’t that intrusive just a very light grumbling of the device.

Nevertheless, I’m tired of WD forcing this on us and I think after half a decade we should get Cloud access without scanning… and yes cloud access without the scans does work wonderfully; just no thumbnails nor instant photo viewing…

well Rac8006,

after powering it up and it took about 15 minutes to go through all my directories, so it wasn’t too bad.

Just copied about 15 movie files into it (copied it out from another directory) to see how the device will react to new files and

yup… it is definitely scanning once more… clicking clicking rumble rumble… fan is spinning faster… at about medium level noise.

no thumbnail nor photo resizing… just scanning…

I don’t think that the device handed the filenames to the scanner because it is definitely traversing all my directories and it is comping up about 5 minutes of grumbling from the drive.

fan noise louder… at 7894 RPM… temperature is still only 47 degrees though
…

the clicking has slowed down considerably… so that still took about 8 minutes after 15 files were copied. It is still clicking and the fan is still noisy but not as rampant as before. Perhaps it is just looking at my photos right now.

so… handing the filenames over to the scanner… I don’t think so… if it was true, it would have stopped scanning within a few seconds or minutes after I copied the files.

The noise isn’t really really that loud except the fan is still at 7894 RPM so livable…

A@dswv42

you >>> What type and volume of content do you have stored on your NAS?

To answer your questions about my DL2100:
It is 2-bay 8TB as RAID1, so I only have 4 TB for my data (I know, I need to change to JBOD).
The drive is half full w/2TB open. All data is media files:

Movies - 677 GB, mostly DVD ISO files and some mp4s
Photos - 3.50 GB (My Cloud uploads of iOS things
Music - 122 GB mp3 files
Other - 924 GB I attribute this large amount to Plex keeping a database of my music albums and mp4 movies only! Why is Other using 25% of my 4TB available?

I have kept Cloud Access on since I set it up as I use iPhone and iPad to access data at home and remotely. (Likely Plex needs it on as well). I have never had any issue with Cloud Access) So, give it another try and let it finish processing.)

Thanks, Mike

So I know the encouragement isn’t mine to take as you were encouraging @dswv42 to try again :stuck_out_tongue:

but after letting it settle down and with minimal updates after the five day fanfare, the device actually settles down to sleep with no SSH modifications at all. Adding and deleting a few files causes some activity afterwards but nowhere as hectic as the last five days.

On boot up though, it does take a while to re-settle itself, but again

It does take about 5 to 10 minutes to settle down after copying a couple of GB and it settles down to a steady tick tick tick much like a clock, afterward.

I also discovered “how they are doing this”. WD basically cloned your hard drive, jjust your directory structures and filenames, under /shares/Volume_1. I believe they were created using the “touch command” to create a directory entry with no data substance and I think they are using this structure to compare the before and after changes of your drive, thus bringing validation to @rac8006 on determining which files has been added, changed or deleted. This also accounts for the 5 to 10 minutes of disk activity after a few files has been copied as the scanner compares the names.

edit: I think this directory structure is where the 1024x1024 (saw this on the conversion) photo is stored.

Although it isn’t the best implementation, it is ok and acceptable… almost because I just rebooted and it is gurgling like crazy…

but I still stand that we need a switch.

OK, so bottom line is Cloud Access is not a problem.

Just tried a quick test. I ran strace on the wdmcserverd process. Then copied several
jpg files to the MC. wdmcserverd didn’t do anything. Tried the sctrace on the wdphotodbmergerd process and it didn’t change either. Not sure why it didn’t start up the scanning.

bottom line, cloud access has never been a problem with or without the scans.

The only real problem with the scans is simply the rattling of the device for days until you are sick and tired of it and yell “STOP ALREADY!!!” especially in a bachelor’s apartment where all the devices are silent except for WD’s My Cloud.

However yes, once the scans are done, it is acceptable… the device does sleeps. On boot up, it takes 10 minutes for the rattling to stop. After copying about 1.5GB of data, it takes 5 minutes before the rattling stops.

I have heard that the scans rebuilds itself on every firmware update because the programmers like to change the format. For example, all the thumbnails use to be stored amongst your files, until I found them and ranted about how they are littering my hard drives with hidden files. They then consolidated the files under shares of which I found them today. I always thought it was just a SQL db, but it is the full directory structure.

I think you need a lot more files then 3 :stuck_out_tongue:

Try 30 movie files, or a director fill with about a 1000 epubs. It needs something to trigger the scans and after the trigger, it needs to fill it with activity because a few files won’t do it.

In my case, I have 8TB of data and I’m sure that 10 minute scan is just the traversing of all my directories
and comparing it with its own.

Try connecting to the Twonky server on port 9000 and adjusting the rescan interval. Use http://<Nas_IP>:9000 then go to advanced and change the interval or set it to 0 to prevent scanning. Only issue is you may have to reset it if you reboot the NAS because WD does not preserve across reboots or upgrades.

Twonky is nothing to do with the wdmcserver and wdphotodbmerger daemons. It is an entirely separate function, and has its own scanner and database.

You can turn off all media scanning in the web GUI, no?
I have only these WD processes:

root@MyCloudPR4100 root # ps | grep wd
 1547 root     10228 S    grep wd
 2614 root     1042m S    /opt/wd/bin/wdhws -config=/etc/wd/sprite-wdhw.xml
 2626 root      248m S    /opt/wd/bin/wdlcds -config=/etc/wd/sprite-lcd.xml
 2635 root     46456 S    /opt/wd/bin/power_monitor -config=/etc/wd/sprite-power_monitor.xml
 2643 root      267m S    /opt/wd/bin/wdtms -config=/etc/wd/BNFA-thermal.xml
 2644 root     31204 S    wdlcd_sub_daemon
 4112 root      468m S    /usr/local/bin/wdnotifier
 5157 root      211m S    /usr/sbin/wdappmgr

No wdmcserver / wdphotodbmerger here.