My Cloud OS5 - Indexing taking forever

@chanchalmail, if you do not need indexing, you can turn it off by turning off the Cloud Access function.

Here is a Kb article that may help you:

Hi all,

Iā€™ve just downloaded and installed (manually) the version 5.06.115, available here: My Cloud OS 5 Firmware Release Note v5.06.115

This version was released yesterday (Nov 19, 2020) and it can only be installed manually at this moment in time.

I had absolutely no issues with my EX2 Ultra 6Tb with this release. I have my shared folders organized in a very good way so Iā€™m enabling Cloud Access one by one thus controlling the indexing process on the folders I want and when I want. If you donā€™t have your shared folders distributed in several specific categories like I do, I strongly recommend you organize your folders BEFORE upgrading to OS5, moving folders/files from one folder to another using the My Cloud OS 3 mobile phone App because it is WAY faster than doing the same job in your PC/Mac (long story, canā€™t explain it here but I know some of you know exactly what I mean).

Right now Iā€™m indexing my familyā€™s videos and pictures which consume 130Gb. Itā€™s 10% complete (yes, you can trust the real time statistics in this release), fan is running at 8800RPM approx. and CPU/Memory are high as I expected. Disk Temp is 48 degrees C, which is not different than the regular temperature on an ordinary day. The environment temperature where I am at now is 20 degrees C.

Having said this, I donā€™t expect my unit to stay days or weeks in the process because again: I am controlling exactly what is indexed and when, and I have a lot of stuff that I simply donā€™t need available via Cloud Access so they wonā€™t be indexed.

Hope this helps at least some of you.

Hi WDStaff,
I decided in general to disable cloud access incl cloud shares on my device EX4100 after OS5 update.
As in kba_30021 itā€™s mentioned that the index content wonā€™t be removed for this device type.
My EX4100 is filled up to 21TB, and would not waste additional disk space by the index which is obsolete for me.
Could you please rethink to put a switch to the dashborad to get index data deleted as cloud access and cloud shares are disabledā€¦just to run a onetime housekeeping cleanup job?

Back to normal after 8 days of non-stop indexing on my EX2 Ultra 8T.

2:2 tb indexed.
75,000 pics
600 videos
5,000 docs

I was running no apps and left the Cloud active.

I rebooted and hibernated my NAS several times and this had no impact on the indexing process.

Yes I would second this request from @crucifix
Also from my side:
It should be an option NOT to have the thumbnails and itā€™s actually disgraceful that itā€™s not optional already. The disks are normally much more expensive than the NAS units and no user should be forced to waste the disk lifetime on fripperies like thumbnails. All this extra indexing in OS5 is entirely due to this nonsense.
Ditto the pretranscoding.

I third the request

  1. Thumbnail generation should be an optional part of indexing.
  2. Pretranscoding . . . .where to start? Letā€™s start with that it should be an optional part of indexing

Further;

  1. Indexing should be decided at the DIRECTORY, not the SHARE level.

The indexing features are cause of most of the heat on this board. . . and deserve some serious attention.

Later on. . .we will need to talk about the actual (lack of) functionality of the web/phone apps; but only after this indexing issue is addressed.

Indexing 255000 picts (~1.2Tb) at pace between 17/min to 60/min. (more 20/min!)

Day 18: 61% ā€œcompleteā€. (Day 13 installed new firmware 5.05.111 which helped)

Estimate: 5 more days !

It was a big mistake to move to OS 5.whatever. It does not work as expected. I would rather say that WD issued the downgrade of their OS to screw us all up. Canā€™t believe they could commit thatā€¦

I can only guess for how long the new blessed OS 5.whatever will be processing my 16TB of disk space

I have a PR4100ā€¦it took 8 days to index. My EX4100 has been stuck at 17% complete. I rebootedā€¦it went to 17% and got stuck. I removed nearly all shares from the cloudā€¦got to 17% and got stuck. I even did a system refreshā€¦got to 17% and, you guess it, got stuck.

What else can I try?

Turn Indexing off, I had so many issues since I Updated to OS5, the drive would suddenly fall off the network, indexing took so so long, the App ( on my iPad ) didnā€™t work very well, everything was so slow, the worse thing is either the App ( on iPad ) and the web browser option will not play any file that is MKV, yet they will play MP4 files. So I have given up and turned Indexing off, a waste of time if you ask me, WD have ā€˜notā€™ do due-diligence or ā€˜fit for purposeā€™. Since I have turned off indexing things are now running fine, but thatā€™s not the point is it. It should all workā€¦ it doesnā€™t . Sorry for the little rant, just turn it off .

Steve

Same here. No web access but indexing still going on. Not impressed at all.

Indexing on Incremental changes I understand but I have not added or deleted any files for over a week but indexing. Is it indexing an index? would love to know,

Indexing completed, ios app real slow/unresponsive, web configuration site slow/unresponsive, cloud web access unresponsive, samba access works ok, CPU/RAM heavily loaded without any activity on my side, FAN 9800RPM for 10 days now ,I had to move the unit to different room just to be able to sleepā€¦Not to be rude, but I hope you guys are working 24/7 on new update as your future customer base is shrinking every day

Indexing 255k picts (~1.2Tb)

Day 21: Indexing 93% ā€œcompleteā€. Day 20 installed new firmware 5.06.115. Indexing after some time cam back to what is was ~91%. Installing the newest firmware has helped so far.

tail -f /var/log/analyticprivate.log
ā€¦status:[in progress] scan_file_count:[246232] expect_file_count:[255017]"}

Estimate: 1 more day ?!?

Thanks for sharing the path to the log with some more info.
I have been (patiently) waiting for almost a month now and the box is still indexing, so far itā€™s managed to cover 18%. So it will be 6 months before itā€™s finished? WOW!!!

And BTW, how much space will the index files chew away from my user-available space? Any idea where this data goes?

OK so I turn off all cloud access to shares (after 3 weeks of indexing and 18% doneā€¦).
Then how do I delete the files that the indexing process has created?
Can I simply reboot the device after turning off all cloud access and then go and delete the restdsk-info and restdsk-data folders?

Hi, I have been following this forum since the beginning and sharing the same frustrations as everyone else. I have a PR4100 with 4x 3TB WD Red drives. I updated the RAM to Timetec Hynix IC 16GB KIT(2x8GB) DDR3L / DDR3 1600MHz PC3L-12800 / PC3-12800 at Amazon for $55.00.

I have approximately 7 TB of storage ranging from big video files, to zillions of photo files (I usually shoot Jpg + RAW. I also keep about seven years of administrative records of a software company). I was happy with OS3 but felt that it was getting long in the tooth, so I was happy to see that OS5 had arrived. I did one thing right ā€“ I did a full USB backup of the entire array onto a 10 TB external bare drive. Then the problems started:

The original download of OS5 went smoothly, but I instantly lost contact with my other PR4100 (un-updated) used for normal backups. Then I noticed that the drive started indexing at a very slow rate and the Cloud access barely worked. After a while two more updates were dropped by WD, and I installed them ending up with 5.06.115. The indexing process continued for over a week and at about 73% it ran into a brick wall. I finally went into hibernation and unplugged the drive. It re-started at about 40% indexing. It took it a while to get back to 70% but after about one week it got stuck at 97% and there was no more visible progress except for random disc activity. Furthermore, there were a couple of Shares that did not show up on the remote cloud access. About this time, I decided that the situation was FUBAR, and I started to figure what I was going to do with this boat anchor. I shifted my attention to doing actual work, but I still left the drive on, unattended.

Yesterday, 11/25 I decided to turn off the drive, but when I logged in to the dashboard to do a soft power-down, I noticed that the progress indicator said Connected . I ran some tests ad found that everything was working properly. The Cloud remote worked great, the thumbnails seem to work fine, the much maligned internal trans-coding makes streaming videos a snap and I was able to identify and share clips like I never have before.

I was still mistrustful (and I still am) of the stability of this whole rig, so I hibernated the drive, pulled the power cord, re started at 11:00AM today and again it gave me the dreaded Indexing message at 54%, at 11:06AM-82%, at 11:07AM- 96% and at 11:08AM ā€“ CONNECTED.

I think that there is hope, although the CPU and memory of my setup is the best you can do and it is possible that other systems may still be too slow ā€“ the additional memory seemed to help, but there is not much you can do about the CPU. Since the indexing is only really done once. It may be worth waiting a long time in hopes that it will finish. I also believe that WD is having a wake-up call and they will probably keep on issuing updates which may include speedups and/or a more granular approach as to what the user wants to index.

I hope that this post answers some of the questions. I will follow-up if I find other things to report.

~C

This thread is so long. But I have question.
My Cloud fisnished indexing but all the time wakes up itselfs - without demand.
Do You observed it? On OS3 MyCloud was sleeping non stop, disks spin only when I open the content. On OS5 every few hours it starts disks for a while and sleep again. WTF?

Noticed this on PR4100, maybe multiple devices have this issue.