How do I delete OS5 thumbnail cache via SSH?

My Cloud OS 5 Personal & Network Attached Storage My Cloud Pro Series
1

Hi folks,

Is there a way to delete the OS5 thumbnails using SSH? My drive is a PR4100.

When I first upgraded from OS3 to OS5 (some years ago now) I had two complete sets of backups (one daily, one weekly). I didn’t know any better and these were saved on a Backup share which had cloud access enabled.

Now I do know better and cloud access is disabled for the backup share. However despite cloud access only being enabled for two very small shares (0.1TB each), indexing never finishes and the photos I see on iCloud contain 52 copies of every image I had in each backup set (2 sets x 26 copies).

I’ve tried mv’ing the backup data out of the Backup share into another share, turning cloud access on for the empty Backup share and rebooting, etc, but no matter what I try, I can’t seem to trigger deletion of the damaged thumbnails.

Any help on blowing away the thumbnail cache would be appreciated. I don’t want to delete the backup folders because I need them - that’s why I bought the drive.

Online help seems to say a 40 second reset should delete the thumbnail cache. Two reasons I only want to do this as a last resort. One it destroys the user and all other config info (though it does keep the data). More importantly, I’m worried if I do this, the drive might get stuck forever deleting the cache and become unusable. FYI, creating a system report takes 2hrs and the step taking such a long time is:

du -h -d 2 /mnt/HD/HD_a2/restsdk-data

I have opened a support request. (I’ve opened a series of support requests over the years which have simply gotten nowhere.)

I’ve included some numbered screenshots so you can see the problem:

  1. Every time I enable cloud access, OS5 says it’s updating the database.
  2. If I look on MyCloud during this time, I see the same.
  3. That completes in a few minutes and indexing begins (it takes a long time to finish, but it does finish).
  4. I can see shares for which I’ve enabled cloud access. I can add and remove shares to the list and the contents synchronise.
  5. However, when I look at the photos on iCloud, it contains a lot of junk
  6. Checking the photo location, I can see the junk files are coming from my backup sets - i.e. folders for which cloud access is not enabled.

Any help much appreciated.

1UpdatingDatabase
1UpdatingDatabase971×670 64.6 KB

2MyCloudUpgrading
2MyCloudUpgrading1271×891 61.1 KB

3Indexing
3Indexing973×646 48.1 KB

4MYCloudFilesGood
4MYCloudFilesGood1278×448 33.8 KB

5MyCloudPhotosBad
5MyCloudPhotosBad1270×685 170 KB

6MyCloudPhotoLocation
6MyCloudPhotoLocation1270×821 116 KB

2

Hi Folks,

It appears I had a faulty drive - as well as index corruption. Replacing the drive and performing a system only restore appears to have fixed the issues, i.e.: I now have a quiet drive that isn’t indexing all the time; access via os5.mycloud.com is fully functional; and I have 600GB more free disk space.

In case it helps anyone, my report to WD support after factory restore is below.

Before I ran the system only restore, I ran a Quick Disk Test. This took all night to perform (about 8 hours) and it detected that drive 3 had failed.

I replaced the drive with a cold spare. During the RAID rebuild (8 hours) I noticed the unit was much quieter than it had previously been.

Once the RAID rebuild completed, I noticed that the index rebuild was proceeding much more quickly than previously. This took a couple of days to complete, but: it did complete; the OS5 index looked correct, i.e. no photo duplication when looking from os5.mycloud.com; and the process freed up approx 300GB of disk space.

Since then, the drive has been extremely quiet and disk activity only happens in response to identifiable actions I’m taking from a connected computer or during backup etc.

So it would appear there has been a faulty drive in the unit for an extended period of time - presumably since purchase - and which was not detected by OS3 or OS5. But this appears to be the root cause of the issue.

Nevertheless, I proceeded with the system only restore yesterday. I am pleased to report the restore itself was very quick - only a few minutes. I’ve re-setup the drive: all the data survived; users have been recreated; index rebuild has occurred; access via os5.mycloud.com fully functional. The drive is now fully functional.

I note that a further 300GB of disk space became available after the system only restore.

Two points of note re system only restore, one good, one bad:

GOOD: The installed apps (internal and remote backups) remained installed after the system only restore with all jobs intact. I note that internal backup jobs ran as per normal overnight last night. I.e. there has been no interruption. You might want to make sure in future you reassure customers this will be the case, instead of telling them what you told me - that these apps/jobs will be deleted by the restore.

BAD: You set all of the shares to public shares during the rebuild. There was a short period after reboot where this NAS was open to attack - before I manually reset permissions, which was a slow process. This is an unnecessary and easily patched security vulnerability.

For your reference, the system report now completes quickly, so I can do this from the web interface. I am attaching a report created just now, 20231111MyCloudPR4100.zip

Thanks for your help.

One final check: Can you tell me which directories I can check on the device to see whether there is any more space that can be recovered from old remote access indices? I’ve recovered 600GB of space already (i.e. after also creating a good index of all files on the drive).

3

The first thing you should do is create backups, then check each of the drives. Don’t trust the dashboard, it omits critical information. And if any of the drives use SMR technology, replace them ASAP.

  • smartctl -a /dev/sda;
  • smartctl -a /dev/sdb;
  • smartctl -a /dev/sdc;
  • smartctl -a /dev/sdd;

How to Access WD My Cloud Using SSH (Secure Shell)

4

Thanks for the tip re smartctl - much more info available than dashboard.

Pleasantly, no remaining issues (to my untrained eye).

I accidentally ordered a WD40EFAX SMR drive to replace the cold spare after rebuilding the array. When it arrived I noticed one letter difference in the product code and after a quick google sent it back in favour of the WD40EFRX CMR drive.

If you’re following along, this explains the difference between Conventional Magnetic Recording (CMR) and Shingled Magnetic Recording (SMR) https://www.howtogeek.com/803276/cmr-vs.-smr-hard-drives-whats-the-difference/

This is the WD Red SMR v. CMR performance benchmark referred to WD Red SMR vs CMR Tested Avoid Red SMR - Page 2 of 2 - ServeTheHome

My summary of the benchmark: SMR drives are cheaper but slower when writing. They get around this by using a large write cache, which is fine for non RAID usage (burst writes). But during a RAID rebuild, the 17 hr rebuild time in the study for CMR increased to nearly 230 hrs for SMR. Given that a further disk loss before the rebuild is complete means the array is lost, I sent the SMR drive back and bought CMR.

smartctl_sda.txt (4.8 KB)
smartctl_sdb.txt (4.8 KB)
smartctl_sdc.txt (4.7 KB)
smartctl_sdd.txt (4.8 KB)

5

All of your WD40EFRX drives look fine, and none of them use SMR technology.

Indeed, the performance of SMR hard drives is abysmal, which often causes drives to be kicked out of a RAID array for no apparent reason, as was the case with a user I helped get back online a few days ago. Fortunately, they appear to have heeded my advice to create backups ASAP, because all four drives of their RAID 5 array used SMR technology, and there was a very high risk of data loss.

6

Thanks, Cerberus, for the advice and for double checking the SMART reports.

FWIW, my DR backups are offsite, mirrored and powered down.

1 Like