Solution: Monitorio and the 7 second wake up

edit: final fix to monitorio.sh is on page two of this post. 

edit: added mount noatime solution by Rac8006

So after rummaging through all the scripts, I am starting to get the big picture of how this tiny linux NAS box works. The non-sleeping issue has been a major problem since the WD-live days of 2011. In fact the WD-live and the WD-Cloud has the same problems since they run almost identical OS’es with slightly different program names.

Cron jobs!!

If you check /etc/cron.d, you will see probably most of the nightly wake up jobs, including the 20-checkRaid that runs at 3:05 everynight and the auto-update that runs at 3:00 regardless of the fact that you un-check the auto update option.  Other jobs include system stats that indicates that it reports every 10 minutes everyday and at 23:59 it rotates the statistics files. In total there are 10 jobs for cron.d and 12 similar jobs in cron.daily. 

I’m not sure how many of these jobs are necessary, however if I stop the cron before putting the drive to sleep,

/etc/init.d/cron stop

My New Sleep Pattern

I get the following sleep pattern.

Mar 3 21:16:48 WDMyCloud logger: “Enter Standby in 30 seconds”
Mar 3 22:29:19 WDMyCloud logger: exit standby after 4351 (since 2014-03-03 21:16:48.940000001 -0800)
Mar 3 22:34:21 WDMyCloud logger: “Enter Standby in 30 seconds”
Mar 4 00:50:35 WDMyCloud logger: exit standby after 8174 (since 2014-03-03 22:34:21.650000001 -0800)
Mar 4 00:55:38 WDMyCloud logger: “Enter Standby in 30 seconds”
Mar 4 05:17:54 WDMyCloud logger: exit standby after 15736 (since 2014-03-04 00:55:38.180000001 -0800)
Mar 4 05:22:58 WDMyCloud logger: “Enter Standby in 30 seconds”
Mar 4 08:06:47 WDMyCloud logger: exit standby after 9830 (since 2014-03-04

which is a big reduction in the number of wake-ups (I use to get at least 1 wakeup per hour), now I get 4.

My Stopped Services

Now, of course, I have stopped all the services that I had deemed un-needed and they are as follows:

/etc/init.d/nfs-kernel-server stop

/etc/init.d/nfs-common stop

/etc/init.d/upnp_nas stop

/etc/init.d/mDNSResponder stop

/etc/init.d/wdphotodbmergerd stop

/etc/init.d/wdnotifierd stop

/etc/init.d/wdmcserverd stop

/etc/init.d/wddispatcherd stop

/etc/init.d/cron stop

and of course I re-wrote monitorio.sh, or rather I peeled of the “threshold trigger code” that monitors your hard drive for changes and if the change is more then 1GB it sets a flag for MEDIACRAWLER_REWALK (at least I think it does).

Monitorio.sh

Basically Monitorio is your sleep loop.

It has two infinite loops that the inner loop at one minute intervals watches your drive for activity and when the sleep countdown has been reached it issues a hdparm -y (putting your drive on standby) and turns on the flashing LED then exits the inner loop back to the outer loop.

The outer loop watches for your drives to wake up using hdparm -C on all your drives. Once it detects that all your drives have woken, it turns your blinky LED light back into a solid blue light.

The 7 Second Wakeup

Now I noticed a problem. In my other Sleep threads, there is always a series of 7 seconds wakeups; put the drive to sleep and 7 seconds later, if you query the drive, it is awake. Not always though. It seems like it is always the first couple of times before settling down into a longer sleep.

The problem comes from here in /usr/local/sbin/monitorio.sh

At the bottom of monitorio.sh the file /tmp/standby  is created when we go into standby by

    touch /tmp/standby

.

.

.

    for i in ${drivelist[@]}; do
       hdparm -y $i >/dev/null
    done

The Solution

Apparently the touch even though is accessing a Ramdisk causes a delay write in the hardware (not always) and by immediately performing a hard drive standby after a touch, you get the 7 seconds wake up.

This seems to fix it.

    touch /tmp/standby

    sync           <======= new solution, this sync seems to work better

    sync           <======= second sync suggested by Rac8006 reduces the total wakeups down to 1 or 2 per night

    sleep 5      <======= sleep 5 seconds waiting for the access to settle

.

.

.

    for i in ${drivelist[@]}; do
       hdparm -y $i >/dev/null
    done

My Routine

My daily routines now is once the device awakes in the morning, I turn off the power saver and leave the drive running for the whole day; thus I don’t have to worry about spin-ups and spin-downs during the day.

I also turn on the cron so the daily jobs has a chance to run. I even run /etc/init.d/ramlog restart to ensure the ramlogs gets written to disk. 

Before I head for bed, I turn on sleep and the LED, disable cron, and un-map my drive. 

Although it may be a satisfactory solution for my needs, it is an unnecessary chore to do everyday. I am tempted just leave the device On rather then go through the rituals of putting it to sleep. As a WD Red drive, the NAS drive is meant to be left on indefinitely. 

edit: 5 minute Sleep Delay

I also change my 10 minute sleep timer in /etc/standby.conf to 5 minutes simply for the fact that I would waste 10 minutes and more, every night waiting for my device to sleep; this way I only waste 5 :stuck_out_tongue:

standby_enable=enabled

standby_time=5

edit: adding mount noatime

Final ingrediants to Sleep (credits goes to Rac8006, so give him the kudos for being persistent)

  1. adding an extra sync to the monitorio script helps a lot with reducing the number of 7 second wake ups down to only one or two per night.

  2. after SSH’ing into the device issue the following mount command… don’t worry nothing untoward will happen and if it does, blame Rac8006 please…

mount -o remount,noatime,nodiratime /dev/root /

you could also use mine

mount -o remount,noatime /dev/root /

The difference is that I assume noatime encompassing nodiratime.

9 Likes

Dude.   You’re on a crusade!

Nice detail and good info here.  

Thanks TonyPh12345… I figure it is a nice change to see solutions rather then complaints :stuck_out_tongue:

2 Likes

Ralphael wrote:

Thanks TonyPh12345… I figure it is a nice change to see solutions rather then complaints :stuck_out_tongue:

Amen to that !!!  Great work, all this is way over my head, but I certainly can tell you are making headway on this issue. :smiley: I hope someone from WD is paying attention to all the work you are putting forth.

1 Like

Found a new solution… using sync followed by a 5 second sleep. 

I’d appreciate if you describe this solution in layman’s terms.

LardVader wrote:

I’d appreciate if you describe this solution in layman’s terms.

Layman’s Version 

It seems that one of the problems is that data is being cached (held in memory for the purpose of speed) from some process and when the drive is set to standby, the drive wakes up immediately to write the data.  

Thus the sync which writes any data before going into standby. For most cases, this works.

How I discovered this problem

I was trying to trace the problem by throwing out ps -ALL  >>/tmp/process.log as well as using logger “I am here at …” and I immediately noticed that every standby call resulted in an immediate wakeup. Killed all the logger messages and the problem dissipated.

Hi Ralphael

I admire your skill and persistence at discovering the flaws in WD software. Does your post mean your MyCloud actually sleeps as it should now?
What I would still appreciate is if you let us know what to do (layman’s terms ;-)), unless you reckon it’s your time spent, your solution and everybody else figure it out for themselves, which I of course respect.

Ralphael, can you post a new summary of how the Sync change affects sleep versus your original post?


LardVader wrote:

Hi Ralphael

 

I admire your skill and persistence at discovering the flaws in WD software. Does your post mean your MyCloud actually sleeps as it should now?
What I would still appreciate is if you let us know what to do (layman’s terms ;-)), unless you reckon it’s your time spent, your solution and everybody else figure it out for themselves, which I of course respect.

    • *> I tried and failed :( > TonyPh12345 wrote:

Ralphael, can you post a new summary of how the Sync change affects sleep versus your original post?

 

 

I still get one 5 second exit whenever I activate the Energy Saver, or restart monitorio services (no idea why). 

Mar 9 01:11:24 WDMyCloud logger: exit standby after 5 (since 2014-03-09 01:11:19.090000001 -0800)
Mar 9 04:06:55 WDMyCloud logger: exit standby after 6628 (since 2014-03-09 01:16:27.340000001 -0800)
Mar 9 09:46:05 WDMyCloud logger: exit standby after 20048 (since 2014-03-09 04:11:57.570000001 -0700)

So as a final update to this… the 7 second wake up is now officially fixed (at least on my MyCloud) :P 

Although my device still wakes up due to other system activities other then the cron, the 7 second wake up on initial standby turn on that occurs always once before settling in for the rest of the night has been isolated.

So for those who understand BASH script… 

edit monitorio.sh in /usr/local/sbin

This seems to be a better fix. touch Sync, Sync sleep 5 

    touch /tmp/standby

    sync           <======= new solution, this sync seems to work better

    sync           <======= suggested by Rac8006 and it does seem to work better reducing the number of wakeups

    sleep 5      <======= sleep 5 seconds waiting for the access to settle

.

.

.

    for i in ${drivelist[@]}; do
       hdparm -y $i >/dev/null
    done

My code for monitorio is actually a lot more complicated then the above. After the sync, my code checks if any data has been read or written from the drive and if there is, I skip the standby and let the timer run for another 5 minutes. In this way I save myself 1 standby because the drive is just going to wake itself anyways. However, to my surprise, the second loops of another 5 minutes never executes as it sleeps the first time everytime now. 

Hi Ralphael, you seem to have been messing with this little box for a long time by now. 

I followed your simple yet complete explanation and changed the monitorio.sh file with your fix, and it kinda worked for some time. But yesterday, to my horror, I could hear more than once the spin down - click - spin up noise (no need to describe how creepy it sounds), and decided to unplug the drive, in order to get some sleep (NAS is in my bedroom, bleh…).

So today I plugged it again and set the Standby option to “No” and left to work, but I need to solve this… 

I even tried “sleep 10” instead of sleep 5 as you recommended, but to no avail. Is there any chance that the 

turn on solid blue if applicable

ledCtrl.sh LED_EV_DISK_STBY LED_STAT_IN_PROG

line is waking up the drive? I’m about to disable this whole monitorio cr#p and replace it by a simple hdparam setting (i don’t give a d@mn about the blue “breathing” light, I just want my drive to last at least for a couple years and sleep quietly).

I cannot thank you enough for all your hard work and sharing it with us, it’s nice to know that I’m not the only one.

Cheers from Argentina,

Mariano

You didn’t mention if you had samba running or not?  I would assume that you do.  On my system I find

that samba is writing to browse.dat about every 10 minutes or so.  Which causes me to see wake ups about every 110

seconds along iwht a few 7 or 8 seconds wake ups. 

I’ve been monitoring the I/O and can see that when samba is running I get Kjournald writes to /dev/md1 frequently.

If I stop samba this disk  activity almost disappears.  Most of the disk activity appears to be updating ctime in the different inodes that samba uses.

RAC

Have you tried disabling Samba logging?

rac8006 wrote:

You didn’t mention if you had samba running or not?  I would assume that you do.  On my system I find

that samba is writing to browse.dat about every 10 minutes or so.  Which causes me to see wake ups about every 110

seconds along iwht a few 7 or 8 seconds wake ups. 

I’ve been monitoring the I/O and can see that when samba is running I get Kjournald writes to /dev/md1 frequently.

If I stop samba this disk  activity almost disappears.  Most of the disk activity appears to be updating ctime in the different inodes that samba uses.

 

 

RAC

Hello Jac70,

Rac8006 and I went through this samba thing on another thread and Rac8006 found a much more interesting find then just Samba.

I did disable Samba overnight and it did not make a huge difference; unlike the (multinas) Samba problem that Rac8006 had encountered and eventually found a fix for Samba which was submitted to WD hopefully for inclusion in the next firmware release in 2020 :stuck_out_tongue:

I did however wanted to revisit this thread and edit my first post to include Rac8006s’ findings with changing mounting with relatime to mounting the root partion with noatime.

Thus here are the final ingrediants in reducing the overall wakeups through the night. It doesn’t eliminate all, but is adquate for now until WD completely eliminates them all in the year 2020.

 Check out the following threads if you want to understand why changing the atime works. 

http://community.wd.com/t5/WD-My-Cloud/partition-mount-options/td-p/874547

also this thread of which I thought I was just entertaining crazy Rac8006 on his theory of atime and to discover that it actually was a true fix.

http://community.wd.com/t5/WD-My-Cloud/disk-standby/td-p/874032

Final ingrediants to Sleep (credits goes to Rac8006, so give him the kudos for being persistent)

  1. adding an extra sync to the monitorio script helps a lot with reducing the number of 7 second wake ups down to only one or two per night.

  2. after SSH’ing into the device issue the following mount command… don’t worry nothing untoward will happen and if it does, blame Rac8006 please…

mount -o remount,noatime,nodiratime /dev/root /

you could also use mine

mount -o remount,noatime /dev/root /

The difference is that I assume noatime encompassing nodiratime.

Check the threads from Rac8006 on the services that he disable and you might get 14 or more hours of sleep. For the minimal services that I disable, I am able to get just a little over 9 hours however I woke mine up this morning but this was the longest sleep that I have seen.

My system only woke up 2 times last night.

May 23 01:09:16 WDMyCloud logger: exit standby after 4220 (since 2015-05-22 23:58:51.589811000 -0700)
May 23 01:18:46 WDMyCloud logger: exit standby after 262 (since 2015-05-23 01:14:19.439811000 -0700)
May 23 08:09:14 WDMyCloud logger: exit standby after 24321 (since 2015-05-23 01:23:48.789811000 -0700)

Also once a week I turn on my cron and turn off sleep (otherwise the cron would wake the HD up every 30 or 40 minutes).

/etc/init.d/cron start

So until WD fixes this or adds a hardware sleep in their new models, this is probably as good as it gets.

Ralphael

I don’t have cron stopped.  This is my sleep pattern for the last few days.

May 21 12:18:02 15:14:50 10608 2:56:48
May 21 15:21:58 15:32:29 631 0:10:31
May 21 15:37:34 18:40:32 10978 3:02:58
May 22 23:56:13 03:00:12 11039 3:03:59
May 22 03:05:06 08:04:58 17992 4:59:52
May 22 08:15:09 10:27:44 7955 2:12:35
May 22 18:21:13 19:24:18 3785 1:03:05
May 22 19:49:45 20:04:34 889 0:14:49
May 23 20:09:39 00:27:27 15468 4:17:48
May 23 00:32:32 00:46:39 847 0:14:07
May 23 00:51:44 03:00:11 7707 2:08:27
May 23 03:05:06 03:20:30 924 0:15:24
May 23 03:25:35 08:05:35 16800 4:40:00
May 23 08:20:51 08:58:59 2288 0:38:08
May 23 09:04:04 09:05:18 74 0:01:14
May 23 09:10:23 13:08:20 14277 3:57:57

RAC

Ralphael wrote:

  1. after SSH’ing into the device issue the following mount command… don’t worry nothing untoward will happen and if it does, blame Rac8006 please…

 

mount -o remount,noatime,nodiratime /dev/root /

 

you could also use mine

 

mount -o remount,noatime /dev/root /

 

The difference is that I assume noatime encompassing nodiratime.

 

Thanks, Ralphael.  How would I make the mount command permanent?

Ralphael, JAC70

I’ve done some testing and found that the new firmware seems to have fixed the problem.  I checked my other system

that has the new firmware and it went to sleep after the cron stuff at 3:05AM and woke up when I login at 3:50PM.

I’m running samba and the /dev/root is mounted using relatime.  Not sure why I don’t see the problem.  I don’t think the

new firmware has the samba fix.  The /tmp/browse.dat file is being updated like it was before. 

I suspect that a fix was made to the relatime option.  Because relatime is not supposed to update the atime value

except under special circumstances.

RAC

RAC, I haven’t updated fw for a long time, which version do you have currently?
Could you post the wakeup log after some days? And tell us exactly what you have enabled and what disabled? This way we can tell if its worth the update or not.
Thank you!