Clean OS (Debian), OpenMediaVault and other "firmwares"

It seems that getting spindown to work is an uphill battle. I decided to start with media player/server which runs Kodi on Ubuntu- very similar to Debian, so I am not going far offtopic. The box has a system drive and 2 storage disks, I only wanted to stop the storage disks, which I hoped would be easier. Not really. I used all recommendation from the http://info4admins.com/tips-to-spindown-your-hard-disk-in-debian-or-ubuntu/. I also monitored drive activities with iotop, so I knew that the disks were not accessed. Still they stayed active. I think that a single disk device like MyCloud is even harder to get into the standby mode.

ok rac, i used blktrace, its installed on our os.
If point it to sda, its not usefull, but pointing it to md0 (my root-partition) i get this:

  9,0    0       10    16.726433754   437  Q  WS 1579496 + 8 [kjournald]
  9,0    0       10    16.726433754   437  Q  WS 1579496 + 8 [kjournald]
  9,0    0       11    16.726464639   437  Q  WS 1579504 + 8 [kjournald]
  9,0    0       11    16.726464639   437  Q  WS 1579504 + 8 [kjournald]
  9,0    0       12    16.726493634   437  Q  WS 1579512 + 8 [kjournald]
  9,0    0       12    16.726493634   437  Q  WS 1579512 + 8 [kjournald]
  9,0    0       13    16.727604569   437  Q WBS 1579520 + 8 [kjournald]
  9,0    0       13    16.727604569   437  Q WBS 1579520 + 8 [kjournald]
  9,0    1       10    20.256090308  3406  Q   R 2756536 + 8 [hdparm]
  9,0    1       10    20.256090308  3406  Q   R 2756536 + 8 [hdparm]
  9,0    1       11    20.256200116  3406  Q   R 2756552 + 8 [hdparm]
  9,0    1       11    20.256200116  3406  Q   R 2756552 + 8 [hdparm]
  9,0    1       12    20.256387347  3406  Q   R 2756592 + 8 [hdparm]
  9,0    1       12    20.256387347  3406  Q   R 2756592 + 8 [hdparm]
  9,0    1       13    20.273284436  3406  Q   R 2756576 + 8 [hdparm]
  9,0    1       13    20.273284436  3406  Q   R 2756576 + 8 [hdparm]
  9,0    1       14    20.273402140  3406  Q   R 2756600 + 8 [hdparm]
  9,0    1       14    20.273402140  3406  Q   R 2756600 + 8 [hdparm]
  9,0    1       15    20.273516499  3406  Q   R 2756624 + 40 [hdparm]
  9,0    1       15    20.273516499  3406  Q   R 2756624 + 40 [hdparm]
  9,0    1       16    35.778909815   713  Q   W 992 + 8 [flush-9:0]
  9,0    1       16    35.778909815   713  Q   W 992 + 8 [flush-9:0]
  9,0    1       17    40.469217004   713  Q   W 3248 + 8 [flush-9:0]
  9,0    1       17    40.469217004   713  Q   W 3248 + 8 [flush-9:0]

I started blktrace in my shell and opened another shell with ipad, pushing the hdd in standby with hdparm. Its showed in blktrace-log; after 15 sec the hdd spin up. In log is something like flush. Is this the reason for spinning up? How can i find out the deamon or script?

Thanks

For other user to use blktrace:
blktrace -d /dev/sda -o trace (writes output in file “trace”, stop blktrace with ctrl+C)
blkparse -i trace* (shows files trace*, with blktrace we get more than one file)

First things first. Your btrace shows double entries for each I/O. The kjournald entries are for the ext3 log. The flush entries are for flushing cache. The hdparm are reads which is most likely reading in the code to run the program. What you should look at is the fourth column. It is the time that the I/O was done relative to the start of blktrace. You only have 20 seconds. You need more output. On the standard My Cloud firmware there needs to be no I/O for ten minutes. Your output shows that the longest it went with out an I/O was 15 seconds. The next step is very difficult to explain. You need to use some method to dump the disk contents for the value just before the + sign. For the Kjournald writes will contain a group of inodes. One of these inodes will have its date updated. In that inode will be the address on disk for the file. Using debugfs it is possible to get the inode then convert the inode to a filename. To do this you really need to understand how to read hex data.
It might be easier to use your script to check for files that have been modified. Understand that there are three time values.

RAC
PS try this command it helped on my system.
mount -o remount,noatime,nodiratime /dev/root /

Forgot to add I’ve seen a problem with samba updatig the permissions on the /etc/samba/smbpasswd file.

1 Like

Did a SMART test, 8 pending sector counts. HDD is probably failing, which explains my issues :frowning:

Fox_exe, a thousand thanks for your amazing work. It must have cost you 100s of hours working it all out. Your scripts and the firmwares work like a charm, tried some variations and finally found the best configuration for my needs. The device is so much more reliable now.

Even the recoveryx via network hapened to work merely “out of the box” as i had to discover last sunday, when i made a stupid mistake forgetting to copy some files before rebooting MyCloud.

I can´t thank you and all contributors to this thread enough!!

Спасибо

Finally it works. My HDD goes to standby aka spindown after 20 minutes (time is for testing, will increase it to 1 or 2 hours).
My last steps were:

  • disable collectd and monit (rrdcached was just disabled)
  • restart samba after reboot: after reboot, /var/run/samba is not mounted, only after samba-restart it is…
  • increase dirty-writeback-time 5s to 60s, - add commit=60 to mount-options in fstab.

My complete list for other users:

1.
*noatime, nodiratime and commit=60 added for ext3(root) und ext4:
nano /etc/fstab

2.
*install ramlog:
cd /
wget http://www.tremende.com/ramlog/download/ramlog_2.0.0_all.deb
dpkg -i ramlog_2.0.0_all.deb
rm /ramlog_2.0.0_all.deb

2a.
*bootlogd deamon rename:
cd /etc/rcS.d/
mv S04bootlogd _S04bootlog.d

*change boot-order: add ramlog in required-start/stop:
nano /etc/init.d/rsyslog
nano /etc/init.d/php5-fpm
nano /etc/init.d/bootlogs
nano /etc/init.d/samba
insserv -v -d

3.
*cronjobs edit; comment "#" hourly jobs, others at same o'clock
nano /etc/crontab
*php5 from 30min/period to daily, omv no graph (comment #) in
cd /etc/cron.d
*fake-hwclock daily update
cp /etc/cron.hourly/fake-hwclock /etc/cron.daily

4.
*edit samba with mount/umount
nano /etc/init.d/samba
* in start-command after first row "deamon..."
mount -t tmpfs tmpfs /var/run/samba
mount -t tmpfs tmpfs /var/cache/samba
* in stop-command before "log_end_msg 0"
umount /var/run/samba
umount /var/cache/samba

5.
*disable rrdcached
nano /etc/init.d/rrdcached
nano /etc/default/rrdcached
*disable collectd
nano /etc/init.d/collectd
nano /etc/default/collectd
*disable monit
nano /etc/default/monit -> start=no

6.
*increase dirty-writeback-time 5s to 60s
echo 6000 > /proc/sys/vm/dirty_writeback_centisecs

7.
*after reboot /var/run/samba is not mounted, need samba restart;
create bash (samba_restart.sh) with "sleep 120" and "/etc/init.d/samba restart" and put it in /usr/local/bin.
Now edit /etc/rc.local and add bash-start like "/usr/local/bin/samba_restart.sh"

8.
reboot

*ramlog is running?
/etc/init.d/ramlog status

*why not?
more /var/log/ramlog

*ramlog is working with tmpfs? samba-mounts to tmpfs exists?:
df -h

Now i will backup this image…

Thanks for your support.

EDIT:
If you put “/etc/init.d/samba restart” in rc.local it doesnt work for me. I had to create another bash with delayed samba-restart and refer it in rc.local. See point 7

4 Likes

@Fox_exe
I have a question. In your run_me_after_reboot.sh the variable “disk” is always /dev/sda1:

Disk="/dev/sda1"
currentRootDevice=`cat /proc/cmdline | awk -F= 'BEGIN{RS=" "}{ if ($1=="root") print $2 }'`
if [ "${currentRootDevice}" == "/dev/md0" ]; then
    oldDevice="/dev/md1"
else
    oldDevice="/dev/md0"
fi

It doesnt depend from currentRootDevice? Something like:

    currentRootDevice=`cat /proc/cmdline | awk -F= 'BEGIN{RS=" "}{ if ($1=="root") print $2 }'`
    if [ "${currentRootDevice}" == "/dev/md0" ]; then
        oldDevice="/dev/md1"
        Disk="/dev/sda2"
    else
        oldDevice="/dev/md0"
        Disk="/dev/sda1"
    fi

Sometimes after installing new image, my omv-gui File-systems-List shows me both of md-device (md0 and md1). Actually, I dont know the reason…
Im newbie at shell scripts, so I do not insist that I am right :wink:

To observe the standby-routine, i wrote a small script, its based on @Scooby_2 's (thank you :+1:) script.
This one changes the led-color blue if “status = standby” and writes the status on “steady.log” in /var/log. It checks the status any 60 seconds and put a log-line everytime when status is changed. It keeps the log small-sized…

Log-File example:

root@MyCloud:~# more /var/log/steady.log
2016-02-16 18:00:34   reboot
2016-02-16 18:00:34   active/idle
2016-02-16 18:24:42   standby
2016-02-16 19:44:16   active/idle
2016-02-16 20:02:17   reboot
2016-02-16 20:02:17   active/idle

Here is the script; put it in /usr/local/bin and autostart it by each reboot with an entry in /etc/rc.local.
Its an endless while-loop, so be attend to insert the start-command in rc.local’s last line before exit.

#!/bin/bash

# Change color to blue when standby and write steady.log in /var/log

spindown() 
{ echo blue > /sys/class/leds/system_led/color; }
steady()
{ echo green > /sys/class/leds/system_led/color; }

datum=`date +%Y-%m-%d\ %H:%M:%S`
laststate="$(hdparm -C /dev/sda|grep state|awk '{print $4}')"

# create log if doesnt exists
touch /var/log/steady.log

echo "${datum}   reboot" >> /var/log/steady.log
echo "${datum}   ${laststate}" >> /var/log/steady.log

while :
do
    sleep 60;
    state="$(hdparm -C /dev/sda|grep state|awk '{print $4}')"
    if [ "$state" != "$laststate" ]; then

        datum=`date +%Y-%m-%d\ %H:%M:%S`

        if [ "$state" == "active/idle" ]; then
            echo "${datum}   ${state}" >> /var/log/steady.log
            steady
        elif [ "$state" == "standby" ]; then
            echo "${datum}   ${state}" >> /var/log/steady.log
            spindown
        fi
        laststate=$state
    fi
done

I thought, the echo-command to led-system wakes up the hdd, but its still on standby when led changes color. Someone has an explanation? I dont really understand… Its in “/sys/class/leds/system_led/color” and not in tmpfs.

Bye, Giuseppe

1 Like

Can I double-check, does the latest version of the DSM installation instructions cause you to lose your data partition?
I see instructions above to install without losing the data, and they differ from the ones on the Google drive, but Fox_exe says above that “Change firmware to DSM not destroy data (Now).” - I’m not sure if he means he updated the installation files to do this.

Fixed.

@Fox_exe

What would you need to get clean debian on gen 2 mycloud? I got a replacement for my old one and it’s gen 2 with a totally different processor and double the ram etc. so now I’m kind of pissed at the lack of omv and all the good stuff.

Kernel sources are available but I’m not sure how to get started on building debian for it, do you have any tips to get started?

I dont have this device, so can’t build anything for hem.
But Gen2 have same CPU arch (CortexA9, armHF, vfp16+neon etc), so you can use any software for armhf.
All what you need - use debootstrap package for prebuild debian env., then - just add kernel modules and edit network startup scripts (load network module, preconfigure eth0 for dhcp). And dont forget install openssh-server.
Maybe you need UART for see errors on starup.

Ps: somebody wants to exchange with me devices?

Hi, I had problems so I restored original v4 firmware and after making step “perform quick reset to display disk size” (Как загрузится - заходим в админку и выполняем быстрый сброс (Чтобы размер диска отображался)) I lost everything in /shares/Public. Is there a way to recover lost files???

If you chose “Full reset” - you loose all data. Need extract HDD and use recovery software (Like R-Studio) for get you data back.
But on “fast reset” - just check if files exist on hdd (SSH to device and find your files)

Is there any way to add a serial number onto DSM (from an old Synology, for example)?

Also, is there any way to install Plex on DSM? I don’t see it in package manager, and I have tried the ARM and ARM7 versions direct from Plex, but they aren’t compatible: “This package is not supported on the platform of MyCloud”

I bricked my wdcloud during dsm installation. After Connection to webgui i Had to choose a .pat file. Did not work
Now i tried recovery
Which Mac adress to enter in BAT file?
I have Steady White light but cannot Ping to recovery

MAC address of your WDMC (See sticker on bottom side)

Thanks for the great work.

I installed clean Debian Jessie and Openmediavault 3.0. The WebGui somewhat unstable but already usabale.

However, I am running low on disk space on the raid partitions. Is it possbilbe to resize them? Connectint the drive to my PC and booting a linux live system for this is an option.

Main proble - need move all caches to Data partition (Like MiniDLNA database).
Another option - change mdraid from “mirroring” to “stripe” (Doubles the suze, but, possible, lowered speed… But who cares?)
One more option - custom layout (Need custom boot script in /dev/sda7)