Firmware 4.05.00-315 Discussion

Not getting an error mirroring files from an external usb hard drive attached to a Win 10 PC to the My Cloud when using Free File Sync.

There is the following over on the FFS forum: https://www.freefilesync.org/forum/viewtopic.php?t=1650

what version ffs are you using i tried 9 and the beta

The fix is trivial: Samba’s implementation for GetVolumeInformation should not return FILE_SUPPORTS_SPARSE_FILES or just support FSCTL_SET_SPARSE. which is beyond my understanding

Currently using 9.0 (April 16, 2017 build).

currently in the process of incremental manual updates leading up to the current firmware will report back once i can retest my system. which now also has the same ffs build as you.

The domain master was already configured for no in the smb-global.conf on my unit. Here are the three “master” entries in my smb-global.conf.
local master = yes
domain master = no
preferred master = auto
os level = 5

I wonder if local master, preferred master and os level need be changed?

Yep.

After some testing over the last few days. This is my sleep time. This is with samba, cron ,atop running.
Not sure if the method I use is clean. I mean I don’t have the test environment to make sure that every feature
works. I know that I can map a drive. so samba appears to work. I can open the dashboard. But I can’t change the hostname for some reason. I’m basically running the gen1 cloud from a USB stick using chroot. I still have
a couple of processes that have not been moved to the USB. Which might increase the sleep time.

05 11 23:37:27 00:01:06 1419 0:23:39
05 11 00:11:23 02:58:07 10004 2:46:44
05 11 03:15:31 03:17:56 145 0:02:25
05 11 03:28:13 05:00:27 5534 1:32:14
05 11 05:10:45 05:15:06 261 0:04:21
05 11 05:25:23 06:17:06 3103 0:51:43
05 11 06:27:23 07:17:15 2992 0:49:52
05 11 07:27:32 08:01:06 2014 0:33:34
05 11 08:11:23 15:45:45 27262 7:34:22
05 11 15:56:02 18:30:14 9252 2:34:12
Total Sleep Time: 17:34:12
Start 1494475266
End 1494541814
Total Up Time: 18:29:08 93% Sleep 7% Wake

currently in the process of updating my win 10 pro system to the creator’s edition.
With my gen 1 cloud and the newest v4 firmware. it seems that either my computer implementation of smb3 broke or files got to messed up and since it was time for a reformat i figured it would not hurt to run a test with the upgrade to see if the systems improved.

i spent the time processing the individual updates after downgrading to v3. which in the process fixed some issues that i had with transfers until 315 was installed. last time it was the sparse error. This time NFS was working and samba was not… but NFS was read only and slow. so i rebooted the device to let cron and all the default things run while i upgrade my pc to test again hopefully all will be well if not no lost sleep most files are not on the main system and while i wait for the format and reinstall to finish i could always downgrade now that SSH is working properly last time it was not.

Edit: well it worked briefly and very slow win 10 did use smb 3.1 to perform the transfer then files that where sparse still failed to copy usings ffs. i was able to find a tool that removes the spare attrib but a better fix would be to track down a fix… of course if i set my cloud to smb2 the share disappears

I think printer support in samba causes some wake-ups. Try

disable spoolss = yes

in your smb-global. Without it samba looks for printers

[2017/05/13 02:21:09.696614, 10, pid=11928, effective(0, 0), real(0, 0), class=registry] (reghook_cache_find)
  reghook_cache_find: Searching for keyname [\HKLM\SOFTWARE\Samba\smbconf\printers]

and tries to do some spool housekeeping.

[2017/05/13 02:22:09.754966,  5, pid=11928, effective(0, 0), real(0, 0)] (print_queue_housekeeping)
  print queue housekeeping

somehow when samba does a config file check, if spoolss is not disabled, it causes a spin-up (or it looks for some other file and that causes the spin-up)

[2017/05/13 02:29:18.236272,  6, pid=12341, effective(0, 0), real(0, 0)] (lp_file_list_changed)
  lp_file_list_changed()
  file /etc/samba/tm_config.conf -> /etc/samba/tm_config.conf  last mod_time: Sat May 13 01:19:17 2017
  
  file /etc/samba/overall_share -> /etc/samba/overall_share  last mod_time: Sat May 13 01:01:40 2017
  
  file /etc/samba/smb-global_veto.conf -> /etc/samba/smb-global_veto.conf  last mod_time: Sat Apr  1 00:40:02 2017
  
  file /etc/samba/smb-global.conf -> /etc/samba/smb-global.conf  last mod_time: Sat May 13 02:20:07 2017
  
  file /etc/samba/smb.conf -> /etc/samba/smb.conf  last mod_time: Sat May 13 00:49:20 2017
1 Like

Hello again,

After wasting 4+ days (I’m not even sure) I think I’ve managed to strip as much as I can and ended up with a minimal installation with (hopefully) great standby time (still testing) and even greater performance.

http://levo.pro/2017/05/13/barebone-my-cloud-gen1-fix-sleepstandby-problems-and-more/

1- disabled as much service as possible, including:

– afp: performance hog, smb3 works great with mac os.
– nfs: never used
– ftp: ssh already offers sftp
– external usb: never used
– logs: waste of memory, never checked before
– winbind: I don’t have any PCs
– cron, media server, etc.

2- cleanup: removed some obvious useless files/folders. once you do it, it may take 10-20mins to reboot, don’t worry and just wait for file system check to complete.

3- ramlog: dropped its size to 10M to prevent swapping.

4- samba
– applied msg.sock tmpfs fix, integrated to init.d samba file
– disable spoolss: disables printer support: improves disk standby time
– added streams_xattr: gets rid of dot underscore files, lightning fast directory listings, and drops cpu usage
– increased deadtime: default is unlimited, improves disk standby time
– disable netbios: disables nmbd which improves disk standby time
– disabled log file: save memory to prevent swapping

5- mDNSResponder
– added smb & removed afp: connect from sidebar

6- monitorio
-– removed tally (/usr/local/bin/tally doesn’t even exists)
–- removed mediacrawler
-– removed individual share size calculations
–- disabled rotateLogs (run-parts /etc/nas/wakeup.d)
–- mount / noatime,nodiratime
–- decrease vm.swappiness
–- added afp fix (killall cnid_dbd) (if you decide to keep afp, this one prevents spin-ups)
–- added sleeptime logging (/var/log/monitorio.log)
–- added log truncation
– decreased swappiness: keep it in memory
– stop cron (a couple scripts restart it)
– added debug mode: stop the service and run it from terminal to pinpoint processes that read/write

Results:

  • no more 7 seconds sleeps
  • stays asleep hours (hopefully will standby days)
  • read/write speeds jumped from 55/40 to 100/85 (due to low cpu use)

Once again, thank you Bennor, rac8006, and Ralphael!

Bonus: https://dl.dropboxusercontent.com/u/81196443/WD%20My%20Cloud%20Files.zip

1 Like

@Levent_Ozler, good job. What kind of sleep times are you looking at now?

In looking through your changes its not clear what impact it would have for those using Windows PC’s. Is there any impact or edit that shouldn’t be made to allow for better Windows compatibility?

Beyond basic file access what other features remain active? It appears you turned off media serving so this means no DLNA I assume. I assume your changes also mean no remote access as well (for those who use remote access)?

Amazing how far we are going to try and fix the mess the WD firmware coders have made with the firmware.

Thanks, last time I left it alone it slept 6+ hours.

Sorry about the windows compatibility, I didn’t have the time to check them one by one. But nmbd (netbios for samba) definitely spins up the disk (every 5 mins or so). So I nixed as much as I can.

monitorio.sh I’ve posted is quite good for debugging sleep problems, you can run it in terminal, and as soon as the disk wakes up it displays process/kernel disk activity and files that were modified/created in last minute.

that is funny… in all my years debugging WD, I never thought of running monitorio by itself in terminal :stuck_out_tongue: If I had to start it up in terminal, I would throw an ampersand behind it to put it into background.

That is unfortunate because now I’m on gen 2 and I no longer have a running monitorio.sh :frowning: Some mysterious program like monitorio now runs in the background and I think it is all compiled into the init.

but Good work on your detective work, now we just need @rac8006 to port it to gen 2 :slight_smile:

Blktrace will give you the processes that write and the addresses that they write. I don’t see in your monitorio.sh
file where you get the process name.
This is my latest sleep numbers. With most processes running. Was going to test the read/write speeds. But currently
streaming video on the same network.
05 13 03:27:22 08:01:11 16429 4:33:49
05 13 08:11:22 15:13:39 25337 7:02:17
05 13 15:25:52 16:01:21 2129 0:35:29
05 13 16:11:32 19:20:40 11348 3:09:08

Let me ask a question. I talk to many IT professionals who manage corporate NAS systems and they all say that spinning down and up of the drives is the #1 reason they fail over time. Almost all of them turn off any sleep feature the NAS has so they in fact stay online. So why is it a bad thing for the MyCloud to not sleep?

There isn’t really any definitive proof one way or the other on spinning up or down a hard drive affecting a drive failure rate or causing a failure. Many IT people “assume” that the constant power up/power down of a drive may or can impact or cause a drive failure.

The problem with the My Cloud, at least mine, is it constantly spins up and down if Drive Sleep is enabled through the Dashboard > Settings > General page. The drive would go to sleep every ten minutes, then spin up 7 or 8 seconds later for example, as indicated previously in the thread. It would do this all day long.

One advantage to spinning down a hard drive (one that isn’t a “green drive”) is the slight power/energy savings when the drive isn’t being used when its in sleep or off mode. Over time that energy savings could potentially (eventually) add up (if one was running a large server farm or the like). For people who don’t use their My Cloud for long periods of time during the day they may want to either spin down or power down the My Cloud (for what ever reason).

Performed Levent_Ozler method and noticed a few things using Windows 10. Comments are in no particular order. Note: I don’t have a WD Red drive in my unit, drive failed and it was replaced with a Seagate ST31000333AS 1TB drive.

Using Putty on Windows 10 the following errors were displayed during the edit process. Used WinSCP to perform various edits instead, particularly the file editing.

WDMyCloud:~# service assimilateNewDrive.sh stop
mdadm: Cannot get exclusive access to /dev/md0:Perhaps a running process, mounted filesystem or active volume group?

WDMyCloud:~# service wdphotodbmergerd stop
[....] Stopping wdphotodbmerger: wdphotodbmergerstart-stop-daemon: warning: failed to kill 10280: No such process

WDMyCloud:~# service twonky stop
PID file /var/run/mediaserver.pid not found, stopping server anyway...
twonkystarter: no process found

WDMyCloud:~# service lltd stop
cat: /sys/class/net/wifi0/operstate: No such file or directory
Shutting down lltd services: OK

WDMyCloud:~# service vsftpd stop
Stopping FTP server: No /usr/sbin/vsftpd found running; none killed.
vsftpd.

WDMyCloud:~# service nspt stop
umount: /media/WDSAFE/*: not found

WDMyCloud:~# service commgrd stop
Sun May 14 08:16:53 EDT 2017 - %W - Communication Manager - process is not running

WDMyCloud:~# edit /etc/default/ramlog
Warning: unknown mime-type for "/etc/default/ramlog" -- using "application/octet-stream"
Error: no "edit" mailcap rules found for type "application/octet-stream"

WDMyCloud:~# edit /etc/init.d/samba
Warning: unknown mime-type for "/etc/init.d/samba" -- using "application/octet-stream"
Error: no "edit" mailcap rules found for type "application/octet-stream"

WDMyCloud:~# edit /etc/samba/smb-global.conf
Warning: unknown mime-type for "/etc/samba/smb-global.conf" -- using "application/octet-stream"
Error: no "edit" mailcap rules found for type "application/octet-stream"

WDMyCloud:~# edit /etc/nas/mDNSResponder.conf
Warning: unknown mime-type for "/etc/nas/mDNSResponder.conf" -- using "application/octet-stream"
Error: no "edit" mailcap rules found for type "application/octet-stream"

WDMyCloud:~# edit /etc/init.d/mDNSResponder
Warning: unknown mime-type for "/etc/init.d/mDNSResponder" -- using "application/octet-stream"
Error: no "edit" mailcap rules found for type "application/octet-stream"

WDMyCloud:~# edit /usr/local/sbin/monitorio.sh
Error: no "edit" mailcap rules found for type "application/x-sh"

Left DHCP to auto since router reserves IP address for My Cloud.

Rebooted unit after making changes (approx 8:43am).

After reboot Dashboard Diagnosis field indicated “Warning” due to no temperature value because it was turned off.

Dashboard seems more responsive after reboot.

Didn’t test read/write speed prior to edit/updates. Forgot. Post edit/update reboot:

WDMyCloud:~# hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   512 MB in  2.00 seconds = 255.46 MB/sec
 Timing buffered disk reads: 304 MB in  3.01 seconds = 101.09 MB/sec

WDMyCloud:~# dd if=/dev/zero of=/tmp/output bs=8k count=10k; rm -f /tmp/output
10240+0 records in
10240+0 records out
83886080 bytes (84 MB) copied, 0.484201 s, 173 MB/s

Because USB edits were made it appears a USB drive is no longer mounted by My Cloud post edits/updates. So no Dashboard Safepoint access.

Windows 10 (didn’t reboot):
While the My Cloud shows up in Windows File Explorer, error is generated when trying to access it by name (assume due to no netbios).

Use My Cloud IP address for Windows File Explorer access.

Copying files across Gigabit network average about 30 Mb/s per Windows File Explorer which I think is where it was prior to the edit/changes.

Because Media Serving is off, My Cloud not seen as DLNA server. Enabled Media Serving through Dashboard > Settings > Media. Once enabled media files (video/music) played in DLNA client (Roku) including subtitles in SRT files.

Will be doing further testing throughout the day to see how it sleeps with Media Serving enabled.

I thought that the object of this exercise was to increase sleep time while maintaining the devices functionality. The
above mentioned changes disable a lot of the devices functions.

The above changes are extreme in the quest to get better sleep times. Unfortunately because of the way the firmware is coded one does have to perform some or many of those changes mentioned above to achieve better than 50% sleep time. On my unit and in my use the biggest thing to improve sleep time with the 315 firmware seems to be disabling Samba.

That is not entirely true. I used to get 80% before firmware 315. Currently I getting more than 80%. Using the chroot method allows for more sleep while leaving most processes running. When I say chroot I mean chroot to a USB disk with the current firmware installed. This stops most reads and writes to the hard disk. It is a little more involved than just chroot. But using a couple of scripts to do the mounts and stop then restart processes. It seems to make sleep better.