Hanging smbd processes taking 100% CPU

Hi,

I’m trying my luck with this forum, hoping to get some useful feedback or things to try. Here’s the scenario: my parents have a MBL, and two computers which are set up with SmartWare to create a backup for them. That’s the only purpose the MBL has, they’re not interested in it as a media server. Both of them use Outlook, and for both of them SmartWare is backing up their Outlook .pst file. Especially my mom likes to keep many of her mails and her Outlook .pst file has grown to 1.7 GB. My dad’s Outlook file is much more modest, at 150 MB.

At some point the MBL’s transfer speeds slowed down to a crawl: 55 kB/s. After enabling SSH, I was able to trace it down to (then) two smbd processes running for user ‘nobody’, each competing for the CPU, effectively taking all of the processing power of the MBL. We power cycled the MBL, and after reboot transfer speeds were back for normal. However, this kept happening and I started to check more carefully. I tracked down via the log files that all of the hanging smbd processes were connected to via the SmartWare service. With the smbstat command, I found out that all of the hanging processes were busy backing up my mom’s Outlook .pst file, but never finished. They got stuck creating a temporary file such as

SmartWare/WD SmartWare.swstor/USER-PC/Volume.01a1ebc6.3791.11e1.9fe7.806e6f6e6963/Users/User/AppData/Local/Microsoft/Outlook/ad33e22a-9e16-4f1e-ac4b-677b9e94d518.tmp

Now here’s what I think is happening: my mom checks her e-mail, and because of the change in the Outlook file, SmartWare starts to make a backup to the MBL. However, my mom has the habit of quickly checking mail before going to bed, so after having closed Outlook she simply shuts down the computer. Therefore, the transfer to the MBL gets interrupted. According to the logs, most of the times this is handled graciously by samba, and the process governing the connection is simply terminated. But for some reason, 1 out of 6-8 times this goes wrong and the process hangs without closing the connection. According to the netstat command however, there is no actual connection anymore. I have gone as far as connecting to a hanging process with strace, and the output was this:

[…]
splice(0x16, 0, 0x12, 0xbffbffa8, 0x10000, 0) = 0
splice(0x16, 0, 0x12, 0xbffbffa8, 0x10000, 0) = 0
splice(0x16, 0, 0x12, 0xbffbffa8, 0x10000, 0) = 0
splice(0x16, 0, 0x12, 0xbffbffa8, 0x10000, 0) = 0
splice(0x16, 0, 0x12, 0xbffbffa8, 0x10000, 0) = 0
splice(0x16, 0, 0x12, 0xbffbffa8, 0x10000, 0) = 0
[…]
etc.

This is as far as I got. It seems to be really a bug in the MBL software that I can’t fix. I think it’s a very common usage scenario that my parents have, and I believe that therefore many customers can be affected by this behavior. As a side effect, the temporary files that are created are never removed, yet they are all the same size as the Outlook .pst file, 1.7 GB. Therefore, disk space is going down much more rapidly than the expected 1.7 GB x 5 versions.

I hope someone here can point me into the right direction, so far customer support’s best answer was to just press the reset button :frowning:

Thanks for any help, info, clues. I’m happy to provide more info if necessary to debug this.

EDIT: The issue I’m seeing looks a lot like this: https://bugzilla.novell.com/show_bug.cgi?id=475998

The main difference is that I actually can kill the smbd processes with kill -9

Check if you have the latest firmware installed and the last Smartware version. 

Thanks for the reply. It’s definitely not the latest firmware. When I unpacked the MBL, it notified me of a new version of the firmware, so I let it try and update itself. After an hour of waiting without any progress what so ever, I decided to reboot the thing and was very relieved that it wasn’t bricked. For some reason the update process just stopped somewhere, and I vowed not to try again unless really necessary.

It’s the latest version of SmartWare that’s installed. The current kernel version is 2.6.32.11-svn45595 and samba is version 3.2.5. If anyone can tell me if one or both are updated in the newer firmware, I would reconsider updating the MBL.

Thanks!

Can anyone tell me the version of the kernel and samba supplied with the latest firmware?

Thanks!

It’s not the same fault, but it reminds me of an issue I had with the Samba service trying to use Microsoft’s SyncToy tool. Samba kept panicking and the kernel restarting the service. If it’s an issue on the MBL, it’ll be with the Samba service.

I’ve not seen this particular error, but I am wondering if it’s related to the SyncToy issue I experienced quite a while ago.

I’m not sure, but I at least think that your answer from ‘support’ was equally crappy as mine :wink:

Seriously though, Samba has been updated extensively since 3.2.5, and if not going for the bleeding-edge version, WD should at least install a stable version that still receives bug fixes from the Samba team. Who knows how much misery a newer version would solve?

Anyone? Plenty of people here have SSH enabled, so if anyone with the most recent firmware could type in

 uname -a and smbd -V , which will provide the version information, and post the results? Thanks

As a sidenote, I got an interesting reply from support:

“I would not recommend you to update the version of the firmware. There’s a 1% chance of having the UI locked up”. That is for the current firmware update. I’m even more happy now that I didn’t brick the thing after unpacking and letting it auto-update (which failed).

As requested.

NAS:~# uname -a
Linux NAS 2.6.32.11-svn48181 #1 Thu Sep 15 18:22:06 PDT 2011 ppc GNU/Linux
NAS:~# smbd -V
Version 3.2.5
NAS:~# cat /etc/version
02.10.09-124
NAS:~#

Thanks a lot! Means that my kernel ( 2.6.32.11-svn45595) apparently has some bugs that were patched, and samba is untouched. So a firmware upgrade could ameliorate this situation, but there’s no proof that it will, nor a confirmation of the problem by WD. Coupled to a warning by customer support not to upgrade, I guess my parents will simply have to work around the problem by evading SmartWare (though I think it’s a MBL kernel problem).

If it of any help, the last few firmwae upgrades I’ve achieved by manually copying a manually downloaded firmware upgrade file and directly invoking the script to upgrade from a file.

I also have had problems upgrading using the WebUI.  The last attempt the WebUI looked liked it was uploading the firmware file from the computer but nothing was happening.

What I would suggest is to turn off ALL the additional services in the Web UI before the upgrade and reboot the MyBook Live.  Once you’ve done that SSH to the Linux command prompt and stop (not needed but I like to be over-cautious) netatalk service.

Copy the firmware file to the MyBook Live using SFTP (FileZilla can do this) and then use /usr/local/sbin/updateFirmwareFromFile.sh to invoke the upgrade.

Word of warning…  DO NOT TERMINATE THE SSH Telnet SESSION WHILE THE UPGRADE IS IN PROCESS!

Thanks for the additional information, I’ll try it like that, should I find the guts to upgrade. Looks more trustworthy a procedure than via the web UI.

In order to prevent crashing SSH connections/kids tripping over ethernet cables/routers with a power failure mess up the upgrade, I can recommend using the screen command to run the upgrade in a virtual terminal environment that doesn’t get killed when you disconnect (unless you want it to). It needs to be installed, but I got it running without a problem on my own MBL for a month now. I have SABnzbd, CouchPotato and Sickbeard running with it flawlessly.