My cloud nightmare!

Digging into the Load Average question: 

Have you actually looked to see why the Load Average is greater than 1?

I have…

CloudNAS:/# top -b -n 1 | awk '{if (NR <=7) print; else if ($8 == "D") {print; count++} } END {print "Total status D: "count}'
top - 07:57:46 up 4 days, 17:05, 1 user, load average: 3.00, 3.10, 3.25
Tasks: 90 total, 1 running, 89 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.7 us, 1.1 sy, 0.0 ni, 92.8 id, 1.3 wa, 0.0 hi, 0.1 si, 0.0 st
KiB Mem: 230560 total, 176200 used, 54360 free, 40292 buffers
KiB Swap: 500732 total, 26928 used, 473804 free, 66092 cached

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    3 root 20 0 0 0 0 D 0.0 0.0 0:00.00 cpu1_hotplug_th
  371 root -2 0 0 0 0 D 0.0 0.0 0:59.41 btn_t
 2237 root 20 0 0 0 0 D 0.0 0.0 14:17.55 pfe_ctrl_timer
Total status D: 3

The btn_t process is a kernel thread poling a hardware register to see if the reset button is being pressed.

It’s not dependent on interupts, and not waiting on disk I/O so it’s not interfering with performance.

cpu_hotplug is part of the ARM architecture used to power up and down cores dynamically.   You can tell the process doesn’t actually run much (if it all) after booting (low PID and near-zero runtime.)