My Cloud PR4100 Firmware Analysis


#152

My PR4100 NAS is now running entirely from a USB flash drive, including all required files copied from the wdnas_config (/dev/mmcblk0p6) partition in eMMC flash memory. In fact, I have firmware version 2.30.172 installed in eMMC flash, and version 2.30.165 installed on the USB flash drive.

As long as BIOS doesn’t go haywire, it’s now unbrickable. Rescue firmware… indeed. :wink:


#153

FYI


#154

Hardware control remains an elusive goal, where gaining fan control is the most desirable objective. However, something found within the system_init script has given me an idea.

From: system_init

echo "sysinit: hardware_init.sh"
hardware_init.sh

The hardware_init.sh script currently doesn’t do anything, as it only contains a single echo command. It looks as if the developers are/were planning to move all the hardware related stuff into a new script file.

From hardware_init.sh

#!/bin/sh
echo "hardware init"

Much of the process I employ to examine and modify the firmware involves a divide and conquer approach. First, remove everything that’s not wanted or needed, then isolate and examine the rest.

As painful as it may be, I think I should extract everything related to hardware control, including all dependencies, then further isolate and categorize each element. It’s a long shot, but it might even be possible to transplant the required files to a new Debian installation.

If only WD would decide to release additional source code, especially that which pertains to hardware control… none of this would be necessary.


#155

I looked at using this a while ago to try and get start up scripts to work and found that if I tried using it, there were some other key functions of the NAS that did not start right. I can’t remember the specifics but found it was not going to meet my needs as a result. So you might find if you go down the fun_plug route that some other key scripts and services are not started as a result.


#156

It was just something interesting I discovered while researching other parts of the firmware. As for the route I’m taking… lets just say that I’m altering the firmware from the ground up. Currently, I’m making extensive modifications to the dashboard.

My “modified” firmware works great and it’s dashboard UI is 1000% more intuitive than the factory firmware.


#157

If anyone is wondering why I’m so interested in modifying the firmware, it’s mostly a hobby to keep me busy in my spare time, but I also consider it a good way to learn.

This thread contains a seemingly random collection of findings and observations, but there is a method to my madness. In fact, I’m pursuing three primary objectives.

  1. Rescue and recovery.
  2. Modify existing firmware.
  3. Install clean OS (Debian, etc).

Also, I’m considering the possibility of extracting portions of the existing firmware, for use with a clean OS installation. The firmware is a tangled web of code and compiled binary files, so this objective may not be feasible.

I only wish there were more interest…


#158

I think you are doing great work on de-tangling the mess WD have left. Unfortunately life is getting in the way for me at the moment and not got the time I was hoping to have to contribute. :frowning:


#159

Have you ever considered working on Github as an open source project?


#160

Yes, but it’s not ready for that yet. Critical portions of the firmware are closed-source, and the portions that are open-source are a chaotic tangle of spaghetti code. Not to mention the fact that multiple programming languages are involved.

Also, I’m not certain how WD would react to such a project. They’ve been very tolerant thus far, so I doubt they’d kick up a fuss as long as it was made crystal clear that the modified firmware is not their product. The My Cloud logos within the firmware could also cause issues because the name is trademarked.


#161

Now that I have the firmware running from a USB flash drive, with infinitely more storage space than eMMC flash memory, I think I’m going to attempt to begin running it from a live file system.

Creating a new image.cfs file each and every time I want to test changes is simply too cumbersome. A live filesystem would enable me to make changes, upload them over the network, and see them live immediately.


#162

did you by pass “chk_image”? Is there any side affects?


#163

You are right about WD’s ownership.


#164

I bypassed the chk_image program using 2 lines of code. No problems were encountered.

mount -t vfat LABEL=USB_BOOT /usr/local/tmp
mount -t squashfs -o loop,offset=2048 /usr/local/tmp/image.cfs /usr/local/modules

Now, I’m preparing to bypass image.cfs entirely, in favor of booting to a live filesystem for easier development and testing.


#165

I did the same once when I was experimenting, but reverted it to the original.


#167

Getting the PR4100 to boot a live file system from USB, rather than from the image.cfs file, was easy enough to do, once I overcame a small self-induced problem. The difference between the following lines of code may not seem like much, but they are huge when it comes to the boot process. Don’t ask me how I know this… it was one of those DUH! moments.

ln -s /usr/local/tmp_wdnas_crfs/crfs /usr/local/modules
ln -s /usr/local/tmp_wdnas_crfs/crfs/* /usr/local/modules

#169

Modifying the firmware sometimes requires watching the boot process via a USB to Serial cable. However, I could not get it to connect using uBuntu until today. This will make life so much easier because I no longer have to reboot into Windows if something goes wrong.

My favorite tool is PuTTY, which is also available for uBuntu.

# add-apt-repository universe
# apt-get update
# apt-get install -y putty

The USB to Serial device I’m using has an FTDI chip, which is natively supported by uBuntu. Here’s how to quickly determine if the device is recognized, and what serial line to connect to. In this case, ttyUSB0 is used. The speed should be changed to 115200.

Verify Connection:

# lsusb

Bus 002 Device 005: ID 0403:6001 Future Technology Devices International, 
Ltd FT232 USB-Serial (UART) IC

Identify Serial Line:

# dmesg | grep ttyUSB

[12680.522848] usb 2-1.6: FTDI USB Serial Device converter now attached to ttyUSB0

#170

#171

There is an unanticipated problem with booting from a live file system on a USB flash drive. Everything works fine until a shut down or reboot command is issued. As it turns out, one of the first things the WD system manager CGI compiled binary does is unmount all USB flash drives. Needless to say, if you unmount the root file system, it’s game over. the system hangs.

It’s funny, but most of the problems I’ve encountered have been caused by the stupid WD compiled binary files.


#172

Replicate output of WD smart.cgi compiled binary, to include the RAW_VALUE attribute.

Base Command:

# smartctl -A /dev/sda

smartctl version 5.38 [x86_64-intel-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   200   200   051    Pre-fail  Always       -       0
  3 Spin_Up_Time            0x0027   231   147   021    Pre-fail  Always       -       7433
  4 Start_Stop_Count        0x0032   098   098   000    Old_age   Always       -       2275
  5 Reallocated_Sector_Ct   0x0033   200   200   140    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   200   200   000    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   080   080   000    Old_age   Always       -       15220
 10 Spin_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   100   100   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       673
192 Power-Off_Retract_Count 0x0032   200   200   000    Old_age   Always       -       148
193 Load_Cycle_Count        0x0032   200   200   000    Old_age   Always       -       2126
194 Temperature_Celsius     0x0022   103   094   000    Old_age   Always       -       49
196 Reallocated_Event_Count 0x0032   200   200   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   200   200   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   200   200   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0008   200   200   000    Old_age   Offline      -       0

Extract Data:

# smartctl -A /dev/sda | tail -n +8 | awk -F " " '{print $1, $2, $4, $5, $6, $10}'

ID# ATTRIBUTE_NAME VALUE WORST THRESH RAW_VALUE
1 Raw_Read_Error_Rate 200 200 051 0
3 Spin_Up_Time 231 147 021 7433
4 Start_Stop_Count 098 098 000 2275
5 Reallocated_Sector_Ct 200 200 140 0
7 Seek_Error_Rate 200 200 000 0
9 Power_On_Hours 080 080 000 15221
10 Spin_Retry_Count 100 100 000 0
11 Calibration_Retry_Count 100 100 000 0
12 Power_Cycle_Count 100 100 000 673
192 Power-Off_Retract_Count 200 200 000 148
193 Load_Cycle_Count 200 200 000 2126
194 Temperature_Celsius 103 094 000 49
196 Reallocated_Event_Count 200 200 000 0
197 Current_Pending_Sector 200 200 000 0
198 Offline_Uncorrectable 200 200 000 0
199 UDMA_CRC_Error_Count 200 200 000 0
200 Multi_Zone_Error_Rate 200 200 000 0

XML Script:

# smartctl -A /dev/sda | tail -n +7 | head -n 18 | awk ' 
 BEGIN {
   printf("<\?xml version=\"1\.0\" encoding=\"UTF-8\"\?>");
   printf("<rows>");
} FS=" " {
   printf("<row id=\""NR-0"\">");
   printf("<cell>"$1"</cell>");
   printf("<cell>"$2"</cell>");
   printf("<cell>"$4"</cell>");
   printf("<cell>"$5"</cell>");
   printf("<cell>"$6"</cell>");
   printf("<cell>"$10"</cell>");
   printf("</row>");
} END {
   printf("<page>1</page>");
   printf("<total>"NR-0"</total>");
   print("</rows>");
}' | sed 's/#//g'

XML Script Output: (formatted)

<?xml version="1.0" encoding="UTF-8"?>
<rows>
   <row id="1">
      <cell>ID</cell>
      <cell>ATTRIBUTE_NAME</cell>
      <cell>VALUE</cell>
      <cell>WORST</cell>
      <cell>THRESH</cell>
      <cell>RAW_VALUE</cell>
   </row>
   <row id="2">
      <cell>1</cell>
      <cell>Raw_Read_Error_Rate</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>051</cell>
      <cell>0</cell>
   </row>
   <row id="3">
      <cell>3</cell>
      <cell>Spin_Up_Time</cell>
      <cell>231</cell>
      <cell>147</cell>
      <cell>021</cell>
      <cell>7433</cell>
   </row>
   <row id="4">
      <cell>4</cell>
      <cell>Start_Stop_Count</cell>
      <cell>098</cell>
      <cell>098</cell>
      <cell>000</cell>
      <cell>2275</cell>
   </row>
   <row id="5">
      <cell>5</cell>
      <cell>Reallocated_Sector_Ct</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>140</cell>
      <cell>0</cell>
   </row>
   <row id="6">
      <cell>7</cell>
      <cell>Seek_Error_Rate</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="7">
      <cell>9</cell>
      <cell>Power_On_Hours</cell>
      <cell>080</cell>
      <cell>080</cell>
      <cell>000</cell>
      <cell>15225</cell>
   </row>
   <row id="8">
      <cell>10</cell>
      <cell>Spin_Retry_Count</cell>
      <cell>100</cell>
      <cell>100</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="9">
      <cell>11</cell>
      <cell>Calibration_Retry_Count</cell>
      <cell>100</cell>
      <cell>100</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="10">
      <cell>12</cell>
      <cell>Power_Cycle_Count</cell>
      <cell>100</cell>
      <cell>100</cell>
      <cell>000</cell>
      <cell>673</cell>
   </row>
   <row id="11">
      <cell>192</cell>
      <cell>Power-Off_Retract_Count</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>148</cell>
   </row>
   <row id="12">
      <cell>193</cell>
      <cell>Load_Cycle_Count</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>2126</cell>
   </row>
   <row id="13">
      <cell>194</cell>
      <cell>Temperature_Celsius</cell>
      <cell>103</cell>
      <cell>094</cell>
      <cell>000</cell>
      <cell>49</cell>
   </row>
   <row id="14">
      <cell>196</cell>
      <cell>Reallocated_Event_Count</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="15">
      <cell>197</cell>
      <cell>Current_Pending_Sector</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="16">
      <cell>198</cell>
      <cell>Offline_Uncorrectable</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="17">
      <cell>199</cell>
      <cell>UDMA_CRC_Error_Count</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <row id="18">
      <cell>200</cell>
      <cell>Multi_Zone_Error_Rate</cell>
      <cell>200</cell>
      <cell>200</cell>
      <cell>000</cell>
      <cell>0</cell>
   </row>
   <page>1</page>
   <total>18</total>
</rows>

Raid Health Degraded
#173

Unlike the factory WD dashboard, mine now shows real-time S.M.A.R.T. Data that includes the RAW_VALUE attribute. I also widened the scroll bar a bit, so one can actually grab it with the mouse cursor.

Data

My dashboard also displays real-time volume utilization for each of the 4 drive bays, based on a percentage (used v/s free) of the total capacity of any given volume.This allows it to function correctly with mixed volume sizes.

Volume Display

If a volume exceeds 90% of it’s total capacity, the volume display will alert the user by turning it’s bar red. The second volume in the example above is only red because I lowered the threshold for testing. The volume bars will also indicate if a drive bay is empty, but I remain undecided about how to indicate this. Perhaps a lighter shade of grey will suffice.

And that’s merely a glimpse of the extensive changes I’ve been making to the firmware.