My Cloud PR4100 Firmware Analysis


#174

As I dig deeper, I’m starting to discover that most WD compiled binary programs appear to be little more than wrappers for standard Linux programs and utilities. Currently, I’m trying to solve a minor issue where data is not displayed correctly if a given hard drive is asleep. This eventually led to set_pwm, which calls hdparm to do the actual work.

# strings ./set_pwm | grep hdparm
hdparm -C /dev/%s
hdparm -y /dev/%s
hdparm -y /dev/%s > /dev/null

The %s portion is just a variable for the device name, and the > /dev/null portion simply discards any program output which may be returned. The hdparm command switches found in set_pwm are as follows.

-C   Check drive power mode status
-y   Put drive in standby mode

WARNING: The hdparm program is very dangerous. In fact, it can permanently damage a hard drive if used incorrectly.


#175

The firmware has a bug that sometimes prevents S.M.A.R.T. data for certain hard drives to be displayed if they are sleeping or in standby mode. Occasionally, it even displays the data for the wrong hard drive.

After another epic battle, I finally located and fixed the bug.


#176

Look what I found hidden within the firmware source code. It seems that the backups functionality originally had the ability to connect to any remote server, but it was later commented out.

The WD backups functionality (binary files) can’t be trusted to perform reliable backups, so I will likely create a back-end CGI script for the dashboard to communicate directly with rsync.


#177

The dashboard’s Web File Viewer is hopelessly broken. Honestly, I can’t even begin to describe just how bad it really is, at least not without using some very colorful language.

Whoever designed the thing seems to have been solely focused on making it pretty, without giving any consideration to whether or not it was even remotely functional from a users perspective. Case in point…

The Web File Viewer only displays 10 items at a time, and one of the folders on my PR4100 has 4226 items, which translates to 423 pages. Another (larger) folder has 13,617 items, which translates to 1362 individual pages a user would have to scroll through if they wanted to see a file on the last page.

Adding insult to injury is the fact that depending on the browser used, the Web File Viewer sometimes won’t scroll at all, completely blocking access to additional item pages. Not that any sane user would bother scrolling through that many pages in the first place.

Fortunately, plenty of open-source web file managers exist for Linux, but getting them to work on the PR4100 may not be easy because the Linux kernel is ancient. If WD would release additional source code this could easily be remedied, but we all know how that goes. Wait, wait, and wait some more… all to the sound of crickets chirping in the background.


#179

After digging through the source code to see if there was any hope for salvaging the Web File Viewer, I found more surprises. Actually, I was more shocked than surprised.

The Web File Viewer appears to have came from a third party, but was modified by WD developers, not once, but several times. Exactly what they were trying to do, is anything but clear. However, the Web file viewer originally had decent paging capabilities, but the developers chose to hide it and add half-broken Facebook-style infinite scroll functionality.

The infinite scroll functionality works relatively well for continuous streams of new content, but it’s terrible for mostly static content, like that which is typically found on NAS file servers. Honestly, it looks to me as if the decision to alter the Web File Viewer might have came from a liberal arts desk jockey with little to no actual programming or user interface design experience.

As you can see, I have restored the original paging functionality, and while it’s a work in progress, it has already proven to be 1000% more functional than it was before. Now, I can move to any point within the file list within seconds, rather than having to cope with the mess that the WD developers created. Ok, so I am coping with the mess, but still. And don’t ask me what the paging bar looked like before I fixed it, it was solid black, and the page number box was too small for a one digit number, let alone multi-digit numbers.

By the way, the test files in the example image were created using the following one liner, which uses a simple for loop to create a series of empty files.

for i in `seq 1 1000`; do touch /shares/Public/test_file_number_$i.txt; done;

#180

The following list is a basic dashboard dependency tree. It doesn’t include everything, and it differs from the original firmware, where some things have been added and/or removed as I make modifications. The code is an absolute mess, so I created this list to help me keep track of everything.

POST/GET requests, images, etc are not included, and the code depends on XML language files outside of the web root directory.

Root:

/web/index.php
	/web/home.php (Admin Account)
		/web/addons/safepoints_api.php
			/web/lib/login_checker.php
		/web/addons/app.php
		/web/storage/storage.html
		/web/dashboard.php
	/web/myhome.php (User Account)
		/web/myHome/myhome.php
		/web/myHome/downloads.html
		/web/addons/web_file_server.html
		/web/myHome/apps.html
	/web/eula.php
		/web/wizardDiag.html
		/web/php/noHDD.php
		/web/setting/fwDiag.html
		/web/noHDDDiag.php
	/web/noHDD.php
	/web/lib/msg.php

Home:

/web/dashboard.php
	/web/activeDiag.html
	/web/setting/fwDiag.html
	/web/users/usersDiag.html
	/web/cloud/cloudDiag.html

Users:

/web/users/users.html
	/web/function/batch_user.js
	/web/users/groups.html
		/web/function/groups.js
		/web/users/groupDiag.html
	/web/users/usersDiag.html

Shares:

/web/shares/shares.html
	/web/function/share.js
	/web/function/ftp.js

Backups:

/web/backups/backups.html
	/web/backups/usb_backup.html
		/web/backups/usb_backup.php
			/web/lib/login_checker.php
	/web/backups/remoteBackup.html
		/web/function/remote.js
		/web/function/safepoints.js
	/web/backups/internal_backup.html
		/web/function/schedule_select.js
		/web/backups/internal_backup.php
	/web/backups/camera_backups.html
		/web/function/camera.js

Apps:

/web/addons/app.php
	/web/jquery/ajaxfileupload/ajaxfileupload.js
	/web/dsdk/js/constants.js
	/web/dsdk/js/util.js
	/web/dsdk/js/language.js
	/web/dsdk/js/application_ui.js
	/web/function/app_menu.js
	/web/function/appsDiag.js
	/web/addons/appsDiag.html
	/web/addons/safepoints.php
	/web/addons/web_file_server.html
		/web/addons/web_file_server_main.html
	/web/addons/iso_manager.html
		/web/function/iso_mount.js
		/web/function/iso_create.js
		/web/setting/isomountDiag.html
		/web/setting/isoCreateDiag.html
	/web/addons/http_downloads.html
		/web/function/http_downloads.js
		/web/function/http_downloadsDiag.js
		/web/addons/http_downloadsDiag.html
	/web/addons/ftp_downloads.html
		/web/function/codepage.js
		/web/function/schedule_select.js
		/web/function/ftp_downloads.js
		/web/addons/jqueryFileTree.php
	/web/addons/p2p.html
		/web/jquery/ajaxfileupload/ajaxfileupload.js
		/web/function/p2p.js
		/web/function/p2pDiag.js
	/web/setting/media.html

Storage:

/web/storage/storage.html
	/web/storage/disk_mgmt.html
		/web/function/disk_mgmt.js
		/web/function/diagnosticsDiag.js
		/web/function/diagnostics.js
		/web/function/raid_define.js
		/web/function/raid_diag.js
		/web/function/system.js
		/web/function/share.js
		/web/setting/diagnostics_diag.html
	/web/storage/raid.php
		/web/function/raid_define.js
		/web/function/raid.js
		/web/function/raid_diag.js
		/web/function/raid_create_diag.js
		/web/function/raid_reformat_diag.js
		/web/function/raid_remain_diag.js
		/web/storage/raid_diag.html
		/web/storage/raid_create_diag.html
		/web/storage/raid_remain_diag.html
	/web/storage/iscsi.html
		/web/function/raid_define.js
		/web/function/raid_diag.js
		/web/function/iscsi.js
	/web/storage/virtual_vol.html
		/web/function/raid_define.js
		/web/function/raid_diag.js
		/web/function/vv_diag.js
		/web/function/vv.js
		virtual_vol_diag.html

Cloud Access:

/web/cloud/cloud.html
	/web/cloud/cloudDiag.html

Settings:

/web/setting/setting.html
	/web/setting/general.php
		/web/function/diagnostics.js
		/web/function/general.js
		/web/function/time.js
		/web/function/time_machine.js
		/web/function/power_mgr.js
		/web/function/port.js
		/web/function/device.js
		/web/function/recycle_bin.js
	/web/setting/network.php
		/web/function/ftp.js
		/web/function/ip.js
		/web/function/ipDiag.js
		/web/function/ipv6.js
		/web/function/ipv6Diag.js
		/web/function/lltd.js
		/web/function/afp.js
		/web/function/nfs_service.js
		/web/function/ddns.js
		/web/function/device.js
		/web/function/portforwarding.js
		/web/function/remote.js
		/web/function/snmp_v3.js
		/web/function/snmp_v3_diag.js
		/web/function/snmp.js
		/web/function/codepage.js
		/web/function/network_ups.js
		/web/function/dfs.js
		/web/function/active_directory.js
		/web/setting/ftp_Diag.html
		/web/setting/ups_Diag.html
		/web/setting/sshDiag.html
		/web/setting/adsDiag.html
		/web/setting/dfsDiag.html
		/web/setting/snmpDiag.html
	/web/setting/diagnostics.html
		/web/function/diagnosticsDiag.js
		/web/function/diagnostics.js
		/web/function/raid_define.js
		/web/function/raid_diag.js
		/web/function/system.js
		/web/function/share.js
		/web/function/iso_mount.js
		/web/function/iso_create.js
		/web/setting/diagnostics_diag.html
	/web/setting/notifications.html
		/web/setting/notification_diag.html
	/web/setting/media.html
		/web/function/codepage.js
		/web/function/itunes.js
		/web/function/itunes_diag.js
		/web/function/dlna.js
		/web/function/dlna_diag.js
		/web/setting/media_diag.html
	/web/setting/firmware.html
		/web/function/fw_select.js
		/web/setting/upload.html

#183

The PR4100 firmware comes with Docker, which I’ve known for some time, but I never took the time to investigate further. I know what Docker is, just not why it’s installed on the PR4100. As I recall, Docker is not advertised as a feature, and WD certainly didn’t install it without a reason. What is it’s purpose?

Version:

# docker version
Client version: 1.7.0
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 0baf609
OS/Arch (client): linux/amd64
Server version: 1.7.0
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 0baf609
OS/Arch (server): linux/amd64

Info:

# docker info
Containers: 0
Images: 0
Storage Driver: devicemapper
 Pool Name: docker-8:2-9699334-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop0
 Metadata file: /dev/loop3
 Data Space Used: 305.7 MB
 Data Space Total: 107.4 GB
 Data Space Available: 107.1 GB
 Metadata Space Used: 733.2 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Udev Sync Supported: false
 Deferred Removal Enabled: false
 Data loop file: /mnt/HD/HD_a2/Nas_Prog/_docker/devicemapper/devicemapper/data
 Metadata loop file: /mnt/HD/HD_a2/Nas_Prog/_docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.13
Operating System: <unknown>
CPUs: 4
Total Memory: 3.714 GiB
Name: PR4100
ID: XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX:XXXX

Process:

/usr/sbin/docker -d -s devicemapper --storage-opt dm.override_udev_sync_check=true

#184

@dswv42,

Docker used to be WDs tool of choice for installing custom apps onto these NAS units. Then they came up with the new Framework which they released and is now driven from the “Apps” tab in the Dashboard. I personally have docker used on my NAS and run SoftEther VPN in a docker container. Definitely great to have it on there although the docker version installed is getting a bit dated now.

Cheers,

JediNite


#185

Apps installation crossed my mind, but there wasn’t much information to go on, other than the fact that Docker is installed and runs at system startup. Eventually, updates may become possible, but only time will tell.


#186

Here’s something I’ve been experimenting with as a potential addition to the dashboard. It has some limitations, but having a shell prompt available from the dashboard could be handy. Granted, it would have to use a restricted user account, plus I think only making it available if the user enables SSH access would be a good idea.


#187

Check out http://web.archive.org/web/20160706224806/http://developer.mycloud.com/MCDSDKindex.html


#190

The following code is a JavaScript function I created to replace the convoluted function the dashboard uses to convert disk space into a human-readable format. The original function is far more complicated than it needs to be, it doesn’t pad decimals, and I’m not entirely certain that it outputs the correct values.

Decimal (1 TB = 1000 GB):

<script>
// kilobytes = (bytes / 1000)
// megabytes = (bytes / (1000 * 1000))
// gigabytes = (bytes / (1000 * 1000 * 1000))
// terabytes = (bytes / (1000 * 1000 * 1000 * 1000))

function convert_XD(bytes) {
   var str = "";
   if (isNaN(bytes)) {
      str = "0 KB";
   } else {
      if (bytes >= 1000000000000) { // Terabytes
         str = parseFloat(bytes / 1000000000000).toFixed(2).toString()+" TB";
      } else if (bytes >= 1000000000) { // Gigabytes
         str = parseFloat(bytes / 1000000000).toFixed(2).toString()+" GB";
      } else if (bytes >= 100000) { // Megabytes
         str = parseFloat(bytes / 100000).toFixed(2).toString()+" MB";
      } else { // Kilobytes
         str = parseFloat(bytes / 1000).toFixed(2).toString()+" KB";
      }
   }
   return str;
}
</script>

Binary (1 TB = 1024 GB):

<script>
// kilobytes = (bytes / 1024)
// megabytes = (bytes / (1024 * 1024))
// gigabytes = (bytes / (1024 * 1024 * 1024))
// terabytes = (bytes / (1024 * 1024 * 1024 * 1024))

function convert_XB(bytes) {
   var str = "";
   if (isNaN(bytes)) {
      str = "0 KB";
   } else {
      if (bytes >= 1099511627776) { // Terabytes
         str = parseFloat(bytes / 1099511627776).toFixed(2).toString() + " TB";
      } else if (bytes >= 1073741824) { // Gigabytes
         str = parseFloat(bytes / 1073741824).toFixed(2).toString() + " GB";
      } else if (bytes >= 1048576) { // Megabytes
         str = parseFloat(bytes / 1048576).toFixed(2).toString() + " MB";
      } else { // Kilobytes
         str = parseFloat(bytes / 1024).toFixed(2).toString() + " KB";
      }
   }
   return str;
}
</script>

The Linux df command can be used to get disk utilization values, but one has to be careful of the block sizes, which must be converted to bytes before being used in byte-level calculations. Use the -k switch to output 1024 byte blocks for easier conversion. The total block size (1K-Blocks) can’t be used for calculations because it includes system-reserved blocks. To get the total number of available blocks, simply add the used and available columns.

# df -k | grep "/dev/sd[a-zA-Z][0-9]" | sort -g

Filesystem    1K-blocks     Used         Available    Use%   Mounted on
/dev/sda2     1918592012    582813360    1316269244   31%    /mnt/HD/HD_a2
/dev/sda4     951204        1048         933772        0%    /mnt/HD_a4
/dev/sdb2     5810059936    3436562624   2314917828   60%    /mnt/HD/HD_b2
/dev/sdb4     951204        68           934752        0%    /mnt/HD_b4
/dev/sdc2     1918592012    1043817560   855265044    55%    /mnt/HD/HD_c2
/dev/sdc4     951204        68           934752        0%    /mnt/HD_c4
/dev/sdd1     509984        10050        499934        2%    /mnt/USB/USB1_d1
/dev/sdd2     55797102      423500       52388164      1%    /mnt/USB/USB1_d2
/dev/sdd2     55797102      423500       52388164      1%    /usr/local/tmp_wdnas_crfs

The dumpe2fs command can be used to view the reserved block count. Note the block size, which is 4096 bytes per block. Also, the numbers reported by df and dumpe2fs are slightly different, which is likely caused by rounding errors and/or variances in the calculations used by each program.

Block count: 1464077568 * 4096 = 5996861718528 Bytes (5.99 TB)
Reserved block count: 14640775 * 4096 = 59968614400 Bytes (59.97 GB)
Free blocks: 593374328 * 4096 = 2430461247488 Bytes (2.43 TB)

#192

Here’s an example of the stupidity I have to deal with while trying to modify the firmware.

I discovered that under certain circumstances files can’t be deleted using the Web File Viewer. I traced the problem back to the webfile_mgr.cgi compiled binary program, which returns a status message formatted as XML after the operation is complete. In this case, it simply says “error”.

<?xml version="1.0" encoding="UTF-8"?>
<result>
    <status>error</status>
</result>

The Einstein who created the program must have never imagined that one might actually need to know what the error was, and I can’t easily debug the program because… it’s a compiled binary file with no source code provided.


#193

Here’s an strace (installed with entware) of fan_control in debug mode:

https://pastebin.com/By8jGJrt

It’s a loop with a countdown of 60 seconds and then prints temperature and fan info.
Might contain something of interest…

EDIT with more info:

fan_control 0 d

[fan_control.c:424] standby_flag=0
[fan_control.c:511] HD1 temperature 30
[fan_control.c:511] HD2 temperature 31
[fan_control.c:511] HD3 temperature 30
[fan_control.c:511] HD4 temperature 31
[fan_control.c:1939] current board temperature is 30
[fan_control.c:1940] current hdd temperature is 31
[fan_control.c:656] temperature=31
[fan_control.c:1438] current fan_rpm=510
[fan_control.c:776] uP cmd:up_send_ctl temperature 31 87
[fan_control.c:887] calculate temperature=31
[fan_control.c:890] WD daemon send alert
[fan_control.c:268] sleep duration is 60

Dig deeper to get any info on fan_rpm

strace fan_control -g 4
...
connect(6, {sa_family=AF_UNIX, sun_path="/var/run/wdtms.sock"}, 21) = 0
...
sendmsg(6, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="<?xml version=\"1.0\" encoding=\"ut"..., iov_len=202}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 202
...
recvmsg(6, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="<?xml version=\"1.0\" encoding=\"ut"..., iov_len=2048}], msg_iovlen=1, msg_controllen=0, msg_flags=0}, 0) = 253
...

This shows communication with the wdtms socket.

strace -e read=6 -e write=6 fan_control -g 4
...
SEND 
    <?xml version="1.0" encoding="utf-8"?>.
     <data_message>
       <internal_info>
         <version_int>1</version_int>
         <type_string>get_fan_rpm</type_string>
       </internal_info>
       <data>
         <index_int>0</index_int>
       </data>
     </data_message>
RECV
    <?xml version="1.0" encoding="utf-8"?>.
    <data_message>
      <internal_info>
        <version_int>1</version_int>
        <type_string>get_fan_rpm_response</type_string>
      </internal_info>
      <data>
        <fan_rpm_int>510</fan_rpm_int>
        <return_code_int>0</return_code_int>
      </data>
    </data_message>

#194

I’ve been thinking about installing entware so I can finally have proper tools installed on the actual hardware, but I would prefer to do it manually, rather than using the WD app ecosystem. Currently, my firmware is highly customized, and much of the WD app functionality is disabled.

Can you provide any insight?


#195

Sure, have a look at the source code.
LINK

The install.sh script contains all you need.
When you have entware, just use

opkg install strace

A lot of info comes from running wdtms like this:

export LD_LIBRARY_PATH=/opt/wd/lib:/opt/wd/lib/boost
pkill wdtms
strace -f /opt/bin/wdtms -config=/etc/wd/BNFA-thermal.xml

I’m still going through, but you see commands passing by such as

i2cget -y 8 0x4c 0x10

This is polling the fractional byte of the temperature sensor… seems a bit useless to me but whatever.
More interesting is address 0x00 for the internal sensor and address 0x01 / 0x23 for the external sensors…


[FIRMWARE] FreeNAS on PR2100/PR4100 - updated!
#196

It looks like it should be fairly easy to install. Currently, I have some video files copying to the NAS, after they have been copied I will try installing it.

By the way, the fan_control program looks like a side show. The following programs appear to be the main attraction. I had some success getting them to run in a stripped Linux environment, but they will not run on newer Linux kernels. I can run each of them manually, but they refuse to start from the system_init script, and I have no idea why.

#/etc/init.d/S14wdhws start
/opt/wd/bin/wdhws -config=/etc/wd/sprite-wdhw.xml &

#/etc/init.d/S20wdtmsd start
/opt/wd/bin/wdtms -config=/etc/wd/BNFA-thermal.xml &

#197

Digging deeper into the wdtms.

<?xml version="1.0" encoding="utf-8"?>.
<data_message>
  <internal_info>
    <version_int>1</version_int>
    <type_string>HWSetFanSpeed</type_string>
  </internal_info>
  <data>
    <handle_string>0x1eb92a0<handle_string>
    <fan_int>0</fan_int>
    <type_int>1</type_int>
    <speed_int>30</speed_int>
  </data>
</data_message>

And the response

<?xml version="1.0" encoding="utf-8"?>.
<data_message>
  <internal_info>
    <version_int>1</version_int>
    <type_string>HWSetFanSpeed_response</type_string>
  </internal_info>
  <data>
    <error_code_int>0</error_code_int>
    <HWStatus_int>0</HWStatus_int>
    <data_int>0</data_int>
  </data>
</data_message>

And using strace on wdhws gives a lot more info.

Fan control probably works over the serial bus at /dev/ttyS2, with params B9600 -opost -isig -icanon -echo.
Set the fan speed to 30 RPM as follows

{iov_base="FAN=1E\r", iov_len=7}

#198

I think entware is installed, but I’m not familiar with using it. Executing opkg install strace results in a sh: opkg: not found message.


#199

/opt/bin/opkg if you don’t update your path :slight_smile: