[GUIDE] Building packages for the new firmware... someone tried it?

My Cloud OS 3 Personal & Network Attached Storage My Cloud
101

I got the root cause and which is not highlighted in the guide:

It’s easy to miss this step (47) because this snippet can’t execute with default user which needs root access right

Just make sure install binutils*.deb provided by WD to your rootfs.

build-armhf-package.sh

46 if [ “$BUILD_64K” == “true” ]; then
47 mkdir -p $build_dir/root/binutils
48 echo “$build_dir/root/binutils”
49 tar xvf $binutils_tar -C $build_dir/root/binutils
50
51 sudo chroot $build_dir /bin/bash -x <<EOF
52 cd /root/binutils
53 dpkg -i binutils_.deb
54 dpkg -i binutils-multiarch_.deb
55 EOF

102

To the OP (mauromol) or someone with a working 64k wheezy build/compile environment, would it be possible to compile the Zram module in 64k for kernel 3.2.26 on Gen1 myclouds? It could make a mind blowing improvement on those units for compressed swap.

The Zram module appears to have been available regular debian wheezy running that kernel, which is encouraging, but I can’t tell if the 3.2.26 kernel on the mycloud has the Zram support flag enabled because I don’t see any kernel config info in /proc or /boot through SSH. If a recompiled kernel is needed with the module then I’m curious if that would be possible too.

103

Zram is in staging and buggy on the 3.2.26, same goes to ZCache. I also tried porting over Zswap but all these are unstable on the old kernel without panics while I have no issues with them on recent 4.x kernels except for Zcache which has been removed from recent kernels. I successfully compiled 4.1 with the pfe module though without the jnl module (Paragon NTFS, can’t find the armhf source code) but can’t seems to boot on the MyCloud. Had tried several times till I finally gave up. I don’t have the time or UART to access the bootloader to debug further.

105

I’d be interested in the steps that you used to compile and test a kernel?

107

Been looking for the steps to boot from a USB stick.

109

That’s great! Awaiting for your kernel 4.x guides, best if you managed to get the jnl module for armhf working as well due to low NTFS performance of the 3G modules also it’s used by WD auto mount. The source I got from Paragon is not armhf. I’m running a custom kernel but based on the 3.x, just some tweaks and enough to get loop mount, iptables and ipset working for my needs.

111

@mauromol When I built a package for wheezy I was getting resolve errors for the debian mirrors. I had to copy my /etc/resolv.conf to the build/etc directory. You should add that to your guide.

112

What do you have in your /etc/resolv.conf and what’s in the provided /etc/resolv.conf?

113

For v04.xx firmware: BuildTools and scripts
If need - change Jessie to Wheezy or Sid in make_buildenv.sh
Read _howto.txt for usage info.

114

There was no /etc/resolv.conf in the gpl-source 04.00.01-623 I am using. My Ubuntu system’s resolv.conf only contains my ISP’s dns and this line: nameserver 127.0.1.1

115

Just an update.

After some years, since I had some spare time (ugh!), I decided to update the MyCloud firmware from version 4.00.01-623 to version 4.05.00-320 (latest one as of now, April 2018), because it brings a lot of security updates. I did this after verifying that nothing had changed in the process required to install custom packages. The latest GPL source code version published on WD website is for firmware version 4.04.03-113, which was published in 2016, so quite old (I already wrote to WD to request them to publish the latest firmware source code), but it’s already containing the My Cloud OS 3, which seems to be the latest one available (according to the firmware release notes), so I was confident enough that nothing disrupting was brought on by WD with regards to custom package building.

Comparing the two source code bundles, I see that WD has fixed some scripts, but in general all I wrote in the original guide + the patched scripts I provided here are still 100% valid to build custom packages. The only “strange” part is that they commented out the following row in build-armhf-package.sh:

##export DEB_CFLAGS_APPEND='-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE'

I don’t know what could be the implications, anyway in my virtual machine I use to cross-compile packages I left that row because it never hurt. Maybe anyone here with more experience can shred some light on this?

Anyway, the upgrade process went well, although with a huge amount of panic… The first reboot after firmware update took all night long (!!!), with the white solid led being on during this process and the WD My Cloud web UI leading to an almost blank page (with no login prompt) and no way to access the device (no SSH, no Samba, etc.). Since a lot of disk activity was in progress, I left it there overnight, although I already made many research on the Internet and started to think I had to unbrick it in some complicated way :sob:
Next morning, the led had turned blue, so I had new hope! SSH access was not refused any more (although the connection was hanging before requesting the login prompt) and the web UI was keeping the browser connected rather than just spit an almost blank page. Long story short, after about 8 more hours, I was able to access my device again (SSH, web UI and Samba shares) and all seems to be fine. I still have to reinstall my custom-built packages (meanwhile, I built some updated ones because security updates have been published by Debian), but I feel good now. There are some “find” processes still running on the WD My Cloud which seems to be adjusting permissions on the whole /shares/ folder (and this explains why it takes so much time, I have a lot of data in this unit).

Anyway, there’s a severe design flaw in the software WD is providing for this device, it’s unacceptable that a firmware upgrade brings your unit offline for almost an entire day, with around 8 hours just to complete the first reboot. And what if a power outage occurred meanwhile? Fortunately, I have an UPS… but hey, this is a consumer product and I think that any common user would have at least tried to switch the unit off/on and/or to reset it to try to make it respond again… and I can’t imagine what could have happened…

So, be VERY patient when you do a firmware upgrade.

On a side note: some built-in packages provided by WD (like rsync) are not really updated with all the security fixes provided by Debian. For instance, the provided rsync package is still at version 3.0.9-4, while for wheezy the latest available version is 3.0.9-4+deb7u2, containing security fixes. This is one more reason to learn how to build and install packages, although it’s not that simple for common users.
(by the way, for rsync specifically I suggest to build the latest version from jessie repository).

116

I am attempting to follow the post on getting Duplicity working as per the post here: Backup WD MyCloud to S3/Glacier with duplicity (build instructions included , therefore following this post as a pre-requisite.

My unit is My Cloud EX2 running firmware 2.11.176, therefore GPL: WDMyCloud_EX2_GPL_v2.11.176_20180503.tar.gz

Following the instructions above with my limited Linux knowledge, I got to the point of ./build.sh htop.
Initially I got “permission denied” even though logged in as root, I forced the permission changed, then I was told it couldn’t find build-armhf-package.sh. Looking in the GPL download there is no build-armhf-package.sh under the packages directory - there is no “build_tools” directory - and a search for build-armhf-package.sh yields nothing in the download.

Is your excellent post relevant for my model/version of software?

I did try to apt-get via SSH on the box before starting and got nowhere…

117

Hello and thank you, @Nazar78 for your inspiring guide on cross-compiling transmission.
Btw, I noticed that sometimes there is a $-sign missing in the code (e.g. directly prior to the HOME variables). Probably got lost when copy/pasting it into the forum here…

With the recent release of Transmission v3.00, I am interested to follow your guide building it from source for my WD MyCloud gen1 (latest firmware v4.x).
I am wondering about the following aspects:

Is your guide still valid, after almost 6 years?
Or which changes/updates would we have to do to make it work in todays environment and for transmission 3.00?

Can we still go with the original gcc-linaro-archive, i.e. the one provided by WD in the latest GPL-source-file?

Would Ubuntu 14.04 (32-bit) still the OS to build it on? Actually, why did you use the 32-bit version?
Should a newer OS be used today, e.g. Ubuntu 20.04?
And for which reason did you use Ubuntu instead of Debian?

In the step when you get the sources of the dependencies, why is it that you exclude the ubuntu-debian ones (“… grep -v debian …”) and only proceed with the original tar-files?

Looking forward to helpful contributions :slight_smile:

118

Hello @mauromol,
thank you for your guide, I tried your approach too.

As already mentioned here a long time ago, with your patches it easily works to build transmission in the wheezy-chroot, however it is only version 2.52-3+nmu2.

In the jessie-chroot, transmission would be available in version 2.84-0.2+deb8u2.
But, as reported in this thread, the build process aborts with errors from qemu-arm-static, even with the latest one available from jessie repositories.

Now I did a test an simply took the latest qemu-arm-static I could get in my Linux Mint 19.
It is reported being qemu-arm version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.26)
I copied it over into the jessie-chroot (in virtualbox running guest system Debian Wheezy 64bit 7.11.0) and replaced the qemu-arm-static there.
With that change, the build of jessie’s latest transmission finishes successfully! :grinning:

Btw, I am using gpl-source-wd_my_cloud-04.05.00-342.zip

I wonder if this procedure has any way to use parameters for the transmission-daemon, e.g. --enable-lightweight…
Any ideas?

119

This is good news, thanks for reporting!!

I guess a change to the default source package build configuration is needed for this. So perhaps a multi-step approach is needed (download the source package, unpack, change the build configuration, then build the binary packages, assuming the build succeeds). However, this goes beyond my knowledge, sorry.

120

I just found out that the “enable-lightweight” option only changes the default values for some settings:

  • cache-size-mb
  • prefetch-enabled
  • encryption

But of course we are free to simply change these values whenever we want in settings.json.
Source: https://forum.transmissionbt.com/viewtopic.php?t=11780

121

As general feedback:
I successfully built Transmission for the 64k environment of current WDMyCloud v4 firmware.

I based my process on Nazar’s instructions in post #7 in this thread.
It has several typos, though, which I already reported a few posts above, but apparently Nazar isn’t active anymore here in this forum.

Back in last June, I still had struggled and encountered some errors.
But then in December, I did another try:
I didn’t use Virtualbox anymore, but instead worked directly on my Linux Mint 20 (64-bit).
And I used other (i.e. older) versions of libevent and openssl sources.
Then it worked!
So evidently, a 32-bit environment is not required, but you can build on 64-bit environment too.

The transmission versions I built were v2.94 and v3.00

I tested both with success.
Transmission v2.94 happily runs on my MyCloud since then. :slight_smile:

122

try to follow this post’s suggestion but fail.
may i know if there have update for OS5 or not?
or somebody had transferred OpenVPN as bin file , wish who can share it.

123

You might want to try changing to vanilla Debian with the method described at fox-exe
Fox_exe file archive (fox-exe.ru)