Clean OS (Debian), OpenMediaVault and other "firmwares"

My Cloud OS 3 Personal & Network Attached Storage My Cloud
1440

Hi, I recently bought a my cloud gen2 nas. I’m at a point where I have to change a config that I cannot do without stupid workarounds. I noticed that this my cloud has a castrated busybox running :confused:
For changes in the future I’m now considering an alternative firmware.
Seems like @Fox_exe has done some great work so far.
But I read about slow transfer speed via samba so I want to ask:
Is this issue resolved now? I plan to use debian + omv.

With original firmware now I get ~ 110 MB/s

Is this still status quo?

1441

Yep.
WD make few changes for increase speed:

  1. Custom patch for samba (DMA, Large memory page size, etc support). Boost speed to 70-80mbps.
  2. Custom patch for kernel (Large memory page size). With samba patch - boost speed to 100-110mbps.

First patch - need port to newer version of samba. (Hard for me), second - brokes compatibility with normal software (Like in v04.xx firmware).

1442

Thx for the insight! That’s unfortunate :confused:
That somehow defeats the purpose of the nas. I mean 60-70 MB/s isn’t bad at all but it’s not a gigabit nas anymore.
Is there a way to modify the original firmware so that changes are saved upon reboot or firmware upgrade?
I found this but didn’t get it to work, should it still work?

Other than that I saw your post about custom apps you can install but I cannot find an apps that helps saving changes upon reboot.
Right now I only have to change /etc/exports all_squash to no_root_squash to ensure my rsync backup to the nas is working correct. But in case of more changes I now consider changing the system.
Also I hate vi. :smiley: Isn’t there a nano app?

1443

Rootfs is compressed and stored on nand/hdd. Every boot its unpack to ramdisk, so there two ways for make changes permament:

  1. Modify rootfs: store in different format (r/w) + comress at powerdown or file changes
  2. Modify or make custom application which will make necessary changes at start (Modify start.sh or init.sh in any of installed app’s or make you own)
1444

Why does everyone ignore the builtin capability to make changes on reboot? The gen2 has a system_init script that contains the following code snippets:
nas_model_name=xmldbc -g /hw_ver
for i in ls /mnt/USB/
do
#echo “/mnt/USB/${i}/mfg_${nas_model_name}”
#echo “/mnt/USB/${i}/fun_plug”
if [ -e /mnt/USB/${i}/mfg_${nas_model_name} -a -e /mnt/USB/${i}/fun_plug ]; then
#echo “Into MFG mode”
touch /tmp/mfg_mode
fi
done
if [ -e /tmp/mfg_mode ]; then
touch /tmp/boot_finished
touch /tmp/system_ready

for mfg

mfg_start
else
touch /tmp/boot_finished

restore rebuild speed to default

md_sync_speed.sh max
fi
If it finds these two files it executes the mfg_start. mfg_start executes the fun_plug script. The mfg_WDMyCloud file name depends on the hw_ver as shown above. The mfg_WDMyCloud file must contain the following two lines
mfg_WDMyCloud
sn00py

So you can put any commands in the fun_plug script. It will be executed on every reboot. If you make a mistake and cause the My Cloud to be unstable just pull the USB and reboot. You will be back to a standard MyCloud firmware.

1445

:confused: I fear this is too advanced/risky for me…

Should I save a copy of exports somewhere on the hdd, edit this copy and change the above?
After that copy custom file back to /etc/exports?
Can you help me what to execute and where to place it in the fun_plug script?

1447

There are a few threads describing fun_plug. I think you wrote most of them…

https://community.wd.com/search?context=topic&context_id=93714&q=Fun_plug&skip_context=true

If I had a gen2, I’d certainly be going down that route for my customisation needs.

1448

I don’t understand the post withdrawn entry?

1449

Don’t worry about it; should have just edited it…

1450

Hi Fox_Exe and thanks for your work. I use a single drive Mycloud Gen2 (also have 2-drive but I just use that with the original firmware because of the fans spinning too much, and I also use it to sync my files and backup of iphone photos)
I saw the new Alpine Linux document on your page.
Could you build a newer Kernel (4.14), enable USB Soundcards, and could you also patch in exfat_nofuse? I’d like to use my single drive as a self contained MPD machine.

Thanks!

1451

Thanks @Fox_Exe - I successfully installed OpenMediaVault on one of my 6TB MC Gen2. However, to be able to create shares on this device, two things need to be done additionally:

  • Set the file system type of /dev/sda2 to Linux Filesystem (15)
  • Remove /dev/sda2 from /etc/fstab

Otherwise the device won’t show up when creating shares.

1452

Now I wanted to do the very same thing again on another device which is exactly the same. Followed every step, no error messages. When rebooting, sshd did not come up, thus I don’t have connectivity to the device. nmap shows that there is an IP address though:

Nmap scan report for 192.168.0.244
Host is up (0.00047s latency).
Nmap done: 256 IP addresses (30 hosts up) scanned in 2.70 seconds

$ ssh root@192.168.0.244
ssh: connect to host 192.168.0.244 port 22: Connection refused

$ sudo nmap -sS 192.168.0.244
Starting Nmap 6.47 ( http://nmap.org ) at 2017-10-19 07:11 CEST
Nmap scan report for 192.168.0.244
Host is up (0.00053s latency).
All 1000 scanned ports on 192.168.0.244 are closed
MAC Address: 00:14:EE:0B:XX:XX (Western Digital Technologies)
Nmap done: 1 IP address (1 host up) scanned in 1.97 seconds

Any suggestions? I crosschecked the commands I executed via telnet (still there in another window, everything is correct and the way it is suppsed to be)

1453
  1. Create USB-recovery flash with alpine linux ( see @Fox_exe ftp )
  2. Load recovery
  3. Mount you hdd drive in /mnt/hdd
  4. Check network interface config

cat /mnt/hdd/etc/network/interfaces

auto lo
iface lo inet loopback
auto eth0
allow-hotplug eth0
iface eth0 inet dhcp

  1. remove file

/mnt/hdd/etc/udev/rules.d/70-persistent-net.rules

  1. Reboot and remove USB-flash

  2. Scan nnmap

1454

Quick question: I anyone running alpine linux on the my cloud? I’m interested if the file transfer speed via SMB is better than the 60-70 MB/s on debian firmware here.

1456

@fox_exe thanks again for your works…
just a question regarding to hardware info in your google drive
I have WDBCTL0040HWT-SESN in model and its running My_Cloud_GLCR_2.21.119 firmware …so Its version 2…wdmc gen2,
but on your hardware info BCTL is Gen1…
so can you confirm which is right product number for gen 1/2 in related to WDBCTL

1457

After add pathes (Improved slice) - upto 80MB/s

Hello @Fox_exe,
what patch do you speak? and how to apply it?

1458

See samba sources provided by WD (firmware source code).
Simple find same version of Samba, compare and extract all changes (make patch).

1459

Hello,

I purchased a gen2 My Cloud single bay 4TB a few days ago. It is still in its virgin state, I did not configure anything except passwords and SSH access. I decided I want to install OMV on it, maybe also NextCloud and Omeka, along with TimeMachine and other backups (should the memory allow for it). This means, I might need to install a full Linux on it. But before I do, I would like to make sure, that I understood the architecture of the device. I would be very glad, if somebody could comment on this, or even better, walk me through the device architecture and the available options.

As I see it, the gen2 My Cloud device looks like this:

  • Marvell Armada 375 (ARMv7, dual core, 32bit, Linux armhf architecture)
  • 512MB RAM
  • 256MB Flash Rom
  • 4 TB HDD
  • 1Gbit/s (model unknown to me, I guess it will be from Marvell as part of the SOC)

Disk layout, after boot, is:

root@WDMyCloud /etc # blkid -o list
device                         fs_type             label       mount point                        
--------------------------------------------------------------------------------------
/dev/loop0                     squashfs                        /usr/local/modules
/dev/sda1                      linux_raid_member               (in use)       
/dev/sda2                      ext4                            /mnt/HD/HD_a2
/dev/sda3                      ext4                            /boot
/dev/sda4                      ext4                            /mnt/HD_a4
/dev/sda5                      ext4                            (not mounted)
/dev/sda6                      ext4                            (not mounted)
/dev/sda7                      ext4                            /usr/local/config
/dev/md0                       swap                            <swap>

So, /dev/sda[1-7] are partitions on the internal 4TB HDD.

  • I understand, that the data partition (the NAS as exposed to the users on the network) is /dev/sda2.
    What are the other partitions (except /dev/sda3, that contains the boot images) good for?

  • Especially, what is with /dev/sda5 and /dev/sda6? Both are ‘not mounted’ but consume each 1GB of capacity.

  • Where is the bootloader located? I mean, physically. Is it on the MBR of /dev/sda or is it on the flash-disk?

  • Is LILO in use? Not sure, how this is on embedded devices. I read, that U-Boot combines stage 1 and stage 2.

  • /dev/md0 seems to be physically located at /dev/sda1, which as a Linux swap filesystem and is mounted as a RAID1 device, consisting of a single partition. I wonder why they did it this way, rather than just having an ordinary swap partition.

  • Is /dev/loop0 a Ram Disk? Or is it on the internal flash?

As to what I understand, so far,

  • there is a bootstrap environment, U-Boot, that handles the boot. I assume U-Boot must reside in the system’s flash disk? According to IBM Developer Work’s “Inside the Linux boot process” booting on an embedded device differs from booting on a PC. How does the boot process happen on the factory firmware? Does it happen differently on the CleanOS or Alpine firmware? I would be interested in information, as to where what is.

  • When I install CleanOS or Alpine, will it overwrite the original firmware? (I am going to install Debian, since I want OMV, but, out of curiosity, I would be interested, how the Alpine mod has been engineered as well.

  • How do I remake a modified system into it’s virgin, factory-delivered state?

Currently I like the fact very much, that the data disk does not spin up, when I use the web-admin or the do maintenance via SSH. There seems to be some “lobby” in a RAM Disk or on Flash. Will this be the same, if I install @Fox_exe’s Debian?

1460

Уважаемый @Fox_exe!
Вопрос по установке stretch на устройство GEN1.
В общем ставится и работает. Одно но - нет сети.

Что делал. Во первых попытался скомпилировать в qemu-static chroot ядро 4.14 (LTS) . Ядро 3.2.68 не собирается новым gcc из stretch, можно пропатчить, по идее, но решил с последним лтс пробовать.
Значит собралось, устройство грузится, погонял тесты не виснет не греется, то есть в принципе работает.
Попытки скомпилировать pfe провалились. И на железе, и на ББ в чруте. Слишком большие отличия версий ядра, наличие в драйвере большого количество вызовов “объявленных устаревшими” функций и заголовочных файлов ядра. Как мог поборолся, что-то поправил просто игнорированием ошибок компилятором (типа типов возвращаемых указателей) где то убрал вызовы старых заголовочных ядра, заменил на новые и т.д. Короче собралось, но как и предполагал не работает - видимо модуль надо переписывать кардинально, это не мой уровень.
Полазил по сети, нашел что-то от comserto на гуглегит, попробовал выдернуть pfe драйвер оттуда, то-же под старые ядра.

Если оставить любое ядро из предыдущих версий (все собранные Вами 3.хх) которые прекрасно работают на жесси, проблема вылезает опять с пресловутым модулем pfe:
лог загрузки модуля (часть)

[ 2714.523977] pfe_firmware_init: request firmware class_c2000.elf failed
[ 2714.530642] pfe_hif_exit
[ 2714.533214] pfe_hif_release_buffers
[ 2714.536864] pfe_hif_free_descr
[ 2714.540000] pfe_hif_lib_exit
[ 2714.543166] pfe_hw_exit
[ 2714.546830] pfe: probe of pfe.0 failed with error -110
[ 2714.553023] initcall pfe_module_init+0x0/0x1f [pfe] returned 0 after 184812 usecs

Не хочет подгружать фирмварь, и следовательно нет сетевого интерфейса.
В загруженных модулях висит:
[root@MyCloud ~]$ lsmod
Module Size Used by
pfe 353617 0
Но сети нет.

В общем, прошу подсказать куда рыть. Пересобирать Ваши ядра не вижу необходимости - все одно придется собирать с тем-же кросс тулчейном, в общем на выходе будет то-же самое.
Что может мешать в stretch грузить модулю pfe фирмварь class_c2000.elf ума не приложу. Ядро то-же, модули и фирмварь на месте, модуль pfe подгружал и через wd-network.sh, и вручную (задавая параметры в modules.conf) и замаскировал запуск сети systemd совсем, запуская все сервисы вручную и в разных вариантах (думал очередность загрузки что-то ломает) - не пойму в чем дело.
Или в стретч что-то с базовыми библиотеками системных утилит несовместимо с этим pfe - не знаю.
Может кто проходил этот путь?
Извиняюсь за многословие.

1461

@w00zy, по русски лучше на 4pda общяться. Местные не поймут :slight_smile:
В остальном - да. Надо переписывать и основательно (Проще с нуля написать). Но не мой уровень - я всеж не прогер по образованию, хотя и быстро учусь. Если очень хочеться - гугли на тему “Почему не загружается firmawre в linux kernel”. Может наткнешся на похожую ситуацию.