Got it to work. Used the guide I had previously followed and accessed the mycloud server through SSH: http://community.wd.com/t5/WD-My-Cloud/GUIDE-Transmission-2-82/m-p/619597/highlight/true#M1322
I had to stop the transmission client, set transmission to run as root, disable dashboard login and RPC whitelist. Then I started the client again, saved my changes and exit - and now it works hurayyy!