What on earth is convert process doing?
I am running this as a straight file server with everything turned off. What is it converting?
4 single JBOD drives.
Its not serving anything and CPU is maxed with “Convert” process:
Another user had this issue as well. A reset seems to have fixed it for him.
But if you’re familiar with Linux, you can SSH in, find the process ID of the convert process, and look at what files it is accessing…
Mine was going on for dayyys. I followed Tony’s advice (he is a wise man!). SSH , find the process that is causing the issue.
If I remember correctly, my EX4 GUI was showing the CONVERT process and SSH was showing a decompression process - UNLZMA. Found no real reason behind it.
Some things to know: I ordered the NAS with 16tb. All drives already installed when I received it. Set it up, transferred a ton of stuff and waited, and waited, and waited…
Once UNLZMA was discovered and there wasn’t a known reason that I could think of I decided to move all data off of the device, convert to RAID 1, test it with a few files, then converted back to RAID 5. Worked like a charm ever since.
I did move smaller chunks of data after the conversion to see if I could find the culprit that put the EX4 into that ugly stall but nothing triggered it on the 2nd go.
I know it is not what you want to hear but it might be worth considering if that process turns up as the offender inside of SSH.
By the way … after everything was working properly the convert process inside of the GUI would actually show file names that it was working on. So, if it was converting a photo named “chickendinner” the GUI would show something like:
CONVERT chickendinner
Hope that makes sense!
I could us SSH to see, but as another “Moderator” so mistakenly copied his cop out script: “using SSH would void your warranty”!? LOL That really not the point, and yet is the point.
WD!, Whats causing it and FIX IT!
It funny because again. I have nothing enabled! and yet it continues like the energizer bunny…
Honestly, rebooting is not the answer. Its running something “without anything running” and it needs to be fixed! It did stop after a while, but I again moved some more files and it started running again forever. What is it “converting”!? What is the processes purpose!?
Its running for some faultiy reason, when nothing is enabled. WD, FIGURE IT OUT!
I really want to like this thing, but it is such a pain! (and I have debugged devices from B0fA, S0ny, M0torola, and M$! WD, takes the cake with instilling a forum for the users to “figure it out for themselves”… (and if all else fails, “format and recover” or “reset it” or “is it plugged in”!?
Okay! haha… this is what I love about forums and the internet in general. Keeping it constructive would be nice…
I was simply stating what worked in my case. I’m the proactive type and I don’t have a problem with trying to get something working to my level of satisfaction. I don’t wait around for something to magically start working or for the manufacturer to parachute in with a tool kit to start working on it. Typically people come to forums when there’s a problem.
And I think that’s why you are here … no? I’m sure that we’re a pretty small percentage compared to those that are actually working.
Lets clear some things up:
-
Using SSH may void your warranty based on my interpretation of WD’s statements and warnings. The switch woudln’t be there if it was a guarantee void of warranty. If you read what they wrote - modifying the device outside of normal operation is going to get you burned. Checking on a process via SSH and the top command shouldn’t (in my opinion!). And if it does, and you’re worried about it, then don’t post it.
-
I wasn’t suggesting that you reboot to solve your problem. If you read my post I detailed exactly what happened in my case since it looks extremely similar to yours. I reset based on advice. No luck.
You keep talking about the symptom (which appears to be exactly what I had happen) and I’m promising you … that CPU is going to stay that way. It is not going to change if it is stuck in a decompression routine like mine was. I gave it 8 days… nothing changed.
Why mine went into a dcompression fit is beyond me. I had thrown too much data on there to trouble shoot and once it was all off, it kept going. If you want a 100% fool-proof method of archiving data then I suggest a pack of pencils from Staples - don’t forget a sharpener!
And to be fair - I’ve poked around other NAS manufacturer’s boards and guess what?!?! They have issues too!
RobStach wrote:
Okay! haha… this is what I love about forums and the internet in general. Keeping it constructive would be nice…
And to be fair - I’ve poked around other NAS manufacturer’s boards and guess what?!?! They have issues too!
My appologies if it was not recieved as constructive , because your right, thats why we are here. And yes, of course I know other mfgs have issues. (I do live in the real world) ;]
My post was more cynical and based out of frusteration from:
- What incorrect support I have recieved.
a. The complely proposterous statment of telling me inserting a drive not on the whitelist would void my warranty.
b. The equallly obsurd of using SSH.
2. The lack of support / response now.
a. Seems like it is a common 4 of us actually trying to raise issues here, but on the flip side there is no representation from WD asking for any info or factors… (Hard to debug if you dont have all the facts from beta testers actually seeing the issue)
And lets also be clear, if you expect beta support, you should be compensated as a beta tester as is costomary across the industy. ;] (maybe thats what WD is trying to avoid) lol
I deal with a LOT of tech companies as mentioned, and I simplify things down a lot in places like this. The reality is you never really know who people are and what they do for a living. (So its funny to a dissapointing extent)
Yes, I have an EX4.
1 Like
EX4Shot, both of those above posts were directed at Captain_Panaka, not you… 
I still haven’t figured out how to quote here…
Tony’s right. My message was half and half – I should have clarified that! Being contructive was aimed at the Captain…
EX4Shot:
I hear ya man … honestly, I just want to help. I know exactly what you’re feeling and I too was highly disappointed during my first couple of weeks of owning this NAS.
I had shopped around, compared, and even with the price-to-performance rankings I still went for it. Mostly because my other WD products have all held up in the past.
Got it set up then the initial buyers remorse set in… I was really PO’d since I had paid a premium and things weren’t even close to being what I considered acceptable for the average end-user.
There were no real answers once the culprit was identified. WD support DID contact me with some generic procedures that didn’t work.
I figured that I’d need to pull all of my data back if I was going to return it anyway so I might as well try the re-format once everything was cleared off. Since I didn’t set it up (it arrived fully configured) I had no idea what the status was prior to receiving it. I didn’t ever think that it would be a problem so I never checked the CPU usage.
I really wanted to like this unit too… I love the look of it, the GUI, and how my other WD devices integrated seamlessly. If only it would cooperate.
I don’t know if you moved or copied your data. I moved it so the process of moving it back was long and painful. Mostly because I had no idea if things would get better but again, it had to happen anyway.
So - it took about a day to get straightened out. The reformats go way quicker than I expected. The wait time is mostly associated with getting data back on the NAS. I went slow and checked for abnormal activity after each chunk. Everything went fine… here’s where I sit today (I just moved over a couple of image files as well!):
I hear, you but his post does have some vale too. I often irritate the PMs in these forums, because I do manage from fact, and that is sometimes hard to hear from who they see as a potential 5th grader with a broken toy. ;]
The fact is simple they have a process leak on something.
Like I said, I really want to like this thing , but Im about to ship it off. (or throw it against the wall) ;] I have invested too much time already… I could be past the point of return… (Transfering 6-8TB off this thing at 15MBs could take more time than I would like if I move to a $ynology…)
Some more info you might find helpful…
During the convert process, most of my high-res images take about 7 seconds to convert. I had a ton of shots from my Canon DSLR that were moved over.
So, with a little math you can quickly figure out how long it should take to convert images. Note: I haven’t found any documentation stating whether or not the convert process happens with media sharing turned off. I haven’t concerned myself with it since the device has been behaving!
Yep - the CPU activity seems to have an impact on transfer speeds as well. So it is a double kick-in-the-@#&# because of the additional wait time.
I hear you man!! Trust me.
But seriously, give it a try once you get your data back locally. It will take about an hour or so. Then move a few big files over and see what happens. I’m betting that you will be happy.
RobStach wrote:
During the convert process, most of my high-res images take about 7 seconds to convert. I had a ton of shots from my Canon DSLR that were moved over.
Note: I haven’t found any documentation stating whether or not the convert process happens with media sharing turned off. I haven’t concerned myself with it since the device has been behaving!
Thanks Rob, but thats my exact issue! I have nothing turned on! And my earlier point of resetting is only going to have the process probably kick up again in the future until its fixed. IF im peged at 100% using nothing what in the world is going to happen when I want to do more with this besides hold my door open! ;]
What is it converting? Is it actually changing images or only indexing? (I pray for the latter) but the fact is we dont know because no one from WD is here.
Its just a plain and simple problem. It all boils down to:
-
less than 1/2 the bandwith (of gigabit realworld throughput) to transer data onto the device = twice the time
-
Then I cant use the device for 8+ days prior becuse it has to “convert” for nothing. = original time x a factor of 8 days makes it almost unusable as the device is intended. (and your right throughput fall further when this is happening in case you need to transfer more!)
And as we all know… time is $$$.
I hear you!! Trust me.
So again - once I reformatted, whatever was causing the convert process to get locked in the loop was gone. Never happened again - with the same exact data.
Did you install your drives or did they come preconfigured? Mine were preconfigured … not sure if that has anything to do with it.
What I do know is that during the initial transfer I was indeed poking around in the GUI. Had 500 errors, had to physically reboot, etc… who knows what happened during that whole mess.
After the reboot I let the transfers happen without getting too involved. My speeds went up considerably (still nothing close to Tony’s though but it could be something on my lan).
I know you’re focused on the “why”, and so was I. Once I cleared the problem up it shifted to “when” for me. And since “when” hasn’t happened again I could only chalk it up to “whatever”. :smiley:
Beyond that I can’t tell ya… if anything, it’s worth a shot no?
EX4Shot wrote:
Thanks Rob, but thats my exact issue! I have nothing turned on!
Do you have Cloud Access turned on?
If you do, I believe that the covert process will run.
This is what I will see if I “Rebuild” the cloud access data. Media Server is turned OFF on mine.
Also, I’m not sure if I would take the CPU percentage shown in the UI to the bank. According to the UI Convert is using anywhere from 90% to 100%, but when I run “top” via SSH my CPU percentages are no where near staying at that level. You have to keep in mind that the UI has somethng like a 5 sec refresh rate, so you’re CPU may not be staying at that level, it’s just that is what it may have been between refreshing.
Yep - that looks exactly like what I was seeing during my legitimate convert process. Not pinned to 100%…
In my case, the UNLZMA process was sticking between 95-98% for the majority of the time. It would drop off occasionally but then it’d be right back.
Convert will definitely take resources but it should look more like an EKG than a massive coronary!
All great points!!!
Well, my graph may be a little misleading in the “EKG”, up and down process.
After the Cloud Access rebuild has been going a while the graph will stay pegged at about 100%
This is what it looks like after about 20 min.
However, I was just trying to explain what was causing the “pegged out” issue. If you have Cloud Access on then that is going to be more than likely what’s causing the issue.
Also, even though the UI shows “Convert” eating up most of the CPU, there’s more processes running than it.
By the way, " What’s it converting?", evey image you have on your EX4.
Thanks for the clarification.
I have about 8k or so images in my public folder with cloud access on. I did let that run overnight without constantly looking at the CPU graph. It was still going in the AM but my “EKG” was definitely had more fluctuations similar to your first graph. If it was pinned all night I cannot say and perhaps I was at the tail end of the whole process. You make a great point there!
My understanding is that the convert process should take 100% of the CPU. So, your 2nd graph makes sense to me as well.
As an estimate (probably high) if it were taking 10 seconds to convert each image we’re looking at 22 hours of conversion time. It can be assumed that your CPU would stay “pegged” for 22 hours. The same scenario with a 7 second conversion rate (which is what I am seeing) drops that down to roughly 15 hours which is probably why when I checked in the AM everything was winding down.
By the way - this was all after my first go at it in which it was stuck in decompression mode (but showing CONVERT in the GUI) for more than a week.
I think that some things on the WD side of the house would help make this whole thing much less dramatic:
-
Clarification on expected conversion times. If you don’t know… 15 hours can seem like an awful long time. Especially if geared towards soho.
-
Maybe a graph within the GUI showing a completion rate. That’d be nice!
-
A warning somehwere within the transfer process stating that conversion will occur and what to expect (**bleep** … even in the manual would be a nice start)
-
As you are pointing out, the trigger settings that will generate a conversion (Cloud, public, etc…)
Knowing all of this going in might cut back on the confusion / questions / etc…
As to what the expected conversion time should be. I don’t know.
Like I said, there multiple operations happening at the same time, plus, I believe that there is multiple coversions for the same image. I’m not really sure about this, but I believe that not only does it convert the image to a lower quality image, but it also converts it to a thumbnail.
I also belive that instead of preforming one operation at a time for all images, that it performs all operations on a single image before it moves to the next image. This is probably why it takes so long for it to finish. But that’s just going by what the “top” output shows.
Per image time though is probably a variable, since it seems that different image sizes are handled differently. I say that only because it seems as though it preforms “convert -size 1024x…” only on images larger than 1024x? and smaller images don’t appear to go through that conversion.
Wasn’t questioning you… just throwing out some numbers and thoughts based on my experiences since December. Obviously, I’m relatively new to the EX4.
Like I said, based on my personal 7 second rule AND the timing of things on my end, the math gets me right in the ballpark of how long I can expect to get my standby mode email and solid blue lights.
My conversions take about 7 seconds… top output shows single images being converted (just like you said). Once finished, it moves on to the next image and so-forth. Also - if you stare at the GUI for long enough you’ll also see the name of the image being converted but it isn’t pretty and can be hard to read (at least on my machine).
If the EX4 converts other file types I’m not sure. My results are based on image conversions only.
Again - if this was documented by WD somewhere it would help end-users understand what they should expect, generally speaking, when they begin to move files to the NAS.
