BTW, the only way that I know to determine the size of a module is to begin with a sector count of 1 and read its first sector. Then change the sector count to reflect the size of the module, and read the entire module.
Thanks Franc, that’s what I did actually. Read the first sector, grab the sector count from Byte number 11 (if we start from 0x00 then it is 0xA, i.e. 2 bytes before the first checksum byte), then read the whole mod by multiplying it with 512.
Currently it is able to :
Read the whole Mod 0x32 and save it into a file.
Set offsets 0x84 - EOF to 00
Clear the checksum in order to calculate the total of each 32 bits value.
Recalculate the checksum and store it in offsets 0xC-0xF
Save the modified Mod 32 into a new file.
*** Since my drive is good, both original and new files’ md5sum are identical ***
Read the new file, determine its length and number of sectors (filesize divide by 512 - a bit lazy to code to read it from Byte 11) into memory buffer and write from memory back to a third file (instead of writing to my hard disk)
*** Validated that the third file’s md5sum is identical with the first two ***
I think we are almost there. Let me think carefully if I want to lose my 160GB if things go unexpectedly Or perhaps generate some non-0 values in 0x84-EOF, write to the hard disk, retrieve and see if my program gets to fix them.
I believe all AF drives still use 512-byte sectoring in the System Area.
Think I’m done coding for Mod 32h. Now for Mod 02.
I’ve read http://malthus.zapto.org/viewtopic.php?f=86&t=848 and it was mentioned that it’s Mod 02 Record 1A Offset 2, but in the example you gave earlier, it was Record 1B Offset 2. Which is correct?
Also, my code will zero fill anything from 0x84 onwards. If this is incorrect, I need you to tell me where to start.
Please give me some time to get my head around MOD 32 (and MOD 02).
Just today I received some feedback from someone who is using a professional tool. There are some differences between what the tool does and what the thread at the HDD Oracle is suggesting. I have also been offered a broken WD drive for testing.
The “section” number for MOD 02 is 1B, not 1A. Initially I couldn’t decide whether the numbers should begin with 0 or 1, but I eventually settled on the latter.
Rather than “contaminating” this thread, I’ll post my findings in respect of MOD 32 to the HDD Oracle “WD slow issue” thread. That way I’ll be sure to attract the attention of interested parties who may be able to help fill in the blanks and correct any errors.
This is what I have managed to come up with:
Offset 0x58 = number of entries in relo-list
0x5A = number of reallocated sectors
0x5C = number of pending sectors
0x4C = number of remaining spare sectors (= total spares - reallocated sectors)
0x52 = 0x78 = total available spares = max entries in relo-list
0x68, 0x6C, 0x80 = somehow related to drive capacity ?
0x30, 0x34, 0x38, 0x3C, 0x40, 0x44, 0x48 = locations of data sections
0x50 = number of entries (words) in first data section ?
I’ve just been informed by a user that the latest demo/trial version of WDMarvel has a solution for the “slow issue”, so it may be that I’ve wasted your time. I’ll still post my findings in regard to MOD 32, though. That said, I know that there are times when one needs to rewrite various MODs, so your code could still be useful in the future. All I would need to do would be to adapt the MOD ID. In fact I suspect that there may be numerous common data recovery cases that could be solved with freeware tools and some inside knowledge.
No worries Franc. It’s good to write some real program, though I needed Christophe’s code as reference since I’m not a programmer and we’re talking about a very low level access to the hard disk. I might be able to modify this further to dump out all the mods but I see very little value in doing so since there are already tools capable of doing this.
I’ll update the code to include the calculations given in those offsets in the next few days.
0x68, 0x6C, 0x80 = somehow related to drive capacity ?
Looks like it. Can’t tell why the After value is smaller than the Before value.
0x68-6B should be related to bytes per sector. I checked a few of the AF drives and they have a very small value. If I multiply the 32bit by 8, I get a value very close to the non-AF drives.
WD5000LPVX : B6 1D AB 07 * 8 = B0 ED 58 3D, very close to the value of the non-AF drive.
B0 ED 58 3D = 1029238192. Multiply by 512 and we get 526 969 954 304 (~500GB)
What’s annoying is that there are obviously many people doing the same kind of “research”, but few, if any, are willing to share. If this information was out there, then we would have numerous freeware tools that would empower DIY-ers.
I guess it’s only a matter of time before more people hop in and publish their findings. The research you’ve done and shared so far has helped a lot.
It just came to my mind that the larger-than-actual-capacity values in 0x68, 0x6C, 0x80 could be due to each sector’s overhead (GAP, Sync, Addy Mark headers and ECC). See http://www.seagate.com/tech-insights/advanced-format-4k-sector-hard-drives-master-ti/. There’s also some books written about this. I’m also making a wild guess that these larger 32 bit values before the defects were cleared represent the total space used (default allocated sectors + spare sectors). So when more spares are used, the larger it becomes.
I’ve been trying to port my APM prog to DOS/Win32 the last few days. Managed to get an EXE file but the IOCTL portion failed to function - apparently a recode is required. Otherwise we can have a Win32/DOS open source to dump the mods I got the source from hdparm 6.9 and managed to compile into EXE and run it. hdparm 9.x (and probably versions after 6.9) uses sgio (sg = SCSI Generic) - this probably explains why no one made any Win32 versions after that.
Update : The sgio.c looks very similar to scsiata.cpp from the smartmontools project. Let me see if I can utilize scsiata.cpp to port this to Windows or the sg3_utils
I’ve been analysing MOD 32 on and off. I’m having difficulty finding consistency between my various examples. It seems as if each example has some aspect that differs from the others.
Did you manage to get your “slow fix” code working? Apparently WDMarvel does “deal with slow responding”, but not in the demo version. Instead one would need to purchase a one-month licence (US$9).
I’ve also been examining the relevant section of MOD 02 (0x1B). It appears to consist of several time constants or timeout values. I suspect that one or more of these determine the amount of time that the drive allows for dealing with retries.
BTW, if you need to access the HDD Oracle forum, it is now at http://www.alexsoft.org. Some psychopath has been attacking the web site, presumably because it champions freedom of information for DIY-ers. It seems that our work has struck a nerve.
Oh you just reminded me that malthus.zapto.org was down for quite a while. Didn’t know it was DoSed.
I haven’t really spend more time on it since you said that WDMarvel does it for free. Instead I spent quite some time trying to port it to Windows/DOS and gave up. I thought it’d be more useful to port it to Windows instead of reinventing the wheel.
Now that you said that it doesn’t do it for free, sure I can resume my coding. I think I can probably leave those unknown sections as it is since your existing MHDD script does not touch them and it does fix the problem.
There is an SG3_utils that compiles on Mingw32 and runs on Windows but the sg3raw does not seem to be able to capture the part of the response from the drive that we need.
Incidentally, I got a 320gb drive today with 57 pending sectors, but I itchily zerofilled part of it, leaving me with 12 pending sectors when I thought of dumping out the mods. Hope it’s good enough for us to determine how are the remaining 3x32bit sections calculated. I can repeat the process and stop once the pending sectors reduce and do more dumps.
Comparing Mod32h of a good identical model.
58h Good : 00
58h Pending 12 : 0D
5Ch Good : 00
5Ch Pending 12 : 0C
60h Good : 00
60h Pending 12 : 0C
68h Good : E9E57B25
68h Pending 12 : 577B7E25
6Ch Good : 15EC7425
6Ch Pending 12 : 32F27325
70h Good : 86
70h Pending 12 : F9
74h Good : 00
74h Pending 12 : 01
7Ch Good : C7DF2A
7Ch Pending 12 : 64FB26
80h Good : DCCB9F25
80h Pending 12 : 96ED9A25
So it appears that, at least for a drive with pending sectors, it has a larger 68h value but smaller 6Ch and 80h values compared to a good drive.
I repeated my zerofill and the current pending sectors count went down from 12 to 3 by the time I stopped it.
Between 00h and 83h, apart from the 32 bit checksum, only 3 bytes changed, 58h, 5Ch, and 60h, and they all changed to 0x03.
Update : Next zerofill brought the pending sectors count to 0.
Between 00h and 83h, apart from the 32 bit checksum, only 3 bytes changed, 58h, 5Ch, and 60h, and they all changed to 0x00.
I’m unsure how much does a Reallocated Sector impact 68h, 6Ch, and 7Ch. I just realized that despite being the same model, both 320gb were made 1 year apart, so I’m not surprised that these values are different. For sure, there are no changes in the 3 locations after the Pending Sectors were cleared with a zerofill on the same drive.
Thanks very much for your continued indulgence. ISTM that it may be better to create two tools, one to modify MOD 02 (the “slow fix”) and a second to clear MOD 32 (the relo-list).
Furthermore, it now seems to me that offset 0x4C should be returned to its original value, namely the same value as at 0x5C. AFAICT, this represents the total number of available spares.
I have seen two results produced by professional tools. In one case there is no before-and-after difference in the values at offsets 0x68 - 0x8F whereas the second tool does have differences in this area. ISTM that it would be safest to leave these locations alone, at least until we can determine what they mean.
As for filling MOD 32 with zeros, I would begin at the offset pointed to by 0x30, not 0x84. That appears to be the beginning of the data area.
Or perhaps generate some non-0 values in 0x84-EOF, write to the hard disk, retrieve and see if my program gets to fix them.
I’ll still post my findings in regard to MOD 32, though. That said, I know that there are times when one needs to rewrite various MODs, so your code could still be useful in the future. All I would need to do would be to adapt the MOD ID. In fact I suspect that there may be numerous common data recovery cases that could be solved with freeware tools and some inside knowledge.