Thursday, January 26, 2012

Careful Painstaking Work Can Recover a Damaged .gz File

http://www.gzip.org/recover.txt - "Here is how you can try to recover your data if it is extremely valuable to you. Do this only if it's worth it, because it will certainly take a lot of time, and it is not even guaranteed to get back all the data correctly.  First, make copies of all the files you still have, to avoid deleting them by mistake. Then work only on the copies. You will also have to patch the gzip sources.

You must hope that all the bad sectors are somewhat grouped together. You can recover the portion before the bad sectors, and you may be able to recover some data after all bad sectors. You can't recover data bewteen those bad sectors, unless they are very far apart from each other.

To recover the portion before the bad sectors, just do:

  gunzip < damaged.gz > part1

gunzip will stop when it sees the bad data. All data in the file "part1" is guaranteed to be correct, but of course the rest will be missing. If the file "damaged" is a .tar file, you can recover some files with:

  gunzip < damaged.tar.gz | tar xvf -

gunzip and tar will complain at some point, but tar may have recovered some files already.

Now let's try to recover something after the bad sectors.  You first have to find the boundary of the first undamaged compression block after the damaged portion.  The boundary is bit aligned. To find the damaged portion, add

            fprintf(stderr, "bytes_in %ld\n", bytes_in);
before:
            error("invalid compressed data--format violated");

in unzip.c. Then round bytes_in this to the next disk block boundary and create a new .gz file by concatenating a valid .gz header and the data believed to be undamaged. Then try repeatedly "gzip -t" on the new .gz file, removing from 1 bit to 8*64K bits from the compressed data portion, until you get a crc error instead of a "format violated" error. At this point do

  gunzip < damaged.gz > damaged

The gzip CRC will always fail because you will miss some 'history', but after some time, the history effect will be reduced and you might be able to recover part of the data. You will have no guarantee that the data will be correct except by manual inspection.

To get a valid .gz header, look at the file algorithm.doc in the gzip distribution, or just copy the header from any valid .gz file. The header ends at the zero terminated file name.  To speed up the search for a block header, the first 3 bits should be 0,0,1 (starting from least significant bit) so that when aligned on a byte boundary you get first_byte & 7 == 4. So you only have to test about 1/8 of all possible bit alignments. Of course if your block was not byte aligned
you have to bit-shift the entire file.

As you can see, all this is not a trivial task, so you should attempt it only if your data is very valuable. gzip 2.0 will have a new blocksize option, allowing to recover easily all undamaged blocks after the damaged portion.

Jean-loup Gailly

jloup@gzip.org
http://www.gzip.org"

Fixgz Fixes Binary Files FTPed as Text Files by CR When it Is Followed by LF

http://www.gzip.org/fixgz.zip - "99.9% of the problems with gzip are due to file transfers done in ASCII mode instead of BINARY mode. In particular, gopher is known to corrupt binary files by considering them as ASCII. Make sure that your local copy of the file has exactly the same byte size as the original. If you have transferred a file in ASCII mode and you no longer have access to the original, you can try the program fixgz to remove the extra CR (carriage return) bytes inserted by the transfer. A Windows 9x/NT/2000/ME/XP binary is here. But there is absolutely no guarantee that this will actually fix your file. Conclusion: never transfer binary files in ASCII mode." - Presumably this works with any file binary transferred as ASCII rather binary. Maybe it works with files mistakenly opened it Word or Notepad and saved...I don't know.

06/16/2014 - Note using fixgz.exe I just tried to recover a DOCX zip format file that looks like it was opened in Notepad and saved, but my effort was not successful.

Removing a Bios - CMOS Password - Free Article

http://www.dewassoc.com/support/bios/bios_password.htm "Unfortunately, access to computers can, at times, be blocked for all of t...