diff --git a/contrib/bzip2/CHANGES b/contrib/bzip2/CHANGES index ecaf4170ef88..d98439543669 100644 --- a/contrib/bzip2/CHANGES +++ b/contrib/bzip2/CHANGES @@ -134,7 +134,7 @@ Several minor bugfixes and enhancements: * Advance the version number to 1.0, so as to counteract the (false-in-this-case) impression some people have that programs - with version numbers less than 1.0 are in someway, experimental, + with version numbers less than 1.0 are in some way, experimental, pre-release versions. * Create an initial Makefile-libbz2_so to build a shared library. @@ -165,3 +165,89 @@ There are no functionality changes or bug fixes relative to version 1.0.0. This is just a documentation update + a fix for minor Win32 build problems. For almost everyone, upgrading from 1.0.0 to 1.0.1 is utterly pointless. Don't bother. + + +1.0.2 +~~~~~ +A bug fix release, addressing various minor issues which have appeared +in the 18 or so months since 1.0.1 was released. Most of the fixes +are to do with file-handling or documentation bugs. To the best of my +knowledge, there have been no data-loss-causing bugs reported in the +compression/decompression engine of 1.0.0 or 1.0.1. + +Note that this release does not improve the rather crude build system +for Unix platforms. The general plan here is to autoconfiscate/ +libtoolise 1.0.2 soon after release, and release the result as 1.1.0 +or perhaps 1.2.0. That, however, is still just a plan at this point. + +Here are the changes in 1.0.2. Bug-reporters and/or patch-senders in +parentheses. + +* Fix an infinite segfault loop in 1.0.1 when a directory is + encountered in -f (force) mode. + (Trond Eivind Glomsrod, Nicholas Nethercote, Volker Schmidt) + +* Avoid double fclose() of output file on certain I/O error paths. + (Solar Designer) + +* Don't fail with internal error 1007 when fed a long stream (> 48MB) + of byte 251. Also print useful message suggesting that 1007s may be + caused by bad memory. + (noticed by Juan Pedro Vallejo, fixed by me) + +* Fix uninitialised variable silly bug in demo prog dlltest.c. + (Jorj Bauer) + +* Remove 512-MB limitation on recovered file size for bzip2recover + on selected platforms which support 64-bit ints. At the moment + all GCC supported platforms, and Win32. + (me, Alson van der Meulen) + +* Hard-code header byte values, to give correct operation on platforms + using EBCDIC as their native character set (IBM's OS/390). + (Leland Lucius) + +* Copy file access times correctly. + (Marty Leisner) + +* Add distclean and check targets to Makefile. + (Michael Carmack) + +* Parameterise use of ar and ranlib in Makefile. Also add $(LDFLAGS). + (Rich Ireland, Bo Thorsen) + +* Pass -p (create parent dirs as needed) to mkdir during make install. + (Jeremy Fusco) + +* Dereference symlinks when copying file permissions in -f mode. + (Volker Schmidt) + +* Majorly simplify implementation of uInt64_qrm10. + (Bo Lindbergh) + +* Check the input file still exists before deleting the output one, + when aborting in cleanUpAndFail(). + (Joerg Prante, Robert Linden, Matthias Krings) + +Also a bunch of patches courtesy of Philippe Troin, the Debian maintainer +of bzip2: + +* Wrapper scripts (with manpages): bzdiff, bzgrep, bzmore. + +* Spelling changes and minor enhancements in bzip2.1. + +* Avoid race condition between creating the output file and setting its + interim permissions safely, by using fopen_output_safely(). + No changes to bzip2recover since there is no issue with file + permissions there. + +* do not print senseless report with -v when compressing an empty + file. + +* bzcat -f works on non-bzip2 files. + +* do not try to escape shell meta-characters on unix (the shell takes + care of these). + +* added --fast and --best aliases for -1 -9 for gzip compatibility. + diff --git a/contrib/bzip2/LICENSE b/contrib/bzip2/LICENSE index 88fa6d88a486..9d4fa4379089 100644 --- a/contrib/bzip2/LICENSE +++ b/contrib/bzip2/LICENSE @@ -1,6 +1,6 @@ This program, "bzip2" and associated library "libbzip2", are -copyright (C) 1996-2000 Julian R Seward. All rights reserved. +copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -35,5 +35,5 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Julian Seward, Cambridge, UK. jseward@acm.org -bzip2/libbzip2 version 1.0 of 21 March 2000 +bzip2/libbzip2 version 1.0.2 of 30 December 2001 diff --git a/contrib/bzip2/Makefile b/contrib/bzip2/Makefile index ab17f4979575..8305235fe24c 100644 --- a/contrib/bzip2/Makefile +++ b/contrib/bzip2/Makefile @@ -1,9 +1,20 @@ SHELL=/bin/sh + +# To assist in cross-compiling CC=gcc +AR=ar +RANLIB=ranlib +LDFLAGS= + +# Suitably paranoid flags to avoid bugs in gcc-2.7 BIGFILES=-D_FILE_OFFSET_BITS=64 CFLAGS=-Wall -Winline -O2 -fomit-frame-pointer -fno-strength-reduce $(BIGFILES) +# Where you want it installed when you do 'make install' +PREFIX=/usr + + OBJS= blocksort.o \ huffman.o \ crctable.o \ @@ -15,20 +26,21 @@ OBJS= blocksort.o \ all: libbz2.a bzip2 bzip2recover test bzip2: libbz2.a bzip2.o - $(CC) $(CFLAGS) -o bzip2 bzip2.o -L. -lbz2 + $(CC) $(CFLAGS) $(LDFLAGS) -o bzip2 bzip2.o -L. -lbz2 bzip2recover: bzip2recover.o - $(CC) $(CFLAGS) -o bzip2recover bzip2recover.o + $(CC) $(CFLAGS) $(LDFLAGS) -o bzip2recover bzip2recover.o libbz2.a: $(OBJS) rm -f libbz2.a - ar cq libbz2.a $(OBJS) - @if ( test -f /usr/bin/ranlib -o -f /bin/ranlib -o \ - -f /usr/ccs/bin/ranlib ) ; then \ - echo ranlib libbz2.a ; \ - ranlib libbz2.a ; \ + $(AR) cq libbz2.a $(OBJS) + @if ( test -f $(RANLIB) -o -f /usr/bin/ranlib -o \ + -f /bin/ranlib -o -f /usr/ccs/bin/ranlib ) ; then \ + echo $(RANLIB) libbz2.a ; \ + $(RANLIB) libbz2.a ; \ fi +check: test test: bzip2 @cat words1 ./bzip2 -1 < sample1.ref > sample1.rb2 @@ -45,14 +57,12 @@ test: bzip2 cmp sample3.tst sample3.ref @cat words3 -PREFIX=/usr - install: bzip2 bzip2recover - if ( test ! -d $(PREFIX)/bin ) ; then mkdir $(PREFIX)/bin ; fi - if ( test ! -d $(PREFIX)/lib ) ; then mkdir $(PREFIX)/lib ; fi - if ( test ! -d $(PREFIX)/man ) ; then mkdir $(PREFIX)/man ; fi - if ( test ! -d $(PREFIX)/man/man1 ) ; then mkdir $(PREFIX)/man/man1 ; fi - if ( test ! -d $(PREFIX)/include ) ; then mkdir $(PREFIX)/include ; fi + if ( test ! -d $(PREFIX)/bin ) ; then mkdir -p $(PREFIX)/bin ; fi + if ( test ! -d $(PREFIX)/lib ) ; then mkdir -p $(PREFIX)/lib ; fi + if ( test ! -d $(PREFIX)/man ) ; then mkdir -p $(PREFIX)/man ; fi + if ( test ! -d $(PREFIX)/man/man1 ) ; then mkdir -p $(PREFIX)/man/man1 ; fi + if ( test ! -d $(PREFIX)/include ) ; then mkdir -p $(PREFIX)/include ; fi cp -f bzip2 $(PREFIX)/bin/bzip2 cp -f bzip2 $(PREFIX)/bin/bunzip2 cp -f bzip2 $(PREFIX)/bin/bzcat @@ -67,7 +77,26 @@ install: bzip2 bzip2recover chmod a+r $(PREFIX)/include/bzlib.h cp -f libbz2.a $(PREFIX)/lib chmod a+r $(PREFIX)/lib/libbz2.a + cp -f bzgrep $(PREFIX)/bin/bzgrep + ln $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzegrep + ln $(PREFIX)/bin/bzgrep $(PREFIX)/bin/bzfgrep + chmod a+x $(PREFIX)/bin/bzgrep + cp -f bzmore $(PREFIX)/bin/bzmore + ln $(PREFIX)/bin/bzmore $(PREFIX)/bin/bzless + chmod a+x $(PREFIX)/bin/bzmore + cp -f bzdiff $(PREFIX)/bin/bzdiff + ln $(PREFIX)/bin/bzdiff $(PREFIX)/bin/bzcmp + chmod a+x $(PREFIX)/bin/bzdiff + cp -f bzgrep.1 bzmore.1 bzdiff.1 $(PREFIX)/man/man1 + chmod a+r $(PREFIX)/man/man1/bzgrep.1 + chmod a+r $(PREFIX)/man/man1/bzmore.1 + chmod a+r $(PREFIX)/man/man1/bzdiff.1 + echo ".so man1/bzgrep.1" > $(PREFIX)/man/man1/bzegrep.1 + echo ".so man1/bzgrep.1" > $(PREFIX)/man/man1/bzfgrep.1 + echo ".so man1/bzmore.1" > $(PREFIX)/man/man1/bzless.1 + echo ".so man1/bzdiff.1" > $(PREFIX)/man/man1/bzcmp.1 +distclean: clean clean: rm -f *.o libbz2.a bzip2 bzip2recover \ sample1.rb2 sample2.rb2 sample3.rb2 \ @@ -93,7 +122,7 @@ bzip2.o: bzip2.c bzip2recover.o: bzip2recover.c $(CC) $(CFLAGS) -c bzip2recover.c -DISTNAME=bzip2-1.0.1 +DISTNAME=bzip2-1.0.2 tarfile: rm -f $(DISTNAME) ln -sf . $(DISTNAME) @@ -112,6 +141,7 @@ tarfile: $(DISTNAME)/Makefile \ $(DISTNAME)/manual.texi \ $(DISTNAME)/manual.ps \ + $(DISTNAME)/manual.pdf \ $(DISTNAME)/LICENSE \ $(DISTNAME)/bzip2.1 \ $(DISTNAME)/bzip2.1.preformatted \ @@ -138,4 +168,25 @@ tarfile: $(DISTNAME)/Y2K_INFO \ $(DISTNAME)/unzcrash.c \ $(DISTNAME)/spewG.c \ + $(DISTNAME)/mk251.c \ + $(DISTNAME)/bzdiff \ + $(DISTNAME)/bzdiff.1 \ + $(DISTNAME)/bzmore \ + $(DISTNAME)/bzmore.1 \ + $(DISTNAME)/bzgrep \ + $(DISTNAME)/bzgrep.1 \ $(DISTNAME)/Makefile-libbz2_so + gzip -v $(DISTNAME).tar + +# For rebuilding the manual from sources on my RedHat 7.2 box +manual: manual.ps manual.pdf manual.html + +manual.ps: manual.texi + tex manual.texi + dvips -o manual.ps manual.dvi + +manual.pdf: manual.ps + ps2pdf manual.ps + +manual.html: manual.texi + texi2html -split_chapter manual.texi diff --git a/contrib/bzip2/Makefile-libbz2_so b/contrib/bzip2/Makefile-libbz2_so index a347c50e9b28..4986fe2ad8a7 100644 --- a/contrib/bzip2/Makefile-libbz2_so +++ b/contrib/bzip2/Makefile-libbz2_so @@ -1,8 +1,9 @@ # This Makefile builds a shared version of the library, -# libbz2.so.1.0.1, with soname libbz2.so.1.0, -# at least on x86-Linux (RedHat 5.2), -# with gcc-2.7.2.3. Please see the README file for some +# libbz2.so.1.0.2, with soname libbz2.so.1.0, +# at least on x86-Linux (RedHat 7.2), +# with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98). +# Please see the README file for some # important info about building the library like this. SHELL=/bin/sh @@ -19,13 +20,13 @@ OBJS= blocksort.o \ bzlib.o all: $(OBJS) - $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.1 $(OBJS) - $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.1 + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS) + $(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.2 rm -f libbz2.so.1.0 - ln -s libbz2.so.1.0.1 libbz2.so.1.0 + ln -s libbz2.so.1.0.2 libbz2.so.1.0 clean: - rm -f $(OBJS) bzip2.o libbz2.so.1.0.1 libbz2.so.1.0 bzip2-shared + rm -f $(OBJS) bzip2.o libbz2.so.1.0.2 libbz2.so.1.0 bzip2-shared blocksort.o: blocksort.c $(CC) $(CFLAGS) -c blocksort.c diff --git a/contrib/bzip2/README b/contrib/bzip2/README index 22945a256c78..07505d8f3d68 100644 --- a/contrib/bzip2/README +++ b/contrib/bzip2/README @@ -1,15 +1,15 @@ This is the README for bzip2, a block-sorting file compressor, version -1.0. This version is fully compatible with the previous public -releases, bzip2-0.1pl2, bzip2-0.9.0 and bzip2-0.9.5. +1.0.2. This version is fully compatible with the previous public +releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1. -bzip2-1.0 is distributed under a BSD-style license. For details, +bzip2-1.0.2 is distributed under a BSD-style license. For details, see the file LICENSE. -Complete documentation is available in Postscript form (manual.ps) or -html (manual_toc.html). A plain-text version of the manual page is -available as bzip2.txt. A statement about Y2K issues is now included -in the file Y2K_INFO. +Complete documentation is available in Postscript form (manual.ps), +PDF (manual.pdf, amazingly enough) or html (manual_toc.html). A +plain-text version of the manual page is available as bzip2.txt. +A statement about Y2K issues is now included in the file Y2K_INFO. HOW TO BUILD -- UNIX @@ -33,34 +33,41 @@ not actually execute them. HOW TO BUILD -- UNIX, shared library libbz2.so. Do 'make -f Makefile-libbz2_so'. This Makefile seems to work for -Linux-ELF (RedHat 5.2 on an x86 box), with gcc. I make no claims +Linux-ELF (RedHat 7.2 on an x86 box), with gcc. I make no claims that it works for any other platform, though I suspect it probably will work for most platforms employing both ELF and gcc. -bzip2-shared, a client of the shared library, is also build, but -not self-tested. So I suggest you also build using the normal -Makefile, since that conducts a self-test. +bzip2-shared, a client of the shared library, is also built, but not +self-tested. So I suggest you also build using the normal Makefile, +since that conducts a self-test. A second reason to prefer the +version statically linked to the library is that, on x86 platforms, +building shared objects makes a valuable register (%ebx) unavailable +to gcc, resulting in a slowdown of 10%-20%, at least for bzip2. -Important note for people upgrading .so's from 0.9.0/0.9.5 to -version 1.0. All the functions in the library have been renamed, -from (eg) bzCompress to BZ2_bzCompress, to avoid namespace pollution. +Important note for people upgrading .so's from 0.9.0/0.9.5 to version +1.0.X. All the functions in the library have been renamed, from (eg) +bzCompress to BZ2_bzCompress, to avoid namespace pollution. Unfortunately this means that the libbz2.so created by -Makefile-libbz2_so will not work with any program which used an -older version of the library. Sorry. I do encourage library -clients to make the effort to upgrade to use version 1.0, since -it is both faster and more robust than previous versions. +Makefile-libbz2_so will not work with any program which used an older +version of the library. Sorry. I do encourage library clients to +make the effort to upgrade to use version 1.0, since it is both faster +and more robust than previous versions. HOW TO BUILD -- Windows 95, NT, DOS, Mac, etc. It's difficult for me to support compilation on all these platforms. My approach is to collect binaries for these platforms, and put them -on the master web page (http://sourceware.cygnus.com/bzip2). Look -there. However (FWIW), bzip2-1.0 is very standard ANSI C and should -compile unmodified with MS Visual C. For Win32, there is one -important caveat: in bzip2.c, you must set BZ_UNIX to 0 and -BZ_LCCWIN32 to 1 before building. If you have difficulties building, -you might want to read README.COMPILATION.PROBLEMS. +on the master web page (http://sources.redhat.com/bzip2). Look there. +However (FWIW), bzip2-1.0.X is very standard ANSI C and should compile +unmodified with MS Visual C. If you have difficulties building, you +might want to read README.COMPILATION.PROBLEMS. + +At least using MS Visual C++ 6, you can build from the unmodified +sources by issuing, in a command shell: + nmake -f makefile.msc +(you may need to first run the MSVC-provided script VCVARS32.BAT + so as to set up paths to the MSVC tools correctly). VALIDATION @@ -138,29 +145,37 @@ WHAT'S NEW IN 0.9.5 ? * Many small improvements in file and flag handling. * A Y2K statement. -WHAT'S NEW IN 1.0 +WHAT'S NEW IN 1.0.0 ? See the CHANGES file. +WHAT'S NEW IN 1.0.2 ? + + See the CHANGES file. + + I hope you find bzip2 useful. Feel free to contact me at jseward@acm.org if you have any suggestions or queries. Many people mailed me with comments, suggestions and patches after the releases of bzip-0.15, -bzip-0.21, bzip2-0.1pl2 and bzip2-0.9.0, and the changes in bzip2 are -largely a result of this feedback. I thank you for your comments. +bzip-0.21, and bzip2 versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, +and the changes in bzip2 are largely a result of this feedback. +I thank you for your comments. At least for the time being, bzip2's "home" is (or can be reached via) -http://www.muraroa.demon.co.uk. +http://sources.redhat.com/bzip2. Julian Seward jseward@acm.org -Cambridge, UK -18 July 1996 (version 0.15) -25 August 1996 (version 0.21) - 7 August 1997 (bzip2, version 0.1) -29 August 1997 (bzip2, version 0.1pl2) -23 August 1998 (bzip2, version 0.9.0) - 8 June 1999 (bzip2, version 0.9.5) - 4 Sept 1999 (bzip2, version 0.9.5d) - 5 May 2000 (bzip2, version 1.0pre8) +Cambridge, UK (and what a great town this is!) + +18 July 1996 (version 0.15) +25 August 1996 (version 0.21) + 7 August 1997 (bzip2, version 0.1) +29 August 1997 (bzip2, version 0.1pl2) +23 August 1998 (bzip2, version 0.9.0) + 8 June 1999 (bzip2, version 0.9.5) + 4 Sept 1999 (bzip2, version 0.9.5d) + 5 May 2000 (bzip2, version 1.0pre8) +30 December 2001 (bzip2, version 1.0.2pre1) \ No newline at end of file diff --git a/contrib/bzip2/README.COMPILATION.PROBLEMS b/contrib/bzip2/README.COMPILATION.PROBLEMS index d621ad59756c..bd1822dffbd0 100644 --- a/contrib/bzip2/README.COMPILATION.PROBLEMS +++ b/contrib/bzip2/README.COMPILATION.PROBLEMS @@ -117,11 +117,11 @@ Known problems as of 1.0pre8: All that said: you might be able to get somewhere by finding the line in Makefile-libbz2_so which says - $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.1 $(OBJS) + $(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.2 $(OBJS) and replacing with - ($CC) -G -shared -o libbz2.so.1.0.1 -h libbz2.so.1.0 $(OBJS) + $(CC) -G -shared -o libbz2.so.1.0.2 -h libbz2.so.1.0 $(OBJS) If gcc objects to the combination -fpic -fPIC, get rid of the second one, leaving just "-fpic". diff --git a/contrib/bzip2/blocksort.c b/contrib/bzip2/blocksort.c index ec426725b1e2..aba3efcd3121 100644 --- a/contrib/bzip2/blocksort.c +++ b/contrib/bzip2/blocksort.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -981,7 +981,14 @@ void mainSort ( UInt32* ptr, } } - AssertH ( copyStart[ss]-1 == copyEnd[ss], 1007 ); + AssertH ( (copyStart[ss]-1 == copyEnd[ss]) + || + /* Extremely rare case missing in bzip2-1.0.0 and 1.0.1. + Necessity for this case is demonstrated by compressing + a sequence of approximately 48.5 million of character + 251; 1.0.0/1.0.1 will then die here. */ + (copyStart[ss] == 0 && copyEnd[ss] == nblock-1), + 1007 ) for (j = 0; j <= 255; j++) ftab[(j << 8) + ss] |= SETMASK; diff --git a/contrib/bzip2/bzip2.1 b/contrib/bzip2/bzip2.1 index 7de54a0118c3..623435c24277 100644 --- a/contrib/bzip2/bzip2.1 +++ b/contrib/bzip2/bzip2.1 @@ -1,7 +1,7 @@ .PU .TH bzip2 1 .SH NAME -bzip2, bunzip2 \- a block-sorting file compressor, v1.0 +bzip2, bunzip2 \- a block-sorting file compressor, v1.0.2 .br bzcat \- decompresses files to stdout .br @@ -197,7 +197,7 @@ to decompress. .TP .B \-z --compress The complement to \-d: forces compression, regardless of the -invokation name. +invocation name. .TP .B \-t --test Check integrity of the specified file(s), but don't decompress them. @@ -211,6 +211,10 @@ existing output files. Also forces .I bzip2 to break hard links to files, which it otherwise wouldn't do. + +bzip2 normally declines to decompress files which don't have the +correct magic header bytes. If forced (-f), however, it will pass +such files through unmodified. This is how GNU gzip behaves. .TP .B \-k --keep Keep (don't delete) input files during compression @@ -239,9 +243,13 @@ information which is primarily of interest for diagnostic purposes. .B \-L --license -V --version Display the software version, license terms and conditions. .TP -.B \-1 to \-9 +.B \-1 (or \-\-fast) to \-9 (or \-\-best) Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. See MEMORY MANAGEMENT below. +The \-\-fast and \-\-best aliases are primarily for GNU gzip +compatibility. In particular, \-\-fast doesn't make things +significantly faster. +And \-\-best merely selects the default behaviour. .TP .B \-- Treats all subsequent arguments as file names, even if they start @@ -352,11 +360,11 @@ undamaged. .I bzip2recover takes a single argument, the name of the damaged file, -and writes a number of files "rec0001file.bz2", -"rec0002file.bz2", etc, containing the extracted blocks. +and writes a number of files "rec00001file.bz2", +"rec00002file.bz2", etc, containing the extracted blocks. The output filenames are designed so that the use of wildcards in subsequent processing -- for example, -"bzip2 -dc rec*file.bz2 > recovered_data" -- lists the files in +"bzip2 -dc rec*file.bz2 > recovered_data" -- processes the files in the correct order. .I bzip2recover @@ -397,27 +405,31 @@ I/O error messages are not as helpful as they could be. tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 1.0 of +This manual page pertains to version 1.0.2 of .I bzip2. -Compressed -data created by this version is entirely forwards and backwards -compatible with the previous public releases, versions 0.1pl2, 0.9.0 -and 0.9.5, -but with the following exception: 0.9.0 and above can correctly -decompress multiple concatenated compressed files. 0.1pl2 cannot do -this; it will stop after decompressing just the first file in the -stream. +Compressed data created by this version is entirely forwards and +backwards compatible with the previous public releases, versions +0.1pl2, 0.9.0, 0.9.5, 1.0.0 and 1.0.1, but with the following +exception: 0.9.0 and above can correctly decompress multiple +concatenated compressed files. 0.1pl2 cannot do this; it will stop +after decompressing just the first file in the stream. .I bzip2recover -uses 32-bit integers to represent bit positions in -compressed files, so it cannot handle compressed files more than 512 -megabytes long. This could easily be fixed. +versions prior to this one, 1.0.2, used 32-bit integers to represent +bit positions in compressed files, so it could not handle compressed +files more than 512 megabytes long. Version 1.0.2 and above uses +64-bit ints on some platforms which support them (GNU supported +targets, and Windows). To establish whether or not bzip2recover was +built with such a limitation, run it without arguments. In any event +you can build yourself an unlimited version if you can recompile it +with MaybeUInt64 set to be an unsigned 64-bit integer. + + .SH AUTHOR Julian Seward, jseward@acm.org. -http://sourceware.cygnus.com/bzip2 -http://www.muraroa.demon.co.uk +http://sources.redhat.com/bzip2 The ideas embodied in .I bzip2 @@ -434,6 +446,8 @@ indebted for their help, support and advice. See the manual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look for faster sorting algorithms, so as to speed up compression. Bela Lubkin encouraged me to improve the -worst-case compression performance. Many people sent patches, helped +worst-case compression performance. +The bz* scripts are derived from those of GNU gzip. +Many people sent patches, helped with portability problems, lent machines, gave advice and were generally helpful. diff --git a/contrib/bzip2/bzip2.c b/contrib/bzip2/bzip2.c index 56adfdcbc46b..807f420aed6b 100644 --- a/contrib/bzip2/bzip2.c +++ b/contrib/bzip2/bzip2.c @@ -7,7 +7,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -113,13 +113,16 @@ /*-- Generic 32-bit Unix. Also works on 64-bit Unix boxes. + This is the default. --*/ #define BZ_UNIX 1 /*-- Win32, as seen by Jacob Navia's excellent port of (Chris Fraser & David Hanson)'s excellent - lcc compiler. + lcc compiler. Or with MS Visual C. + This is selected automatically if compiled by a compiler which + defines _WIN32, not including the Cygwin GCC. --*/ #define BZ_LCCWIN32 0 @@ -156,6 +159,7 @@ --*/ #if BZ_UNIX +# include # include # include # include @@ -164,8 +168,9 @@ # define PATH_SEP '/' # define MY_LSTAT lstat -# define MY_S_IFREG S_ISREG # define MY_STAT stat +# define MY_S_ISREG S_ISREG +# define MY_S_ISDIR S_ISDIR # define APPEND_FILESPEC(root, name) \ root=snocString((root), (name)) @@ -180,19 +185,23 @@ # else # define NORETURN /**/ # endif + # ifdef __DJGPP__ # include # include # undef MY_LSTAT +# undef MY_STAT # define MY_LSTAT stat +# define MY_STAT stat # undef SET_BINARY_MODE # define SET_BINARY_MODE(fd) \ do { \ int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ + O_BINARY ); \ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) # endif + # ifdef __CYGWIN__ # include # include @@ -200,11 +209,11 @@ # define SET_BINARY_MODE(fd) \ do { \ int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ + O_BINARY ); \ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) # endif -#endif +#endif /* BZ_UNIX */ @@ -217,46 +226,23 @@ # define PATH_SEP '\\' # define MY_LSTAT _stat # define MY_STAT _stat -# define MY_S_IFREG(x) ((x) & _S_IFREG) +# define MY_S_ISREG(x) ((x) & _S_IFREG) +# define MY_S_ISDIR(x) ((x) & _S_IFDIR) # define APPEND_FLAG(root, name) \ root=snocString((root), (name)) -# if 0 - /*-- lcc-win32 seems to expand wildcards itself --*/ -# define APPEND_FILESPEC(root, spec) \ - do { \ - if ((spec)[0] == '-') { \ - root = snocString((root), (spec)); \ - } else { \ - struct _finddata_t c_file; \ - long hFile; \ - hFile = _findfirst((spec), &c_file); \ - if ( hFile == -1L ) { \ - root = snocString ((root), (spec)); \ - } else { \ - int anInt = 0; \ - while ( anInt == 0 ) { \ - root = snocString((root), \ - &c_file.name[0]); \ - anInt = _findnext(hFile, &c_file); \ - } \ - } \ - } \ - } while ( 0 ) -# else # define APPEND_FILESPEC(root, name) \ root = snocString ((root), (name)) -# endif # define SET_BINARY_MODE(fd) \ do { \ int retVal = setmode ( fileno ( fd ), \ - O_BINARY ); \ + O_BINARY ); \ ERROR_IF_MINUS_ONE ( retVal ); \ } while ( 0 ) -#endif +#endif /* BZ_LCCWIN32 */ /*---------------------------------------------*/ @@ -338,6 +324,7 @@ typedef struct { UChar b[8]; } UInt64; + static void uInt64_from_UInt32s ( UInt64* n, UInt32 lo32, UInt32 hi32 ) { @@ -351,6 +338,7 @@ void uInt64_from_UInt32s ( UInt64* n, UInt32 lo32, UInt32 hi32 ) n->b[0] = (UChar) (lo32 & 0xFF); } + static double uInt64_to_double ( UInt64* n ) { @@ -364,77 +352,6 @@ double uInt64_to_double ( UInt64* n ) return sum; } -static -void uInt64_add ( UInt64* src, UInt64* dst ) -{ - Int32 i; - Int32 carry = 0; - for (i = 0; i < 8; i++) { - carry += ( ((Int32)src->b[i]) + ((Int32)dst->b[i]) ); - dst->b[i] = (UChar)(carry & 0xFF); - carry >>= 8; - } -} - -static -void uInt64_sub ( UInt64* src, UInt64* dst ) -{ - Int32 t, i; - Int32 borrow = 0; - for (i = 0; i < 8; i++) { - t = ((Int32)dst->b[i]) - ((Int32)src->b[i]) - borrow; - if (t < 0) { - dst->b[i] = (UChar)(t + 256); - borrow = 1; - } else { - dst->b[i] = (UChar)t; - borrow = 0; - } - } -} - -static -void uInt64_mul ( UInt64* a, UInt64* b, UInt64* r_hi, UInt64* r_lo ) -{ - UChar sum[16]; - Int32 ia, ib, carry; - for (ia = 0; ia < 16; ia++) sum[ia] = 0; - for (ia = 0; ia < 8; ia++) { - carry = 0; - for (ib = 0; ib < 8; ib++) { - carry += ( ((Int32)sum[ia+ib]) - + ((Int32)a->b[ia]) * ((Int32)b->b[ib]) ); - sum[ia+ib] = (UChar)(carry & 0xFF); - carry >>= 8; - } - sum[ia+8] = (UChar)(carry & 0xFF); - if ((carry >>= 8) != 0) panic ( "uInt64_mul" ); - } - - for (ia = 0; ia < 8; ia++) r_hi->b[ia] = sum[ia+8]; - for (ia = 0; ia < 8; ia++) r_lo->b[ia] = sum[ia]; -} - - -static -void uInt64_shr1 ( UInt64* n ) -{ - Int32 i; - for (i = 0; i < 8; i++) { - n->b[i] >>= 1; - if (i < 7 && (n->b[i+1] & 1)) n->b[i] |= 0x80; - } -} - -static -void uInt64_shl1 ( UInt64* n ) -{ - Int32 i; - for (i = 7; i >= 0; i--) { - n->b[i] <<= 1; - if (i > 0 && (n->b[i-1] & 0x80)) n->b[i]++; - } -} static Bool uInt64_isZero ( UInt64* n ) @@ -445,49 +362,23 @@ Bool uInt64_isZero ( UInt64* n ) return 1; } -static + +/* Divide *n by 10, and return the remainder. */ +static Int32 uInt64_qrm10 ( UInt64* n ) { - /* Divide *n by 10, and return the remainder. Long division - is difficult, so we cheat and instead multiply by - 0xCCCC CCCC CCCC CCCD, which is 0.8 (viz, 0.1 << 3). - */ + UInt32 rem, tmp; Int32 i; - UInt64 tmp1, tmp2, n_orig, zero_point_eight; - - zero_point_eight.b[1] = zero_point_eight.b[2] = - zero_point_eight.b[3] = zero_point_eight.b[4] = - zero_point_eight.b[5] = zero_point_eight.b[6] = - zero_point_eight.b[7] = 0xCC; - zero_point_eight.b[0] = 0xCD; - - n_orig = *n; - - /* divide n by 10, - by multiplying by 0.8 and then shifting right 3 times */ - uInt64_mul ( n, &zero_point_eight, &tmp1, &tmp2 ); - uInt64_shr1(&tmp1); uInt64_shr1(&tmp1); uInt64_shr1(&tmp1); - *n = tmp1; - - /* tmp1 = 8*n, tmp2 = 2*n */ - uInt64_shl1(&tmp1); uInt64_shl1(&tmp1); uInt64_shl1(&tmp1); - tmp2 = *n; uInt64_shl1(&tmp2); - - /* tmp1 = 10*n */ - uInt64_add ( &tmp2, &tmp1 ); - - /* n_orig = n_orig - 10*n */ - uInt64_sub ( &tmp1, &n_orig ); - - /* n_orig should now hold quotient, in range 0 .. 9 */ - for (i = 7; i >= 1; i--) - if (n_orig.b[i] != 0) panic ( "uInt64_qrm10(1)" ); - if (n_orig.b[0] > 9) - panic ( "uInt64_qrm10(2)" ); - - return (int)n_orig.b[0]; + rem = 0; + for (i = 7; i >= 0; i--) { + tmp = rem * 256 + n->b[i]; + n->b[i] = tmp / 10; + rem = tmp % 10; + } + return rem; } + /* ... and the Whole Entire Point of all this UInt64 stuff is so that we can supply the following function. */ @@ -504,7 +395,8 @@ void uInt64_toAscii ( char* outbuf, UInt64* n ) nBuf++; } while (!uInt64_isZero(&n_copy)); outbuf[nBuf] = 0; - for (i = 0; i < nBuf; i++) outbuf[i] = buf[nBuf-i-1]; + for (i = 0; i < nBuf; i++) + outbuf[i] = buf[nBuf-i-1]; } @@ -566,35 +458,38 @@ void compressStream ( FILE *stream, FILE *zStream ) if (ret == EOF) goto errhandler_io; if (zStream != stdout) { ret = fclose ( zStream ); + outputHandleJustInCase = NULL; if (ret == EOF) goto errhandler_io; } + outputHandleJustInCase = NULL; if (ferror(stream)) goto errhandler_io; ret = fclose ( stream ); if (ret == EOF) goto errhandler_io; - if (nbytes_in_lo32 == 0 && nbytes_in_hi32 == 0) - nbytes_in_lo32 = 1; - if (verbosity >= 1) { - Char buf_nin[32], buf_nout[32]; - UInt64 nbytes_in, nbytes_out; - double nbytes_in_d, nbytes_out_d; - uInt64_from_UInt32s ( &nbytes_in, - nbytes_in_lo32, nbytes_in_hi32 ); - uInt64_from_UInt32s ( &nbytes_out, - nbytes_out_lo32, nbytes_out_hi32 ); - nbytes_in_d = uInt64_to_double ( &nbytes_in ); - nbytes_out_d = uInt64_to_double ( &nbytes_out ); - uInt64_toAscii ( buf_nin, &nbytes_in ); - uInt64_toAscii ( buf_nout, &nbytes_out ); - fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " - "%5.2f%% saved, %s in, %s out.\n", - nbytes_in_d / nbytes_out_d, - (8.0 * nbytes_out_d) / nbytes_in_d, - 100.0 * (1.0 - nbytes_out_d / nbytes_in_d), - buf_nin, - buf_nout - ); + if (nbytes_in_lo32 == 0 && nbytes_in_hi32 == 0) { + fprintf ( stderr, " no data compressed.\n"); + } else { + Char buf_nin[32], buf_nout[32]; + UInt64 nbytes_in, nbytes_out; + double nbytes_in_d, nbytes_out_d; + uInt64_from_UInt32s ( &nbytes_in, + nbytes_in_lo32, nbytes_in_hi32 ); + uInt64_from_UInt32s ( &nbytes_out, + nbytes_out_lo32, nbytes_out_hi32 ); + nbytes_in_d = uInt64_to_double ( &nbytes_in ); + nbytes_out_d = uInt64_to_double ( &nbytes_out ); + uInt64_toAscii ( buf_nin, &nbytes_in ); + uInt64_toAscii ( buf_nout, &nbytes_out ); + fprintf ( stderr, "%6.3f:1, %6.3f bits/byte, " + "%5.2f%% saved, %s in, %s out.\n", + nbytes_in_d / nbytes_out_d, + (8.0 * nbytes_out_d) / nbytes_in_d, + 100.0 * (1.0 - nbytes_out_d / nbytes_in_d), + buf_nin, + buf_nout + ); + } } return; @@ -652,7 +547,7 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) while (bzerr == BZ_OK) { nread = BZ2_bzRead ( &bzerr, bzf, obuf, 5000 ); - if (bzerr == BZ_DATA_ERROR_MAGIC) goto errhandler; + if (bzerr == BZ_DATA_ERROR_MAGIC) goto trycat; if ((bzerr == BZ_OK || bzerr == BZ_STREAM_END) && nread > 0) fwrite ( obuf, sizeof(UChar), nread, stream ); if (ferror(stream)) goto errhandler_io; @@ -668,9 +563,9 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) if (bzerr != BZ_OK) panic ( "decompress:bzReadGetUnused" ); if (nUnused == 0 && myfeof(zStream)) break; - } + closeok: if (ferror(zStream)) goto errhandler_io; ret = fclose ( zStream ); if (ret == EOF) goto errhandler_io; @@ -680,11 +575,26 @@ Bool uncompressStream ( FILE *zStream, FILE *stream ) if (ret != 0) goto errhandler_io; if (stream != stdout) { ret = fclose ( stream ); + outputHandleJustInCase = NULL; if (ret == EOF) goto errhandler_io; } + outputHandleJustInCase = NULL; if (verbosity >= 2) fprintf ( stderr, "\n " ); return True; + trycat: + if (forceOverwrite) { + rewind(zStream); + while (True) { + if (myfeof(zStream)) break; + nread = fread ( obuf, sizeof(UChar), 5000, zStream ); + if (ferror(zStream)) goto errhandler_io; + if (nread > 0) fwrite ( obuf, sizeof(UChar), nread, stream ); + if (ferror(stream)) goto errhandler_io; + } + goto closeok; + } + errhandler: BZ2_bzReadClose ( &bzerr_dummy, bzf ); switch (bzerr) { @@ -832,7 +742,7 @@ void cadvise ( void ) stderr, "\nIt is possible that the compressed file(s) have become corrupted.\n" "You can use the -tvv option to test integrity of such files.\n\n" - "You can use the `bzip2recover' program to *attempt* to recover\n" + "You can use the `bzip2recover' program to attempt to recover\n" "data from undamaged sections of corrupted files.\n\n" ); } @@ -855,28 +765,55 @@ void showFileNames ( void ) static void cleanUpAndFail ( Int32 ec ) { - IntNative retVal; + IntNative retVal; + struct MY_STAT statBuf; if ( srcMode == SM_F2F && opMode != OM_TEST && deleteOutputOnInterrupt ) { - if (noisy) - fprintf ( stderr, "%s: Deleting output file %s, if it exists.\n", - progName, outName ); - if (outputHandleJustInCase != NULL) - fclose ( outputHandleJustInCase ); - retVal = remove ( outName ); - if (retVal != 0) + + /* Check whether input file still exists. Delete output file + only if input exists to avoid loss of data. Joerg Prante, 5 + January 2002. (JRS 06-Jan-2002: other changes in 1.0.2 mean + this is less likely to happen. But to be ultra-paranoid, we + do the check anyway.) */ + retVal = MY_STAT ( inName, &statBuf ); + if (retVal == 0) { + if (noisy) + fprintf ( stderr, + "%s: Deleting output file %s, if it exists.\n", + progName, outName ); + if (outputHandleJustInCase != NULL) + fclose ( outputHandleJustInCase ); + retVal = remove ( outName ); + if (retVal != 0) + fprintf ( stderr, + "%s: WARNING: deletion of output file " + "(apparently) failed.\n", + progName ); + } else { fprintf ( stderr, - "%s: WARNING: deletion of output file (apparently) failed.\n", + "%s: WARNING: deletion of output file suppressed\n", + progName ); + fprintf ( stderr, + "%s: since input file no longer exists. Output file\n", progName ); + fprintf ( stderr, + "%s: `%s' may be incomplete.\n", + progName, outName ); + fprintf ( stderr, + "%s: I suggest doing an integrity test (bzip2 -tv)" + " of it.\n", + progName ); + } } + if (noisy && numFileNames > 0 && numFilesProcessed < numFileNames) { fprintf ( stderr, "%s: WARNING: some files have not been processed:\n" - "\t%d specified on command line, %d not processed yet.\n\n", - progName, numFileNames, - numFileNames - numFilesProcessed ); + "%s: %d specified on command line, %d not processed yet.\n\n", + progName, progName, + numFileNames, numFileNames - numFilesProcessed ); } setExit(ec); exit(exitValue); @@ -915,14 +852,16 @@ void crcError ( void ) static void compressedStreamEOF ( void ) { - fprintf ( stderr, - "\n%s: Compressed file ends unexpectedly;\n\t" - "perhaps it is corrupted? *Possible* reason follows.\n", - progName ); - perror ( progName ); - showFileNames(); - cadvise(); - cleanUpAndFail( 2 ); + if (noisy) { + fprintf ( stderr, + "\n%s: Compressed file ends unexpectedly;\n\t" + "perhaps it is corrupted? *Possible* reason follows.\n", + progName ); + perror ( progName ); + showFileNames(); + cadvise(); + } + cleanUpAndFail( 2 ); } @@ -1038,6 +977,11 @@ void configError ( void ) /*--- The main driver machinery ---*/ /*---------------------------------------------------*/ +/* All rather crufty. The main problem is that input files + are stat()d multiple times before use. This should be + cleaned up. +*/ + /*---------------------------------------------*/ static void pad ( Char *s ) @@ -1081,6 +1025,32 @@ Bool fileExists ( Char* name ) } +/*---------------------------------------------*/ +/* Open an output file safely with O_EXCL and good permissions. + This avoids a race condition in versions < 1.0.2, in which + the file was first opened and then had its interim permissions + set safely. We instead use open() to create the file with + the interim permissions required. (--- --- rw-). + + For non-Unix platforms, if we are not worrying about + security issues, simple this simply behaves like fopen. +*/ +FILE* fopen_output_safely ( Char* name, const char* mode ) +{ +# if BZ_UNIX + FILE* fp; + IntNative fh; + fh = open(name, O_WRONLY|O_CREAT|O_EXCL, S_IWUSR|S_IRUSR); + if (fh == -1) return NULL; + fp = fdopen(fh, mode); + if (fp == NULL) close(fh); + return fp; +# else + return fopen(name, mode); +# endif +} + + /*---------------------------------------------*/ /*-- if in doubt, return True @@ -1093,7 +1063,7 @@ Bool notAStandardFile ( Char* name ) i = MY_LSTAT ( name, &statBuf ); if (i != 0) return True; - if (MY_S_IFREG(statBuf.st_mode)) return False; + if (MY_S_ISREG(statBuf.st_mode)) return False; return True; } @@ -1115,42 +1085,66 @@ Int32 countHardLinks ( Char* name ) /*---------------------------------------------*/ -static -void copyDatePermissionsAndOwner ( Char *srcName, Char *dstName ) -{ +/* Copy modification date, access date, permissions and owner from the + source to destination file. We have to copy this meta-info off + into fileMetaInfo before starting to compress / decompress it, + because doing it afterwards means we get the wrong access time. + + To complicate matters, in compress() and decompress() below, the + sequence of tests preceding the call to saveInputFileMetaInfo() + involves calling fileExists(), which in turn establishes its result + by attempting to fopen() the file, and if successful, immediately + fclose()ing it again. So we have to assume that the fopen() call + does not cause the access time field to be updated. + + Reading of the man page for stat() (man 2 stat) on RedHat 7.2 seems + to imply that merely doing open() will not affect the access time. + Therefore we merely need to hope that the C library only does + open() as a result of fopen(), and not any kind of read()-ahead + cleverness. + + It sounds pretty fragile to me. Whether this carries across + robustly to arbitrary Unix-like platforms (or even works robustly + on this one, RedHat 7.2) is unknown to me. Nevertheless ... +*/ #if BZ_UNIX +static +struct MY_STAT fileMetaInfo; +#endif + +static +void saveInputFileMetaInfo ( Char *srcName ) +{ +# if BZ_UNIX + IntNative retVal; + /* Note use of stat here, not lstat. */ + retVal = MY_STAT( srcName, &fileMetaInfo ); + ERROR_IF_NOT_ZERO ( retVal ); +# endif +} + + +static +void applySavedMetaInfoToOutputFile ( Char *dstName ) +{ +# if BZ_UNIX IntNative retVal; - struct MY_STAT statBuf; struct utimbuf uTimBuf; - retVal = MY_LSTAT ( srcName, &statBuf ); - ERROR_IF_NOT_ZERO ( retVal ); - uTimBuf.actime = statBuf.st_atime; - uTimBuf.modtime = statBuf.st_mtime; + uTimBuf.actime = fileMetaInfo.st_atime; + uTimBuf.modtime = fileMetaInfo.st_mtime; - retVal = chmod ( dstName, statBuf.st_mode ); + retVal = chmod ( dstName, fileMetaInfo.st_mode ); ERROR_IF_NOT_ZERO ( retVal ); retVal = utime ( dstName, &uTimBuf ); ERROR_IF_NOT_ZERO ( retVal ); - retVal = chown ( dstName, statBuf.st_uid, statBuf.st_gid ); + retVal = chown ( dstName, fileMetaInfo.st_uid, fileMetaInfo.st_gid ); /* chown() will in many cases return with EPERM, which can be safely ignored. */ -#endif -} - - -/*---------------------------------------------*/ -static -void setInterimPermissions ( Char *dstName ) -{ -#if BZ_UNIX - IntNative retVal; - retVal = chmod ( dstName, S_IRUSR | S_IWUSR ); - ERROR_IF_NOT_ZERO ( retVal ); -#endif +# endif } @@ -1158,10 +1152,19 @@ void setInterimPermissions ( Char *dstName ) static Bool containsDubiousChars ( Char* name ) { - Bool cdc = False; +# if BZ_UNIX + /* On unix, files can contain any characters and the file expansion + * is performed by the shell. + */ + return False; +# else /* ! BZ_UNIX */ + /* On non-unix (Win* platforms), wildcard characters are not allowed in + * filenames. + */ for (; *name != '\0'; name++) - if (*name == '?' || *name == '*') cdc = True; - return cdc; + if (*name == '?' || *name == '*') return True; + return False; +# endif /* BZ_UNIX */ } @@ -1201,6 +1204,7 @@ void compress ( Char *name ) FILE *inStr; FILE *outStr; Int32 n, i; + struct MY_STAT statBuf; deleteOutputOnInterrupt = False; @@ -1246,6 +1250,16 @@ void compress ( Char *name ) return; } } + if ( srcMode == SM_F2F || srcMode == SM_F2O ) { + MY_STAT(inName, &statBuf); + if ( MY_S_ISDIR(statBuf.st_mode) ) { + fprintf( stderr, + "%s: Input file %s is a directory.\n", + progName,inName); + setExit(1); + return; + } + } if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { if (noisy) fprintf ( stderr, "%s: Input file %s is not a normal file.\n", @@ -1253,11 +1267,15 @@ void compress ( Char *name ) setExit(1); return; } - if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { - fprintf ( stderr, "%s: Output file %s already exists.\n", - progName, outName ); - setExit(1); - return; + if ( srcMode == SM_F2F && fileExists ( outName ) ) { + if (forceOverwrite) { + remove(outName); + } else { + fprintf ( stderr, "%s: Output file %s already exists.\n", + progName, outName ); + setExit(1); + return; + } } if ( srcMode == SM_F2F && !forceOverwrite && (n=countHardLinks ( inName )) > 0) { @@ -1267,6 +1285,12 @@ void compress ( Char *name ) return; } + if ( srcMode == SM_F2F ) { + /* Save the file's meta-info before we open it. Doing it later + means we mess up the access times. */ + saveInputFileMetaInfo ( inName ); + } + switch ( srcMode ) { case SM_I2O: @@ -1306,7 +1330,7 @@ void compress ( Char *name ) case SM_F2F: inStr = fopen ( inName, "rb" ); - outStr = fopen ( outName, "wb" ); + outStr = fopen_output_safely ( outName, "wb" ); if ( outStr == NULL) { fprintf ( stderr, "%s: Can't create output file %s: %s.\n", progName, outName, strerror(errno) ); @@ -1321,7 +1345,6 @@ void compress ( Char *name ) setExit(1); return; }; - setInterimPermissions ( outName ); break; default: @@ -1343,7 +1366,7 @@ void compress ( Char *name ) /*--- If there was an I/O error, we won't get here. ---*/ if ( srcMode == SM_F2F ) { - copyDatePermissionsAndOwner ( inName, outName ); + applySavedMetaInfoToOutputFile ( outName ); deleteOutputOnInterrupt = False; if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); @@ -1364,6 +1387,7 @@ void uncompress ( Char *name ) Int32 n, i; Bool magicNumberOK; Bool cantGuess; + struct MY_STAT statBuf; deleteOutputOnInterrupt = False; @@ -1405,6 +1429,16 @@ void uncompress ( Char *name ) setExit(1); return; } + if ( srcMode == SM_F2F || srcMode == SM_F2O ) { + MY_STAT(inName, &statBuf); + if ( MY_S_ISDIR(statBuf.st_mode) ) { + fprintf( stderr, + "%s: Input file %s is a directory.\n", + progName,inName); + setExit(1); + return; + } + } if ( srcMode == SM_F2F && !forceOverwrite && notAStandardFile ( inName )) { if (noisy) fprintf ( stderr, "%s: Input file %s is not a normal file.\n", @@ -1419,11 +1453,15 @@ void uncompress ( Char *name ) progName, inName, outName ); /* just a warning, no return */ } - if ( srcMode == SM_F2F && !forceOverwrite && fileExists ( outName ) ) { - fprintf ( stderr, "%s: Output file %s already exists.\n", - progName, outName ); - setExit(1); - return; + if ( srcMode == SM_F2F && fileExists ( outName ) ) { + if (forceOverwrite) { + remove(outName); + } else { + fprintf ( stderr, "%s: Output file %s already exists.\n", + progName, outName ); + setExit(1); + return; + } } if ( srcMode == SM_F2F && !forceOverwrite && (n=countHardLinks ( inName ) ) > 0) { @@ -1433,6 +1471,12 @@ void uncompress ( Char *name ) return; } + if ( srcMode == SM_F2F ) { + /* Save the file's meta-info before we open it. Doing it later + means we mess up the access times. */ + saveInputFileMetaInfo ( inName ); + } + switch ( srcMode ) { case SM_I2O: @@ -1463,7 +1507,7 @@ void uncompress ( Char *name ) case SM_F2F: inStr = fopen ( inName, "rb" ); - outStr = fopen ( outName, "wb" ); + outStr = fopen_output_safely ( outName, "wb" ); if ( outStr == NULL) { fprintf ( stderr, "%s: Can't create output file %s: %s.\n", progName, outName, strerror(errno) ); @@ -1478,7 +1522,6 @@ void uncompress ( Char *name ) setExit(1); return; }; - setInterimPermissions ( outName ); break; default: @@ -1501,7 +1544,7 @@ void uncompress ( Char *name ) /*--- If there was an I/O error, we won't get here. ---*/ if ( magicNumberOK ) { if ( srcMode == SM_F2F ) { - copyDatePermissionsAndOwner ( inName, outName ); + applySavedMetaInfoToOutputFile ( outName ); deleteOutputOnInterrupt = False; if ( !keepInputFiles ) { IntNative retVal = remove ( inName ); @@ -1539,6 +1582,7 @@ void testf ( Char *name ) { FILE *inStr; Bool allOK; + struct MY_STAT statBuf; deleteOutputOnInterrupt = False; @@ -1565,6 +1609,16 @@ void testf ( Char *name ) setExit(1); return; } + if ( srcMode != SM_I2O ) { + MY_STAT(inName, &statBuf); + if ( MY_S_ISDIR(statBuf.st_mode) ) { + fprintf( stderr, + "%s: Input file %s is a directory.\n", + progName,inName); + setExit(1); + return; + } + } switch ( srcMode ) { @@ -1603,6 +1657,7 @@ void testf ( Char *name ) } /*--- Now the input handle is sane. Do the Biz. ---*/ + outputHandleJustInCase = NULL; allOK = testStream ( inStr ); if (allOK && verbosity >= 1) fprintf ( stderr, "ok\n" ); @@ -1619,7 +1674,7 @@ void license ( void ) "bzip2, a block-sorting file compressor. " "Version %s.\n" " \n" - " Copyright (C) 1996-2000 by Julian Seward.\n" + " Copyright (C) 1996-2002 by Julian Seward.\n" " \n" " This program is free software; you can redistribute it and/or modify\n" " it under the terms set out in the LICENSE file, which is included\n" @@ -1658,6 +1713,8 @@ void usage ( Char *fullProgName ) " -V --version display software version & license\n" " -s --small use less memory (at most 2500k)\n" " -1 .. -9 set block size to 100k .. 900k\n" + " --fast alias for -1\n" + " --best alias for -9\n" "\n" " If invoked as `bzip2', default action is to compress.\n" " as `bunzip2', default action is to decompress.\n" @@ -1666,9 +1723,9 @@ void usage ( Char *fullProgName ) " If no file names are given, bzip2 compresses or decompresses\n" " from standard input to standard output. You can combine\n" " short flags, so `-v -4' means the same as -v4 or -4v, &c.\n" -#if BZ_UNIX +# if BZ_UNIX "\n" -#endif +# endif , BZ2_bzlibVersion(), @@ -1818,11 +1875,11 @@ IntNative main ( IntNative argc, Char *argv[] ) /*-- Set up signal handlers for mem access errors --*/ signal (SIGSEGV, mySIGSEGVorSIGBUScatcher); -#if BZ_UNIX -#ifndef __DJGPP__ +# if BZ_UNIX +# ifndef __DJGPP__ signal (SIGBUS, mySIGSEGVorSIGBUScatcher); -#endif -#endif +# endif +# endif copyFileName ( inName, "(none)" ); copyFileName ( outName, "(none)" ); @@ -1933,6 +1990,8 @@ IntNative main ( IntNative argc, Char *argv[] ) if (ISFLAG("--exponential")) workFactor = 1; else if (ISFLAG("--repetitive-best")) redundant(aa->name); else if (ISFLAG("--repetitive-fast")) redundant(aa->name); else + if (ISFLAG("--fast")) blockSize100k = 1; else + if (ISFLAG("--best")) blockSize100k = 9; else if (ISFLAG("--verbose")) verbosity++; else if (ISFLAG("--help")) { usage ( progName ); exit ( 0 ); } else diff --git a/contrib/bzip2/bzip2recover.c b/contrib/bzip2/bzip2recover.c index ba3d175632a7..286873b8c07b 100644 --- a/contrib/bzip2/bzip2recover.c +++ b/contrib/bzip2/bzip2recover.c @@ -9,7 +9,7 @@ salvage from damaged files created by the accompanying bzip2-1.0 program. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -57,6 +57,29 @@ #include #include + +/* This program records bit locations in the file to be recovered. + That means that if 64-bit ints are not supported, we will not + be able to recover .bz2 files over 512MB (2^32 bits) long. + On GNU supported platforms, we take advantage of the 64-bit + int support to circumvent this problem. Ditto MSVC. + + This change occurred in version 1.0.2; all prior versions have + the 512MB limitation. +*/ +#ifdef __GNUC__ + typedef unsigned long long int MaybeUInt64; +# define MaybeUInt64_FMT "%Lu" +#else +#ifdef _MSC_VER + typedef unsigned __int64 MaybeUInt64; +# define MaybeUInt64_FMT "%I64u" +#else + typedef unsigned int MaybeUInt64; +# define MaybeUInt64_FMT "%u" +#endif +#endif + typedef unsigned int UInt32; typedef int Int32; typedef unsigned char UChar; @@ -66,13 +89,25 @@ typedef unsigned char Bool; #define False ((Bool)0) -Char inFileName[2000]; -Char outFileName[2000]; -Char progName[2000]; +#define BZ_MAX_FILENAME 2000 -UInt32 bytesOut = 0; -UInt32 bytesIn = 0; +Char inFileName[BZ_MAX_FILENAME]; +Char outFileName[BZ_MAX_FILENAME]; +Char progName[BZ_MAX_FILENAME]; +MaybeUInt64 bytesOut = 0; +MaybeUInt64 bytesIn = 0; + + +/*---------------------------------------------------*/ +/*--- Header bytes ---*/ +/*---------------------------------------------------*/ + +#define BZ_HDR_B 0x42 /* 'B' */ +#define BZ_HDR_Z 0x5a /* 'Z' */ +#define BZ_HDR_h 0x68 /* 'h' */ +#define BZ_HDR_0 0x30 /* '0' */ + /*---------------------------------------------------*/ /*--- I/O errors ---*/ @@ -116,6 +151,23 @@ void mallocFail ( Int32 n ) } +/*---------------------------------------------*/ +void tooManyBlocks ( Int32 max_handled_blocks ) +{ + fprintf ( stderr, + "%s: `%s' appears to contain more than %d blocks\n", + progName, inFileName, max_handled_blocks ); + fprintf ( stderr, + "%s: and cannot be handled. To fix, increase\n", + progName ); + fprintf ( stderr, + "%s: BZ_MAX_HANDLED_BLOCKS in bzip2recover.c, and recompile.\n", + progName ); + exit ( 1 ); +} + + + /*---------------------------------------------------*/ /*--- Bit stream I/O ---*/ /*---------------------------------------------------*/ @@ -254,27 +306,37 @@ Bool endsInBz2 ( Char* name ) /*--- ---*/ /*---------------------------------------------------*/ +/* This logic isn't really right when it comes to Cygwin. */ +#ifdef _WIN32 +# define BZ_SPLIT_SYM '\\' /* path splitter on Windows platform */ +#else +# define BZ_SPLIT_SYM '/' /* path splitter on Unix platform */ +#endif + #define BLOCK_HEADER_HI 0x00003141UL #define BLOCK_HEADER_LO 0x59265359UL #define BLOCK_ENDMARK_HI 0x00001772UL #define BLOCK_ENDMARK_LO 0x45385090UL +/* Increase if necessary. However, a .bz2 file with > 50000 blocks + would have an uncompressed size of at least 40GB, so the chances + are low you'll need to up this. +*/ +#define BZ_MAX_HANDLED_BLOCKS 50000 -UInt32 bStart[20000]; -UInt32 bEnd[20000]; -UInt32 rbStart[20000]; -UInt32 rbEnd[20000]; +MaybeUInt64 bStart [BZ_MAX_HANDLED_BLOCKS]; +MaybeUInt64 bEnd [BZ_MAX_HANDLED_BLOCKS]; +MaybeUInt64 rbStart[BZ_MAX_HANDLED_BLOCKS]; +MaybeUInt64 rbEnd [BZ_MAX_HANDLED_BLOCKS]; Int32 main ( Int32 argc, Char** argv ) { FILE* inFile; FILE* outFile; BitStream* bsIn, *bsWr; - Int32 currBlock, b, wrBlock; - UInt32 bitsRead; - Int32 rbCtr; - + Int32 b, wrBlock, currBlock, rbCtr; + MaybeUInt64 bitsRead; UInt32 buffHi, buffLo, blockCRC; Char* p; @@ -282,11 +344,37 @@ Int32 main ( Int32 argc, Char** argv ) strcpy ( progName, argv[0] ); inFileName[0] = outFileName[0] = 0; - fprintf ( stderr, "bzip2recover 1.0: extracts blocks from damaged .bz2 files.\n" ); + fprintf ( stderr, + "bzip2recover 1.0.2: extracts blocks from damaged .bz2 files.\n" ); if (argc != 2) { fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n", progName, progName ); + switch (sizeof(MaybeUInt64)) { + case 8: + fprintf(stderr, + "\trestrictions on size of recovered file: None\n"); + break; + case 4: + fprintf(stderr, + "\trestrictions on size of recovered file: 512 MB\n"); + fprintf(stderr, + "\tto circumvent, recompile with MaybeUInt64 as an\n" + "\tunsigned 64-bit int.\n"); + break; + default: + fprintf(stderr, + "\tsizeof(MaybeUInt64) is not 4 or 8 -- " + "configuration error.\n"); + break; + } + exit(1); + } + + if (strlen(argv[1]) >= BZ_MAX_FILENAME-20) { + fprintf ( stderr, + "%s: supplied filename is suspiciously (>= %d chars) long. Bye!\n", + progName, strlen(argv[1]) ); exit(1); } @@ -316,7 +404,8 @@ Int32 main ( Int32 argc, Char** argv ) (bitsRead - bStart[currBlock]) >= 40) { bEnd[currBlock] = bitsRead-1; if (currBlock > 0) - fprintf ( stderr, " block %d runs from %d to %d (incomplete)\n", + fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT + " to " MaybeUInt64_FMT " (incomplete)\n", currBlock, bStart[currBlock], bEnd[currBlock] ); } else currBlock--; @@ -330,17 +419,22 @@ Int32 main ( Int32 argc, Char** argv ) ( (buffHi & 0x0000ffff) == BLOCK_ENDMARK_HI && buffLo == BLOCK_ENDMARK_LO) ) { - if (bitsRead > 49) - bEnd[currBlock] = bitsRead-49; else + if (bitsRead > 49) { + bEnd[currBlock] = bitsRead-49; + } else { bEnd[currBlock] = 0; + } if (currBlock > 0 && (bEnd[currBlock] - bStart[currBlock]) >= 130) { - fprintf ( stderr, " block %d runs from %d to %d\n", + fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT + " to " MaybeUInt64_FMT "\n", rbCtr+1, bStart[currBlock], bEnd[currBlock] ); rbStart[rbCtr] = bStart[currBlock]; rbEnd[rbCtr] = bEnd[currBlock]; rbCtr++; } + if (currBlock >= BZ_MAX_HANDLED_BLOCKS) + tooManyBlocks(BZ_MAX_HANDLED_BLOCKS); currBlock++; bStart[currBlock] = bitsRead; @@ -400,10 +494,25 @@ Int32 main ( Int32 argc, Char** argv ) wrBlock++; } else if (bitsRead == rbStart[wrBlock]) { - outFileName[0] = 0; - sprintf ( outFileName, "rec%4d", wrBlock+1 ); - for (p = outFileName; *p != 0; p++) if (*p == ' ') *p = '0'; - strcat ( outFileName, inFileName ); + /* Create the output file name, correctly handling leading paths. + (31.10.2001 by Sergey E. Kusikov) */ + Char* split; + Int32 ofs, k; + for (k = 0; k < BZ_MAX_FILENAME; k++) + outFileName[k] = 0; + strcpy (outFileName, inFileName); + split = strrchr (outFileName, BZ_SPLIT_SYM); + if (split == NULL) { + split = outFileName; + } else { + ++split; + } + /* Now split points to the start of the basename. */ + ofs = split - outFileName; + sprintf (split, "rec%5d", wrBlock+1); + for (p = split; *p != 0; p++) if (*p == ' ') *p = '0'; + strcat (outFileName, inFileName + ofs); + if ( !endsInBz2(outFileName)) strcat ( outFileName, ".bz2" ); fprintf ( stderr, " writing block %d to `%s' ...\n", @@ -416,8 +525,10 @@ Int32 main ( Int32 argc, Char** argv ) exit(1); } bsWr = bsOpenWriteStream ( outFile ); - bsPutUChar ( bsWr, 'B' ); bsPutUChar ( bsWr, 'Z' ); - bsPutUChar ( bsWr, 'h' ); bsPutUChar ( bsWr, '9' ); + bsPutUChar ( bsWr, BZ_HDR_B ); + bsPutUChar ( bsWr, BZ_HDR_Z ); + bsPutUChar ( bsWr, BZ_HDR_h ); + bsPutUChar ( bsWr, BZ_HDR_0 + 9 ); bsPutUChar ( bsWr, 0x31 ); bsPutUChar ( bsWr, 0x41 ); bsPutUChar ( bsWr, 0x59 ); bsPutUChar ( bsWr, 0x26 ); bsPutUChar ( bsWr, 0x53 ); bsPutUChar ( bsWr, 0x59 ); diff --git a/contrib/bzip2/bzlib.c b/contrib/bzip2/bzlib.c index 4a06d9f14b91..7d1cb275f5b9 100644 --- a/contrib/bzip2/bzlib.c +++ b/contrib/bzip2/bzlib.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -93,10 +93,39 @@ void BZ2_bz__AssertH__fail ( int errcode ) "component, you should also report this bug to the author(s)\n" "of that program. Please make an effort to report this bug;\n" "timely and accurate bug reports eventually lead to higher\n" - "quality software. Thanks. Julian Seward, 21 March 2000.\n\n", + "quality software. Thanks. Julian Seward, 30 December 2001.\n\n", errcode, BZ2_bzlibVersion() ); + + if (errcode == 1007) { + fprintf(stderr, + "\n*** A special note about internal error number 1007 ***\n" + "\n" + "Experience suggests that a common cause of i.e. 1007\n" + "is unreliable memory or other hardware. The 1007 assertion\n" + "just happens to cross-check the results of huge numbers of\n" + "memory reads/writes, and so acts (unintendedly) as a stress\n" + "test of your memory system.\n" + "\n" + "I suggest the following: try compressing the file again,\n" + "possibly monitoring progress in detail with the -vv flag.\n" + "\n" + "* If the error cannot be reproduced, and/or happens at different\n" + " points in compression, you may have a flaky memory system.\n" + " Try a memory-test program. I have used Memtest86\n" + " (www.memtest86.com). At the time of writing it is free (GPLd).\n" + " Memtest86 tests memory much more thorougly than your BIOSs\n" + " power-on test, and may find failures that the BIOS doesn't.\n" + "\n" + "* If the error can be repeatably reproduced, this is a bug in\n" + " bzip2, and I would very much like to hear about it. Please\n" + " let me know, and, ideally, save a copy of the file causing the\n" + " problem -- without which I will be unable to investigate it.\n" + "\n" + ); + } + exit(3); } #endif @@ -1402,7 +1431,7 @@ BZFILE * bzopen_or_bzdopen smallMode = 1; break; default: if (isdigit((int)(*mode))) { - blockSize100k = *mode-'0'; + blockSize100k = *mode-BZ_HDR_0; } } mode++; diff --git a/contrib/bzip2/bzlib.h b/contrib/bzip2/bzlib.h index c9447a295876..9ac43a169da5 100644 --- a/contrib/bzip2/bzlib.h +++ b/contrib/bzip2/bzlib.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -110,8 +110,10 @@ typedef #define BZ_EXPORT #endif +/* Need a definitition for FILE */ +#include + #ifdef _WIN32 -# include # include # ifdef small /* windows.h define small to char */ diff --git a/contrib/bzip2/bzlib_private.h b/contrib/bzip2/bzlib_private.h index fb51c7a1d4b4..ff973c3bfd03 100644 --- a/contrib/bzip2/bzlib_private.h +++ b/contrib/bzip2/bzlib_private.h @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -76,7 +76,7 @@ /*-- General stuff. --*/ -#define BZ_VERSION "1.0.1, 23-June-2000" +#define BZ_VERSION "1.0.2, 30-Dec-2001" typedef char Char; typedef unsigned char Bool; @@ -137,6 +137,13 @@ extern void bz_internal_error ( int errcode ); #define BZFREE(ppp) (strm->bzfree)(strm->opaque,(ppp)) +/*-- Header bytes. --*/ + +#define BZ_HDR_B 0x42 /* 'B' */ +#define BZ_HDR_Z 0x5a /* 'Z' */ +#define BZ_HDR_h 0x68 /* 'h' */ +#define BZ_HDR_0 0x30 /* '0' */ + /*-- Constants for the back end. --*/ #define BZ_MAX_ALPHA_SIZE 258 diff --git a/contrib/bzip2/compress.c b/contrib/bzip2/compress.c index cc5e31d6f0ec..56501c115533 100644 --- a/contrib/bzip2/compress.c +++ b/contrib/bzip2/compress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -663,10 +663,10 @@ void BZ2_compressBlock ( EState* s, Bool is_last_block ) /*-- If this is the first block, create the stream header. --*/ if (s->blockNo == 1) { BZ2_bsInitWrite ( s ); - bsPutUChar ( s, 'B' ); - bsPutUChar ( s, 'Z' ); - bsPutUChar ( s, 'h' ); - bsPutUChar ( s, (UChar)('0' + s->blockSize100k) ); + bsPutUChar ( s, BZ_HDR_B ); + bsPutUChar ( s, BZ_HDR_Z ); + bsPutUChar ( s, BZ_HDR_h ); + bsPutUChar ( s, (UChar)(BZ_HDR_0 + s->blockSize100k) ); } if (s->nblock > 0) { diff --git a/contrib/bzip2/crctable.c b/contrib/bzip2/crctable.c index 61c040c4fcfc..b16746ae1942 100644 --- a/contrib/bzip2/crctable.c +++ b/contrib/bzip2/crctable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions diff --git a/contrib/bzip2/decompress.c b/contrib/bzip2/decompress.c index cdced1889061..e9213473acf4 100644 --- a/contrib/bzip2/decompress.c +++ b/contrib/bzip2/decompress.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -235,18 +235,18 @@ Int32 BZ2_decompress ( DState* s ) switch (s->state) { GET_UCHAR(BZ_X_MAGIC_1, uc); - if (uc != 'B') RETURN(BZ_DATA_ERROR_MAGIC); + if (uc != BZ_HDR_B) RETURN(BZ_DATA_ERROR_MAGIC); GET_UCHAR(BZ_X_MAGIC_2, uc); - if (uc != 'Z') RETURN(BZ_DATA_ERROR_MAGIC); + if (uc != BZ_HDR_Z) RETURN(BZ_DATA_ERROR_MAGIC); GET_UCHAR(BZ_X_MAGIC_3, uc) - if (uc != 'h') RETURN(BZ_DATA_ERROR_MAGIC); + if (uc != BZ_HDR_h) RETURN(BZ_DATA_ERROR_MAGIC); GET_BITS(BZ_X_MAGIC_4, s->blockSize100k, 8) - if (s->blockSize100k < '1' || - s->blockSize100k > '9') RETURN(BZ_DATA_ERROR_MAGIC); - s->blockSize100k -= '0'; + if (s->blockSize100k < (BZ_HDR_0 + 1) || + s->blockSize100k > (BZ_HDR_0 + 9)) RETURN(BZ_DATA_ERROR_MAGIC); + s->blockSize100k -= BZ_HDR_0; if (s->smallDecompress) { s->ll16 = BZALLOC( s->blockSize100k * 100000 * sizeof(UInt16) ); diff --git a/contrib/bzip2/dlltest.c b/contrib/bzip2/dlltest.c index f79279cef846..2d7dcca4cb59 100644 --- a/contrib/bzip2/dlltest.c +++ b/contrib/bzip2/dlltest.c @@ -19,7 +19,7 @@ #ifdef _WIN32 -#define BZ2_LIBNAME "libbz2-1.0.0.DLL" +#define BZ2_LIBNAME "libbz2-1.0.2.DLL" #include static int BZ2DLLLoaded = 0; @@ -130,8 +130,8 @@ int main(int argc,char *argv[]) }else{ fp_w = stdout; } - if((BZ2fp_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL) - || (BZ2fp_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){ + if((fn_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL) + || (fn_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){ printf("can't bz2openstream\n"); exit(1); } diff --git a/contrib/bzip2/huffman.c b/contrib/bzip2/huffman.c index 9b446c4b36d1..293095c170c0 100644 --- a/contrib/bzip2/huffman.c +++ b/contrib/bzip2/huffman.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions diff --git a/contrib/bzip2/makefile.msc b/contrib/bzip2/makefile.msc index 3fe42324aca1..799a18a5f1a7 100644 --- a/contrib/bzip2/makefile.msc +++ b/contrib/bzip2/makefile.msc @@ -4,7 +4,7 @@ # Fixed up by JRS for bzip2-0.9.5d release. CC=cl -CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 +CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 -nologo OBJS= blocksort.obj \ huffman.obj \ diff --git a/contrib/bzip2/manual.texi b/contrib/bzip2/manual.texi index 336776ab805a..5bc27d5f9f90 100644 --- a/contrib/bzip2/manual.texi +++ b/contrib/bzip2/manual.texi @@ -2,10 +2,10 @@ @setfilename bzip2.info @ignore -This file documents bzip2 version 1.0, and associated library +This file documents bzip2 version 1.0.2, and associated library libbzip2, written by Julian Seward (jseward@acm.org). -Copyright (C) 1996-2000 Julian R Seward +Copyright (C) 1996-2002 Julian R Seward Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice @@ -30,8 +30,8 @@ END-INFO-DIR-ENTRY @titlepage @title bzip2 and libbzip2 @subtitle a program and library for data compression -@subtitle copyright (C) 1996-2000 Julian Seward -@subtitle version 1.0 of 21 March 2000 +@subtitle copyright (C) 1996-2002 Julian Seward +@subtitle version 1.0.2 of 30 December 2001 @author Julian Seward @end titlepage @@ -40,11 +40,17 @@ END-INFO-DIR-ENTRY @parskip 2mm @end iftex -@node Top, Overview, (dir), (dir) +@node Top,,, (dir) + +The following text is the License for this software. You should +find it identical to that contained in the file LICENSE in the +source distribution. + +@bf{------------------ START OF THE LICENSE ------------------} This program, @code{bzip2}, and associated library @code{libbzip2}, are -Copyright (C) 1996-2000 Julian R Seward. All rights reserved. +Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions @@ -82,14 +88,16 @@ Julian Seward, Cambridge, UK. @code{jseward@@acm.org} -@code{http://sourceware.cygnus.com/bzip2} +@code{bzip2}/@code{libbzip2} version 1.0.2 of 30 December 2001. + +@bf{------------------ END OF THE LICENSE ------------------} + +Web sites: + +@code{http://sources.redhat.com/bzip2} @code{http://www.cacheprof.org} -@code{http://www.muraroa.demon.co.uk} - -@code{bzip2}/@code{libbzip2} version 1.0 of 21 March 2000. - PATENTS: To the best of my knowledge, @code{bzip2} does not use any patented algorithms. However, I do not have the resources available to carry out a full patent search. Therefore I cannot give any guarantee of the @@ -101,7 +109,6 @@ above statement. -@node Overview, Implementation, Top, Top @chapter Introduction @code{bzip2} compresses files using the Burrows-Wheeler @@ -134,7 +141,7 @@ and nothing else. @unnumberedsubsubsec NAME @itemize @item @code{bzip2}, @code{bunzip2} -- a block-sorting file compressor, v1.0 +- a block-sorting file compressor, v1.0.2 @item @code{bzcat} - decompresses files to stdout @item @code{bzip2recover} @@ -264,6 +271,11 @@ This really performs a trial decompression and throws away the result. Force overwrite of output files. Normally, @code{bzip2} will not overwrite existing output files. Also forces @code{bzip2} to break hard links to files, which it otherwise wouldn't do. + +@code{bzip2} normally declines to decompress files which don't have the +correct magic header bytes. If forced (@code{-f}), however, it will +pass such files through unmodified. This is how GNU @code{gzip} +behaves. @item -k --keep Keep (don't delete) input files during compression or decompression. @@ -286,9 +298,13 @@ Further @code{-v}'s increase the verbosity level, spewing out lots of information which is primarily of interest for diagnostic purposes. @item -L --license -V --version Display the software version, license terms and conditions. -@item -1 to -9 +@item -1 (or --fast) to -9 (or --best) Set the block size to 100 k, 200 k .. 900 k when compressing. Has no effect when decompressing. See MEMORY MANAGEMENT below. +The @code{--fast} and @code{--best} aliases are primarily for GNU +@code{gzip} compatibility. In particular, @code{--fast} doesn't make +things significantly faster. And @code{--best} merely selects the +default behaviour. @item -- Treats all subsequent arguments as file names, even if they start with a dash. This is so you can handle files with names beginning @@ -389,21 +405,19 @@ integrity of the resulting files, and decompress those which are undamaged. @code{bzip2recover} -takes a single argument, the name of the damaged file, -and writes a number of files @code{rec0001file.bz2}, - @code{rec0002file.bz2}, etc, containing the extracted blocks. - The output filenames are designed so that the use of - wildcards in subsequent processing -- for example, -@code{bzip2 -dc rec*file.bz2 > recovered_data} -- lists the files in - the correct order. +takes a single argument, the name of the damaged file, and writes a +number of files @code{rec00001file.bz2}, @code{rec00002file.bz2}, etc, +containing the extracted blocks. The output filenames are designed so +that the use of wildcards in subsequent processing -- for example, +@code{bzip2 -dc rec*file.bz2 > recovered_data} -- processes the files in +the correct order. @code{bzip2recover} should be of most use dealing with large @code{.bz2} - files, as these will contain many blocks. It is clearly - futile to use it on damaged single-block files, since a - damaged block cannot be recovered. If you wish to minimise -any potential data loss through media or transmission errors, -you might consider compressing with a smaller - block size. +files, as these will contain many blocks. It is clearly futile to use +it on damaged single-block files, since a damaged block cannot be +recovered. If you wish to minimise any potential data loss through +media or transmission errors, you might consider compressing with a +smaller block size. @unnumberedsubsubsec PERFORMANCE NOTES @@ -435,22 +449,31 @@ I/O error messages are not as helpful as they could be. @code{bzip2} tries hard to detect I/O errors and exit cleanly, but the details of what the problem is sometimes seem rather misleading. -This manual page pertains to version 1.0 of @code{bzip2}. Compressed +This manual page pertains to version 1.0.2 of @code{bzip2}. Compressed data created by this version is entirely forwards and backwards -compatible with the previous public releases, versions 0.1pl2, 0.9.0 and -0.9.5, but with the following exception: 0.9.0 and above can correctly -decompress multiple concatenated compressed files. 0.1pl2 cannot do -this; it will stop after decompressing just the first file in the -stream. +compatible with the previous public releases, versions 0.1pl2, 0.9.0, +0.9.5, 1.0.0 and 1.0.1, but with the following exception: 0.9.0 and +above can correctly decompress multiple concatenated compressed files. +0.1pl2 cannot do this; it will stop after decompressing just the first +file in the stream. + +@code{bzip2recover} versions prior to this one, 1.0.2, used 32-bit +integers to represent bit positions in compressed files, so it could not +handle compressed files more than 512 megabytes long. Version 1.0.2 and +above uses 64-bit ints on some platforms which support them (GNU +supported targets, and Windows). To establish whether or not +@code{bzip2recover} was built with such a limitation, run it without +arguments. In any event you can build yourself an unlimited version if +you can recompile it with @code{MaybeUInt64} set to be an unsigned +64-bit integer. -@code{bzip2recover} uses 32-bit integers to represent bit positions in -compressed files, so it cannot handle compressed files more than 512 -megabytes long. This could easily be fixed. @unnumberedsubsubsec AUTHOR Julian Seward, @code{jseward@@acm.org}. +@code{http://sources.redhat.com/bzip2} + The ideas embodied in @code{bzip2} are due to (at least) the following people: Michael Burrows and David Wheeler (for the block sorting transformation), David Wheeler (again, for the Huffman coder), Peter @@ -461,8 +484,9 @@ indebted for their help, support and advice. See the manual in the source distribution for pointers to sources of documentation. Christian von Roques encouraged me to look for faster sorting algorithms, so as to speed up compression. Bela Lubkin encouraged me to improve the -worst-case compression performance. Many people sent patches, helped -with portability problems, lent machines, gave advice and were generally +worst-case compression performance. The @code{bz*} scripts are derived +from those of GNU @code{gzip}. Many people sent patches, helped with +portability problems, lent machines, gave advice and were generally helpful. @end quotation @@ -1769,16 +1793,20 @@ was compiled with @code{BZ_NO_STDIO} set. For a normal compile, an assertion failure yields the message @example bzip2/libbzip2: internal error number N. - This is a bug in bzip2/libbzip2, 1.0 of 21-Mar-2000. + This is a bug in bzip2/libbzip2, 1.0.2, 30-Dec-2001. Please report it to me at: jseward@@acm.org. If this happened when you were using some program which uses libbzip2 as a component, you should also report this bug to the author(s) of that program. Please make an effort to report this bug; timely and accurate bug reports eventually lead to higher - quality software. Thanks. Julian Seward, 21 March 2000. + quality software. Thanks. Julian Seward, 30 December 2001. @end example -where @code{N} is some error code number. @code{exit(3)} -is then called. +where @code{N} is some error code number. If @code{N == 1007}, it also +prints some extra text advising the reader that unreliable memory is +often associated with internal error 1007. (This is a +frequently-observed-phenomenon with versions 1.0.0/1.0.1). + +@code{exit(3)} is then called. For a @code{stdio}-free library, assertion failures result in a call to a function declared as: @@ -2056,10 +2084,10 @@ Maybe this isn't what you want. If you want a compressor and/or library which is faster, uses less memory but gets pretty good compression, and has minimal latency, consider Jean-loup -Gailly's and Mark Adler's work, @code{zlib-1.1.2} and +Gailly's and Mark Adler's work, @code{zlib-1.1.3} and @code{gzip-1.2.4}. Look for them at -@code{http://www.cdrom.com/pub/infozip/zlib} and +@code{http://www.zlib.org} and @code{http://www.gzip.org} respectively. For something faster and lighter still, you might try Markus F X J diff --git a/contrib/bzip2/randtable.c b/contrib/bzip2/randtable.c index 983089d4684d..5c922e94f3c5 100644 --- a/contrib/bzip2/randtable.c +++ b/contrib/bzip2/randtable.c @@ -8,7 +8,7 @@ This file is a part of bzip2 and/or libbzip2, a program and library for lossless, block-sorting data compression. - Copyright (C) 1996-2000 Julian R Seward. All rights reserved. + Copyright (C) 1996-2002 Julian R Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions diff --git a/contrib/bzip2/sample1.bz2.uu b/contrib/bzip2/sample1.bz2.uu index 0464d68a613a..544568031af1 100644 --- a/contrib/bzip2/sample1.bz2.uu +++ b/contrib/bzip2/sample1.bz2.uu @@ -717,6 +717,6 @@ MVAOZV"VU%6CD_Y+<PH<,*[+@3I*9WZN)\X?A7.P\_S-F8!-?XU$[+/]%[OMZOW[:^XS2& M/_N;0^1F)GVW3)?_9,CJ?,A%],5ZI6ZKRX;`KQ^R4V*K,(;Z,@O$EOCU#?,X M(O7&I3N+30.D@RM96[&_-)0)/\49"=6$!810*A/^FW$`K -F"+`&FNP%5J%=[MCO,A9%D0D'L8&%51TCE#9?_B[DBG"A(9GC:TH* +F"+`&FNP%5J%=[MCO,A9%D0D'L8&%51TCE#9?_B[DBG"A(9GC:TH` ` end diff --git a/contrib/bzip2/sample1.ref.gz.uu b/contrib/bzip2/sample1.ref.gz.uu index 49ab65ecfaf4..473cdec28181 100644 --- a/contrib/bzip2/sample1.ref.gz.uu +++ b/contrib/bzip2/sample1.ref.gz.uu @@ -843,6 +843,6 @@ MJ_+AO[_/_A%_/GK\R?67[_WBLM/@WT_3)UI__$E_7K9RX[(SX`]G\#7KQ:'5 MQQ__H_]X[H'07^OPU\<\^3<7/M__*]Q_]?&S3_G'GU^U[*S)2?H/_W7-\4<_ M_<)TQ_]NK0G?77'-TW_R=_XCU)KKCY]QYMLFGN=_=18?X?35A5NK_E=GX6E/ MN^K7+V[:K\(?J_#-4[\;O&V7_2:^Z_KCR_[PP<:Y_C4+\*C_9]DIK_ABXD?\ -+?_\_)"$UH(B!`0#- ++?_\_)"$UH(B!`0`` ` end diff --git a/contrib/bzip2/sample2.bz2.uu b/contrib/bzip2/sample2.bz2.uu index ca65c3e5180b..b0610ea63c91 100644 --- a/contrib/bzip2/sample2.bz2.uu +++ b/contrib/bzip2/sample2.bz2.uu @@ -1637,6 +1637,6 @@ M;JSY,(NL]P##+]"_92.=1AVN2=FT-/T[%*PKJ^QYK=@&$>,^H>.I?7T3LE&= MAY6T4D'(:8^7=?7_<"!"UYY,N-:5YL.UK';)LA[)K@/D[5>DA39Z*DM/.\M( MCP/<2[?<@'172;O\%SKD:8B*5RN1YZ=]Z>]J%!]1N:86EL$>L?""B&1Y;??$ M1#Q8OPJS'!FKK$72:[U$:6%N38M[R6T9L_-\?S]GKCP'*"/(M^Z'HE>*F.:1 -6"B*@22E%*JXT83_Q=R13A0D'@N8^0&T9 +6"B*@22E%*JXT83_Q=R13A0D'@N8^0``` ` end diff --git a/contrib/bzip2/sample3.bz2.uu b/contrib/bzip2/sample3.bz2.uu index 3ea32a084a88..a2378a3cc17a 100644 --- a/contrib/bzip2/sample3.bz2.uu +++ b/contrib/bzip2/sample3.bz2.uu @@ -4,6 +4,6 @@ MHT](;4T&@H]-0R`!HT`BD%'I--&)H&C32IEOWIG15$$195,\[7SNV^DXEM\G M0\ZRK;7W3[@]P7B>KPBU87N[XI5>!/TRY1JZE9UN&;!6ZH,60K?O=56)J;1@ MAMLR":K)YTDE0*\.``1`$""QGH`*"(N^C.#]!@VC7I&-,2#+KPCP^J.FQY.Z MF8"5*83K]+=A;.A$:#-(KY':7&T/24RT:&EIR)5"L%+S'``4$1985NZ$GE_% -*W)%.%"0NLTSR0.A$ +*W)%.%"0NLTSR0``` ` end diff --git a/contrib/bzip2/words3 b/contrib/bzip2/words3 index 8486a84c8b39..7a6b462443cf 100644 --- a/contrib/bzip2/words3 +++ b/contrib/bzip2/words3 @@ -15,8 +15,8 @@ not actually execute them. Instructions for use are in the preformatted manual page, in the file bzip2.txt. For more detailed documentation, read the full manual. -It is available in Postscript form (manual.ps) and HTML form -(manual_toc.html). +It is available in Postscript form (manual.ps), PDF form (manual.pdf), +and HTML form (manual_toc.html). You can also do "bzip2 --help" to see some helpful information. "bzip2 -L" displays the software license.