9 - Fixed several C99/C11 conformance bugs. Now the code is clean
10 under gcc/clang -fsanitize=undefined. Some of these changes
11 might have a negative effect on performance with old GCC
12 versions or compilers other than GCC and Clang. The configure
13 option --enable-unsafe-type-punning can be used to (mostly)
14 restore the old behavior but it shouldn't normally be used.
16 - Improved API documentation of lzma_properties_decode().
18 - Added a very minor encoder speed optimization.
22 - Fixed a crash in "xz -dcfv not_an_xz_file". All four options
23 were required to trigger it. The crash occurred in the
24 progress indicator code when xz was in passthru mode where
27 - Fixed an integer overflow with 32-bit off_t. It could happen
28 when decompressing a file that has a long run of zero bytes
29 which xz would try to write as a sparse file. Since the build
30 system enables large file support by default, off_t is
31 normally 64-bit even on 32-bit systems.
33 - Fixes for --flush-timeout:
34 * Fix semi-busy-waiting.
35 * Avoid unneeded flushes when no new input has arrived
36 since the previous flush was completed.
38 - Added a special case for 32-bit xz: If --memlimit-compress is
39 used to specify a limit that exceeds 4020 MiB, the limit will
40 be set to 4020 MiB. The values "0" and "max" aren't affected
41 by this and neither is decompression. This hack can be
42 helpful when a 32-bit xz has access to 4 GiB address space
43 but the specified memlimit exceeds 4 GiB. This can happen
44 e.g. with some scripts.
46 - Capsicum sandbox is now enabled by default where available
47 (FreeBSD >= 10). The sandbox debug messages (xz -vv) were
48 removed since they seemed to be more annoying than useful.
50 - DOS build now requires DJGPP 2.05 instead of 2.04beta.
51 A workaround for a locale problem with DJGPP 2.05 was added.
53 * xzgrep and other scripts:
55 - Added a configure option --enable-path-for-scripts=PREFIX.
56 It is disabled by default except on Solaris where the default
57 is /usr/xpg4/bin. See INSTALL for details.
59 - Added a workaround for a POSIX shell detection problem on
64 - Added preliminary build instructions for z/OS. See INSTALL
67 - Experimental CMake support was added. It should work to build
68 static liblzma on a few operating systems. It may or may not
69 work to build shared liblzma. On some platforms it can build
70 xz and xzdec too but those are only for testing. See the
71 comment in the beginning of CMakeLists.txt for details.
73 - Visual Studio project files were updated.
74 WindowsTargetPlatformVersion was removed from VS2017 files
75 and set to "10.0" in the added VS2019 files. In the future
76 the VS project files will be removed when CMake support is
79 - New #defines in config.h: HAVE___BUILTIN_ASSUME_ALIGNED,
80 HAVE___BUILTIN_BSWAPXX, and TUKLIB_USE_UNSAFE_TYPE_PUNNING.
82 - autogen.sh has a new optional dependency on po4a and a new
83 option --no-po4a to skip that step. This matters only if one
84 wants to remake the build files. po4a is used to update the
85 translated man pages but as long as the man pages haven't
86 been modified, there's nothing to update and one can use
87 --no-po4a to avoid the dependency on po4a.
91 - XZ Utils translations are now handled by the Translation
92 Project: https://translationproject.org/domain/xz.html
94 - All man pages are now included in German too.
96 - New xz translations: Brazilian Portuguese, Finnish,
97 Hungarian, Chinese (simplified), Chinese (traditional),
98 and Danish (partial translation)
100 - Updated xz translations: French, German, Italian, and Polish
102 - Unfortunately a few new xz translations weren't included due
103 to technical problems like too long lines in --help output or
104 misaligned column headings in tables. In the future, many of
105 these strings will be split and e.g. the table column
106 alignment will be handled in software. This should make the
107 strings easier to translate.
114 - Allow 0 as memory usage limit instead of returning
115 LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified,
116 which effectively is the same as 0.
118 - Use "noexcept" keyword instead of "throw()" in the public
119 headers when a C++11 (or newer standard) compiler is used.
121 - Added a portability fix for recent Intel C Compilers.
123 - Microsoft Visual Studio build files have been moved under
124 windows/vs2013 and windows/vs2017.
128 - Fix "xz --list --robot missing_or_bad_file.xz" which would
129 try to print an uninitialized string and thus produce garbage
130 output. Since the exit status is non-zero, most uses of such
131 a command won't try to interpret the garbage output.
133 - "xz --list foo.xz" could print "Internal error (bug)" in a
134 corner case where a specific memory usage limit had been set.
141 - Always close a file before trying to delete it to avoid
142 problems on some operating system and file system combinations.
144 - Fixed copying of file timestamps on Windows.
146 - Added experimental (disabled by default) sandbox support using
147 Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL.
149 * C99/C11 conformance fixes to liblzma. The issues affected at least
150 some builds using link-time optimizations.
152 * Fixed bugs in the rarely-used function lzma_index_dup().
154 * Use of external SHA-256 code is now disabled by default.
155 It can still be enabled by passing --enable-external-sha256
156 to configure. The reasons to disable it by default (see INSTALL
159 - Some OS-specific SHA-256 implementations conflict with
160 OpenSSL and cause problems in programs that link against both
161 liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0
164 - The internal SHA-256 is faster than the SHA-256 code in
165 some operating systems.
167 * Changed CPU core count detection to use sched_getaffinity() on
168 GNU/Linux and GNU/kFreeBSD.
170 * Fixes to the build-system and xz to make xz buildable even when
171 encoders, decoders, or threading have been disabled from libilzma
172 using configure options. These fixes added two new #defines to
173 config.h: HAVE_ENCODERS and HAVE_DECODERS.
178 * Fixed bugs in QNX-specific code.
180 * Omitted the use of pipe2() even if it is available to avoid
181 portability issues with some old Linux and glibc combinations.
183 * Updated German translation.
185 * Added project files to build static and shared liblzma (not the
186 whole XZ Utils) with Visual Studio 2013 update 2 or later.
188 * Documented that threaded decompression hasn't been implemented
189 yet. A 5.2.0 NEWS entry describing multi-threading support had
190 incorrectly said "decompression" when it should have said
196 * Fixed a compression-ratio regression in fast mode of LZMA1 and
197 LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases.
199 * Fixed a portability problem in xz that affected at least OpenBSD.
201 * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs
202 from most other mktemp implementations.
204 * Changed CPU core count detection to use cpuset_getaffinity() on
212 * All fixes from 5.0.8
214 * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset
217 * xzdiff: If mktemp isn't installed, mkdir will be used as
218 a fallback to create a temporary directory. Installing mktemp
219 is still recommended.
221 * Updated French, German, Italian, Polish, and Vietnamese
224 Summary of fixes and new features added in the 5.1.x development
229 - Added support for multi-threaded compression. See the
230 lzma_mt structure, lzma_stream_encoder_mt(), and
231 lzma_stream_encoder_mt_memusage() in <lzma/container.h>,
232 lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads()
233 in <lzma/hardware.h> for details.
235 - Made the uses of lzma_allocator const correct.
237 - Added lzma_block_uncomp_encode() to create uncompressed
238 .xz Blocks using LZMA2 uncompressed chunks.
240 - Added support for LZMA_IGNORE_CHECK.
242 - A few speed optimizations were made.
244 - Added support for symbol versioning. It is enabled by default
245 on GNU/Linux, other GNU-based systems, and FreeBSD.
247 - liblzma (not the whole XZ Utils) should now be buildable
248 with MSVC 2013 update 2 or later using windows/config.h.
252 - Fixed a race condition in the signal handling. It was
253 possible that e.g. the first SIGINT didn't make xz exit
254 if reading or writing blocked and one had bad luck. The fix
255 is non-trivial, so as of writing it is unknown if it will be
256 backported to the v5.0 branch.
258 - Multi-threaded compression can be enabled with the
259 --threads (-T) option.
260 [Fixed: This originally said "decompression".]
262 - New command line options in xz: --single-stream,
263 --block-size=SIZE, --block-list=SIZES,
264 --flush-timeout=TIMEOUT, and --ignore-check.
266 - xz -lvv now shows the minimum xz version that is required to
267 decompress the file. Currently it is 5.0.0 for all supported
268 .xz files except files with empty LZMA2 streams require 5.0.2.
270 * xzdiff and xzgrep now support .lzo files if lzop is installed.
271 The .tzo suffix is also recognized as a shorthand for .tar.lzo.
274 5.1.4beta (2014-09-14)
276 * All fixes from 5.0.6
278 * liblzma: Fixed the use of presets in threaded encoder
281 * xz --block-list and --block-size can now be used together
282 in single-threaded mode. Previously the combination only
283 worked in multi-threaded mode.
285 * Added support for LZMA_IGNORE_CHECK to liblzma and made it
286 available in xz as --ignore-check.
288 * liblzma speed optimizations:
290 - Initialization of a new LZMA1 or LZMA2 encoder has been
291 optimized. (The speed of reinitializing an already-allocated
292 encoder isn't affected.) This helps when compressing many
293 small buffers with lzma_stream_buffer_encode() and other
294 similar situations where an already-allocated encoder state
295 isn't reused. This speed-up is visible in xz too if one
296 compresses many small files one at a time instead running xz
297 once and giving all files as command-line arguments.
299 - Buffer comparisons are now much faster when unaligned access
300 is allowed (configured with --enable-unaligned-access). This
301 speeds up encoding significantly. There is arch-specific code
302 for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best
303 results and there's no run-time CPU detection for now).
304 For other archs there is only generic code which probably
305 isn't as optimal as arch-specific solutions could be.
307 - A few speed optimizations were made to the SHA-256 code.
308 (Note that the builtin SHA-256 code isn't used on all
311 * liblzma can now be built with MSVC 2013 update 2 or later
312 using windows/config.h.
314 * Vietnamese translation was added.
317 5.1.3alpha (2013-10-26)
319 * All fixes from 5.0.5
323 - Fixed a deadlock in the threaded encoder.
325 - Made the uses of lzma_allocator const correct.
327 - Added lzma_block_uncomp_encode() to create uncompressed
328 .xz Blocks using LZMA2 uncompressed chunks.
330 - Added support for native threads on Windows and the ability
331 to detect the number of CPU cores.
335 - Fixed a race condition in the signal handling. It was
336 possible that e.g. the first SIGINT didn't make xz exit
337 if reading or writing blocked and one had bad luck. The fix
338 is non-trivial, so as of writing it is unknown if it will be
339 backported to the v5.0 branch.
341 - Made the progress indicator work correctly in threaded mode.
343 - Threaded encoder now works together with --block-list=SIZES.
345 - Added preliminary support for --flush-timeout=TIMEOUT.
346 It can be useful for (somewhat) real-time streaming. For
347 now the decompression side has to be done with something
348 else than the xz tool due to how xz does buffering, but this
352 5.1.2alpha (2012-07-04)
354 * All fixes from 5.0.3 and 5.0.4
358 - Fixed a deadlock and an invalid free() in the threaded encoder.
360 - Added support for symbol versioning. It is enabled by default
361 on GNU/Linux, other GNU-based systems, and FreeBSD.
363 - Use SHA-256 implementation from the operating system if one is
364 available in libc, libmd, or libutil. liblzma won't use e.g.
365 OpenSSL or libgcrypt to avoid introducing new dependencies.
367 - Fixed liblzma.pc for static linking.
369 - Fixed a few portability bugs.
371 * xz --decompress --single-stream now fixes the input position after
372 successful decompression. Now the following works:
374 echo foo | xz > foo.xz
375 echo bar | xz >> foo.xz
376 ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz
378 Note that it doesn't work if the input is not seekable
379 or if there is Stream Padding between the concatenated
382 * xz -lvv now shows the minimum xz version that is required to
383 decompress the file. Currently it is 5.0.0 for all supported .xz
384 files except files with empty LZMA2 streams require 5.0.2.
386 * Added an *incomplete* implementation of --block-list=SIZES to xz.
387 It only works correctly in single-threaded mode and when
388 --block-size isn't used at the same time. --block-list allows
389 specifying the sizes of Blocks which can be useful e.g. when
390 creating files for random-access reading.
393 5.1.1alpha (2011-04-12)
395 * All fixes from 5.0.2
397 * liblzma fixes that will also be included in 5.0.3:
399 - A memory leak was fixed.
401 - lzma_stream_buffer_encode() no longer creates an empty .xz
402 Block if encoding an empty buffer. Such an empty Block with
403 LZMA2 data would trigger a bug in 5.0.1 and older (see the
404 first bullet point in 5.0.2 notes). When releasing 5.0.2,
405 I thought that no encoder creates this kind of files but
408 - Validate function arguments better in a few functions. Most
409 importantly, specifying an unsupported integrity check to
410 lzma_stream_buffer_encode() no longer creates a corrupt .xz
411 file. Probably no application tries to do that, so this
412 shouldn't be a big problem in practice.
414 - Document that lzma_block_buffer_encode(),
415 lzma_easy_buffer_encode(), lzma_stream_encoder(), and
416 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
418 - The return values of the _memusage() functions are now
421 * Support for multithreaded compression was added using the simplest
422 method, which splits the input data into blocks and compresses
423 them independently. Other methods will be added in the future.
424 The current method has room for improvement, e.g. it is possible
425 to reduce the memory usage.
427 * Added the options --single-stream and --block-size=SIZE to xz.
429 * xzdiff and xzgrep now support .lzo files if lzop is installed.
430 The .tzo suffix is also recognized as a shorthand for .tar.lzo.
432 * Support for short 8.3 filenames under DOS was added to xz. It is
433 experimental and may change before it gets into a stable release.
438 * Fixed an old bug in xzgrep that affected OpenBSD and probably
439 a few other operating systems too.
441 * Updated French and German translations.
443 * Added support for detecting the amount of RAM on AmigaOS/AROS.
445 * Minor build system updates.
450 * Fix regressions introduced in 5.0.6:
452 - Fix building with non-GNU make.
454 - Fix invalid Libs.private value in liblzma.pc which broke
455 static linking against liblzma if the linker flags were
456 taken from pkg-config.
461 * xzgrep now exits with status 0 if at least one file matched.
463 * A few minor portability and build system fixes
468 * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing
469 .lzma files that have less common settings in the headers
470 (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed
471 size greater than 256 GiB). The limitations existed to avoid false
472 positives when detecting .lzma files. The lc + lp <= 4 limitation
473 still remains since liblzma's LZMA decoder has that limitation.
475 NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT
476 affected by this change. They still consider uncommon .lzma headers
477 as not being in the .lzma format. Changing this would give way too
478 many false positives.
482 - Interaction of preset and custom filter chain options was
483 made less illogical. This affects only certain less typical
484 uses cases so few people are expected to notice this change.
486 Now when a custom filter chain option (e.g. --lzma2) is
487 specified, all preset options (-0 ... -9, -e) earlier are on
488 the command line are completely forgotten. Similarly, when
489 a preset option is specified, all custom filter chain options
490 earlier on the command line are completely forgotten.
492 Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e"
493 which is equivalent to "xz -6e". Earlier -e didn't put xz back
494 into preset mode and thus the example command was equivalent
495 to "xz --lzma2=preset=5".
497 Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to
498 "xz -7". Earlier a custom filter chain option didn't make
499 xz forget the -e option so the example was equivalent to
502 - Fixes and improvements to error handling.
504 - Various fixes to the man page.
506 * xzless: Fixed to work with "less" versions 448 and later.
508 * xzgrep: Made -h an alias for --no-filename.
510 * Include the previously missing debug/translation.bash which can
511 be useful for translators.
513 * Include a build script for Mac OS X. This has been in the Git
514 repository since 2010 but due to a mistake in Makefile.am the
515 script hasn't been included in a release tarball before.
522 - Fix lzma_index_init(). It could crash if memory allocation
525 - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ
526 filter is used and the application only provides exactly as
527 much output space as is the uncompressed size of the file.
529 - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't
530 check if the last call to lzma_code() really returned
531 LZMA_STREAM_END, which made the program think that truncated
534 - New example programs in doc/examples (old programs are now in
535 doc/examples_old). These have more comments and more detailed
538 * Fix "xz -lvv foo.xz". It could crash on some corrupted files.
540 * Fix output of "xz --robot -lv" and "xz --robot -lvv" which
541 incorrectly printed the filename also in the "foo (x/x)" format.
543 * Fix exit status of "xzdiff foo.xz bar.xz".
545 * Fix exit status of "xzgrep foo binary_file".
547 * Fix portability to EBCDIC systems.
549 * Fix a configure issue on AIX with the XL C compiler. See INSTALL
552 * Update French, German, Italian, and Polish translations.
559 - A memory leak was fixed.
561 - lzma_stream_buffer_encode() no longer creates an empty .xz
562 Block if encoding an empty buffer. Such an empty Block with
563 LZMA2 data would trigger a bug in 5.0.1 and older (see the
564 first bullet point in 5.0.2 notes). When releasing 5.0.2,
565 I thought that no encoder creates this kind of files but
568 - Validate function arguments better in a few functions. Most
569 importantly, specifying an unsupported integrity check to
570 lzma_stream_buffer_encode() no longer creates a corrupt .xz
571 file. Probably no application tries to do that, so this
572 shouldn't be a big problem in practice.
574 - Document that lzma_block_buffer_encode(),
575 lzma_easy_buffer_encode(), lzma_stream_encoder(), and
576 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
578 - The return values of the _memusage() functions are now
581 * Fix command name detection in xzgrep. xzegrep and xzfgrep now
582 correctly use egrep and fgrep instead of grep.
584 * French translation was added.
589 * LZMA2 decompressor now correctly accepts LZMA2 streams with no
590 uncompressed data. Previously it considered them corrupt. The
591 bug can affect applications that use raw LZMA2 streams. It is
592 very unlikely to affect .xz files because no compressor creates
593 .xz files with empty LZMA2 streams. (Empty .xz files are a
594 different thing than empty LZMA2 streams.)
596 * "xz --suffix=.foo filename.foo" now refuses to compress the
597 file due to it already having the suffix .foo. It was already
598 documented on the man page, but the code lacked the test.
600 * "xzgrep -l foo bar.xz" works now.
602 * Polish translation was added.
607 * xz --force now (de)compresses files that have setuid, setgid,
608 or sticky bit set and files that have multiple hard links.
609 The man page had it documented this way already, but the code
612 * gzip and bzip2 support in xzdiff was fixed.
616 * Minor fix to Czech translation
621 Only the most important changes compared to 4.999.9beta are listed
622 here. One change is especially important:
624 * The memory usage limit is now disabled by default. Some scripts
625 written before this change may have used --memory=max on xz command
626 line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED
627 NOW, because they interfere with user's ability to set the memory
628 usage limit himself. If user-specified limit causes problems to
629 your script, blame the user.
631 Other significant changes:
633 * Added support for XZ_DEFAULTS environment variable. This variable
634 allows users to set default options for xz, e.g. default memory
635 usage limit or default compression level. Scripts that use xz
636 must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT
637 instead if they need a way to pass options to xz via an
638 environment variable.
640 * The compression settings associated with the preset levels
641 -0 ... -9 have been changed. --extreme was changed a little too.
642 It is now less likely to make compression worse, but with some
643 files the new --extreme may compress slightly worse than the old
646 * If a preset level (-0 ... -9) is specified after a custom filter
647 chain options have been used (e.g. --lzma2), the custom filter
648 chain will be forgotten. Earlier the preset options were
649 completely ignored after custom filter chain options had been
652 * xz will create sparse files when decompressing if the uncompressed
653 data contains long sequences of binary zeros. This is done even
654 when writing to standard output that is connected to a regular
655 file and certain additional conditions are met to make it safe.
657 * Support for "xz --list" was added. Combine with --verbose or
658 --verbose --verbose (-vv) for detailed output.
660 * I had hoped that liblzma API would have been stable after
661 4.999.9beta, but there have been a couple of changes in the
662 advanced features, which don't affect most applications:
664 - Index handling code was revised. If you were using the old
665 API, you will get a compiler error (so it's easy to notice).
667 - A subtle but important change was made to the Block handling
668 API. lzma_block.version has to be initialized even for
669 lzma_block_header_decode(). Code that doesn't do it will work
670 for now, but might break in the future, which makes this API
673 * The major soname has been bumped to 5.0.0. liblzma API and ABI
674 are now stable, so the need to recompile programs linking against
675 liblzma shouldn't arise soon.