| blob | b415254d88bf374c2d48b2a5474810942b45b248 |
2 XZ Utils Release Notes
3 ======================
5 5.3.4alpha (2022-11-15)
7 * All fixes from 5.2.7 and 5.2.8.
9 * liblzma:
11 - Minor improvements to the threaded decoder.
13 - Added CRC64 implementation that uses SSSE3, SSE4.1, and CLMUL
14 instructions on 32/64-bit x86 and E2K. On 32-bit x86 it's
15 not enabled unless --disable-assembler is used but then
16 the non-CLMUL code might be slower. Processor support is
17 detected at runtime so this is built by default on x86-64
18 and E2K. On these platforms, if compiler flags indicate
19 unconditional CLMUL support (-msse4.1 -mpclmul) then the
20 generic version is not built, making liblzma 8-9 KiB smaller
21 compared to having both versions included.
23 With extremely compressible files this can make decompression
24 up to twice as fast but with typical files 5 % improvement
25 is a more realistic expectation.
27 The CLMUL version is slower than the generic version with
28 tiny inputs (especially at 1-8 bytes per call, but up to
29 16 bytes). In normal use in xz this doesn't matter at all.
31 - Added an experimental ARM64 filter. This is *not* the final
32 version! Files created with this experimental version won't
33 be supported in the future versions! The filter design is
34 a compromise where improving one use case makes some other
35 cases worse.
37 - Added decompression support for the .lz (lzip) file format
38 version 0 and the original unextended version 1. See the
39 API docs of lzma_lzip_decoder() for details. Also
40 lzma_auto_decoder() supports .lz files.
42 - Building with --disable-threads --enable-small
43 is now thread-safe if the compiler supports
44 __attribute__((__constructor__))
46 * xz:
48 - Added support for OpenBSD's pledge(2) as a sandboxing method.
50 - Don't mention endianness for ARM and ARM-Thumb filters in
51 --long-help. The filters only work for little endian
52 instruction encoding but modern ARM processors using
53 big endian data access still use little endian
54 instruction encoding. So the help text was misleading.
55 In contrast, the PowerPC filter is only for big endian
56 32/64-bit PowerPC code. Little endian PowerPC would need
57 a separate filter.
59 - Added --experimental-arm64. This will be renamed once the
60 filter is finished. Files created with this experimental
61 filter will not be supported in the future!
63 - Added new fields to the output of xz --robot --info-memory.
65 - Added decompression support for the .lz (lzip) file format
66 version 0 and the original unextended version 1. It is
67 autodetected by default. See also the option --format on
68 the xz man page.
70 * Scripts now support the .lz format using xz.
72 * Build systems:
74 - New #defines in config.h: HAVE_ENCODER_ARM64,
75 HAVE_DECODER_ARM64, HAVE_LZIP_DECODER, HAVE_CPUID_H,
76 HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR, HAVE_USABLE_CLMUL
78 - New configure options: --disable-clmul-crc,
79 --disable-microlzma, --disable-lzip-decoder, and
80 'pledge' is now an option in --enable-sandbox (but
81 it's autodetected by default anyway).
83 - INSTALL was updated to document the new configure options.
85 - PACKAGERS now lists also --disable-microlzma and
86 --disable-lzip-decoder as configure options that must
87 not be used in builds for non-embedded use.
89 * Tests:
91 - Fix some of the tests so that they skip instead of fail if
92 certain features have been disabled with configure options.
93 It's still not perfect.
95 - Other improvements to tests.
97 * Updated translations: Croatian, Finnish, Hungarian, Polish,
98 Romanian, Spanish, Swedish, and Ukrainian.
101 5.3.3alpha (2022-08-22)
103 * All fixes from 5.2.6.
105 * liblzma:
107 - Fixed 32-bit build.
109 - Added threaded .xz decompressor lzma_stream_decoder_mt().
110 It can use multiple threads with .xz files that have multiple
111 Blocks with size information in Block Headers. The threaded
112 encoder in xz has always created such files.
114 Single-threaded encoder cannot store the size information in
115 Block Headers even if one used LZMA_FULL_FLUSH to create
116 multiple Blocks, so this threaded decoder cannot use multiple
117 threads with such files.
119 If there are multiple Streams (concatenated .xz files), one
120 Stream will be decompressed completely before starting the
121 next Stream.
123 - A new decoder flag LZMA_FAIL_FAST was added. It makes the
124 threaded decompressor report errors soon instead of first
125 flushing all pending data before the error location.
127 * xz:
129 - Using -T0 (--threads=0) will now use multi-threaded encoder
130 even on a single-core system. This is to ensure that output
131 from the same xz binary is identical on both single-core and
132 multi-core systems.
134 - A default soft memory usage limit is now used for compression
135 when -T0 is used and no explicit limit has been specified.
136 This soft limit is used to restrict the number of threads
137 but if the limit is exceeded with even one thread then xz
138 will continue with one thread using the multi-threaded
139 encoder and this limit is ignored. If the number of threads
140 is specified manually then no default limit will be used;
141 this affects only -T0.
143 This change helps on systems that have very many cores and
144 using all of them for xz makes no sense. Previously xz -T0
145 could run out of memory on such systems because it attempted
146 to reserve memory for too many threads.
148 This also helps with 32-bit builds which don't have a large
149 amount of address space that would be required for many
150 threads. The default limit is 1400 MiB on all 32-bit
151 platforms with -T0.
153 Now xz -T0 should just work. It might use too few threads
154 in some cases but at least it shouldn't easily run out of
155 memory. It's possible that this will be tweaked before 5.4.0.
157 - Changes to --memlimit-compress and --no-adjust:
159 In single-threaded mode, --memlimit-compress can make xz
160 scale down the LZMA2 dictionary size to meet the memory usage
161 limit. This obviously affects the compressed output. However,
162 if xz was in threaded mode, --memlimit-compress could make xz
163 reduce the number of threads but it wouldn't make xz switch
164 from multi-threaded mode to single-threaded mode or scale
165 down the LZMA2 dictionary size. This seemed illogical.
167 Now --memlimit-compress can make xz switch to single-threaded
168 mode if one thread in multi-threaded mode uses too much
169 memory. If memory usage is still too high, then the LZMA2
170 dictionary size can be scaled down too.
172 The option --no-adjust was also changed so that it no longer
173 prevents xz from scaling down the number of threads as that
174 doesn't affect compressed output (only performance). After
175 this commit --no-adjust only prevents adjustments that affect
176 compressed output, that is, with --no-adjust xz won't switch
177 from multithreaded mode to single-threaded mode and won't
178 scale down the LZMA2 dictionary size.
180 - Added a new option --memlimit-decompress-mt=LIMIT. This is
181 used to limit the number of decompressor threads (possibly
182 falling back to single-threaded mode) but it will never make
183 xz refuse to decompress a file. This has a system-specific
184 default value because without any limit xz could end up
185 allocating memory for the whole compressed input file, the
186 whole uncompressed output file, multiple thread-specific
187 decompressor instances and so on. Basically xz could
188 attempt to use an insane amount of memory even with fairly
189 common files.
191 The new option works together with the existing option
192 --memlimit-decompress=LIMIT. The old option sets a hard limit
193 that must not be exceeded (xz will refuse to decompress)
194 while the new option only restricts the number of threads.
195 If the limit set with --memlimit-mt-decompress is greater
196 than the limit set with --memlimit-compress, then the latter
197 value is used also for --memlimit-mt-decompress.
199 * Tests:
201 - Added a few more tests.
203 - Added tests/code_coverage.sh to create a code coverage report
204 of the tests.
206 * Build systems:
208 - Automake's parallel test harness is now used to make tests
209 finish faster.
211 - Added the CMake files to the distribution tarball. These were
212 supposed to be in 5.2.5 already.
214 - Added liblzma tests to the CMake build.
216 - Windows: Fix building of liblzma.dll with the included
217 Visual Studio project files.
220 5.3.2alpha (2021-10-28)
222 This release was made on short notice so that recent erofs-utils can
223 be built with LZMA support without needing a snapshot from xz.git.
224 Thus many pending things were not included, not even updated
225 translations (which would need to be updated for the new --list
226 strings anyway).
228 * All fixes from 5.2.5.
230 * xz:
232 - When copying metadata from the source file to the destination
233 file, don't try to set the group (GID) if it is already set
234 correctly. This avoids a failure on OpenBSD (and possibly on
235 a few other OSes) where files may get created so that their
236 group doesn't belong to the user, and fchown(2) can fail even
237 if it needs to do nothing.
239 - The --keep option now accepts symlinks, hardlinks, and
240 setuid, setgid, and sticky files. Previously this required
241 using --force.
243 - Split the long strings used in --list and --info-memory modes
244 to make them much easier for translators.
246 - If built with sandbox support and enabling the sandbox fails,
247 xz will now immediately exit with exit status of 1. Previously
248 it would only display a warning if -vv was used.
250 - Cap --memlimit-compress to 2000 MiB on MIPS32 because on
251 MIPS32 userspace processes are limited to 2 GiB of address
252 space.
254 * liblzma:
256 - Added lzma_microlzma_encoder() and lzma_microlzma_decoder().
257 The API is in lzma/container.h.
259 The MicroLZMA format is a raw LZMA stream (without end marker)
260 whose first byte (always 0x00) has been replaced with
261 bitwise-negation of the LZMA properties (lc/lp/pb). It was
262 created for use in EROFS but may be used in other contexts
263 as well where it is important to avoid wasting bytes for
264 stream headers or footers. The format is also supported by
265 XZ Embedded.
267 The MicroLZMA encoder API in liblzma can compress into a
268 fixed-sized output buffer so that as much data is compressed
269 as can be fit into the buffer while still creating a valid
270 MicroLZMA stream. This is needed for EROFS.
272 - Added fuzzing support.
274 - Support Intel Control-flow Enforcement Technology (CET) in
275 32-bit x86 assembly files.
277 - Visual Studio: Use non-standard _MSVC_LANG to detect C++
278 standard version in the lzma.h API header. It's used to
279 detect when "noexcept" can be used.
281 * Scripts:
283 - Fix exit status of xzdiff/xzcmp. Exit status could be 2 when
284 the correct value is 1.
286 - Fix exit status of xzgrep.
288 - Detect corrupt .bz2 files in xzgrep.
290 - Add zstd support to xzgrep and xzdiff/xzcmp.
292 - Fix less(1) version detection in xzless. It failed if the
293 version number from "less -V" contained a dot.
295 * Fix typos and technical issues in man pages.
297 * Build systems:
299 - Windows: Fix building of resource files when config.h isn't
300 used. CMake + Visual Studio can now build liblzma.dll.
302 - Various fixes to the CMake support. It might still need a few
303 more fixes even for liblzma-only builds.
306 5.3.1alpha (2018-04-29)
308 * All fixes from 5.2.4.
310 * Add lzma_file_info_decoder() into liblzma and use it in xz to
311 implement the --list feature.
313 * Capsicum sandbox support is enabled by default where available
314 (FreeBSD >= 10).
317 5.2.8 (2022-11-13)
319 * xz:
321 - If xz cannot remove an input file when it should, this
322 is now treated as a warning (exit status 2) instead of
323 an error (exit status 1). This matches GNU gzip and it
324 is more logical as at that point the output file has
325 already been successfully closed.
327 - Fix handling of .xz files with an unsupported check type.
328 Previously such printed a warning message but then xz
329 behaved as if an error had occurred (didn't decompress,
330 exit status 1). Now a warning is printed, decompression
331 is done anyway, and exit status is 2. This used to work
332 slightly before 5.0.0. In practice this bug matters only
333 if xz has been built with some check types disabled. As
334 instructed in PACKAGERS, such builds should be done in
335 special situations only.
337 - Fix "xz -dc --single-stream tests/files/good-0-empty.xz"
338 which failed with "Internal error (bug)". That is,
339 --single-stream was broken if the first .xz stream in
340 the input file didn't contain any uncompressed data.
342 - Fix displaying file sizes in the progress indicator when
343 working in passthru mode and there are multiple input files.
344 Just like "gzip -cdf", "xz -cdf" works like "cat" when the
345 input file isn't a supported compressed file format. In
346 this case the file size counters weren't reset between
347 files so with multiple input files the progress indicator
348 displayed an incorrect (too large) value.
350 * liblzma:
352 - API docs in lzma/container.h:
353 * Update the list of decoder flags in the decoder
354 function docs.
355 * Explain LZMA_CONCATENATED behavior with .lzma files
356 in lzma_auto_decoder() docs.
358 - OpenBSD: Use HW_NCPUONLINE to detect the number of
359 available hardware threads in lzma_physmem().
361 - Fix use of wrong macro to detect x86 SSE2 support.
362 __SSE2_MATH__ was used with GCC/Clang but the correct
363 one is __SSE2__. The first one means that SSE2 is used
364 for floating point math which is irrelevant here.
365 The affected SSE2 code isn't used on x86-64 so this affects
366 only 32-bit x86 builds that use -msse2 without -mfpmath=sse
367 (there is no runtime detection for SSE2). It improves LZMA
368 compression speed (not decompression).
370 - Fix the build with Intel C compiler 2021 (ICC, not ICX)
371 on Linux. It defines __GNUC__ to 10 but doesn't support
372 the __symver__ attribute introduced in GCC 10.
374 * Scripts: Ignore warnings from xz by using --quiet --no-warn.
375 This is needed if the input .xz files use an unsupported
376 check type.
378 * Translations:
380 - Updated Croatian and Turkish translations.
382 - One new translations wasn't included because it needed
383 technical fixes. It will be in upcoming 5.4.0. No new
384 translations will be added to the 5.2.x branch anymore.
386 - Renamed the French man page translation file from
387 fr_FR.po to fr.po and thus also its install directory
388 (like /usr/share/man/fr_FR -> .../fr).
390 - Man page translations for upcoming 5.4.0 are now handled
391 in the Translation Project.
393 * Update doc/faq.txt a little so it's less out-of-date.
396 5.2.7 (2022-09-30)
398 * liblzma:
400 - Made lzma_filters_copy() to never modify the destination
401 array if an error occurs. lzma_stream_encoder() and
402 lzma_stream_encoder_mt() already assumed this. Before this
403 change, if a tiny memory allocation in lzma_filters_copy()
404 failed it would lead to a crash (invalid free() or invalid
405 memory reads) in the cleanup paths of these two encoder
406 initialization functions.
408 - Added missing integer overflow check to lzma_index_append().
409 This affects xz --list and other applications that decode
410 the Index field from .xz files using lzma_index_decoder().
411 Normal decompression of .xz files doesn't call this code
412 and thus most applications using liblzma aren't affected
413 by this bug.
415 - Single-threaded .xz decoder (lzma_stream_decoder()): If
416 lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible
417 to use lzma_memlimit_set() to increase the limit and continue
418 decoding. This was supposed to work from the beginning
419 but there was a bug. With other decoders (.lzma or
420 threaded .xz decoder) this already worked correctly.
422 - Fixed accumulation of integrity check type statistics in
423 lzma_index_cat(). This bug made lzma_index_checks() return
424 only the type of the integrity check of the last Stream
425 when multiple lzma_indexes were concatenated. Most
426 applications don't use these APIs but in xz it made
427 xz --list not list all check types from concatenated .xz
428 files. In xz --list --verbose only the per-file "Check:"
429 lines were affected and in xz --robot --list only the "file"
430 line was affected.
432 - Added ABI compatibility with executables that were linked
433 against liblzma in RHEL/CentOS 7 or other liblzma builds
434 that had copied the problematic patch from RHEL/CentOS 7
435 (xz-5.2.2-compat-libs.patch). For the details, see the
436 comment at the top of src/liblzma/validate_map.sh.
438 WARNING: This uses __symver__ attribute with GCC >= 10.
439 In other cases the traditional __asm__(".symver ...")
440 is used. Using link-time optimization (LTO, -flto) with
441 GCC versions older than 10 can silently result in
442 broken liblzma.so.5 (incorrect symbol versions)! If you
443 want to use -flto with GCC, you must use GCC >= 10.
444 LTO with Clang seems to work even with the traditional
445 __asm__(".symver ...") method.
447 * xzgrep: Fixed compatibility with old shells that break if
448 comments inside command substitutions have apostrophes (').
449 This problem was introduced in 5.2.6.
451 * Build systems:
453 - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX
455 - Windows: Fixed liblzma.dll build with Visual Studio project
456 files. It broke in 5.2.6 due to a change that was made to
457 improve CMake support.
459 - Windows: Building liblzma with UNICODE defined should now
460 work.
462 - CMake files are now actually included in the release tarball.
463 They should have been in 5.2.5 already.
465 - Minor CMake fixes and improvements.
467 * Added a new translation: Turkish
470 5.2.6 (2022-08-12)
472 * xz:
474 - The --keep option now accepts symlinks, hardlinks, and
475 setuid, setgid, and sticky files. Previously this required
476 using --force.
478 - When copying metadata from the source file to the destination
479 file, don't try to set the group (GID) if it is already set
480 correctly. This avoids a failure on OpenBSD (and possibly on
481 a few other OSes) where files may get created so that their
482 group doesn't belong to the user, and fchown(2) can fail even
483 if it needs to do nothing.
485 - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on
486 MIPS32 because on MIPS32 userspace processes are limited
487 to 2 GiB of address space.
489 * liblzma:
491 - Fixed a missing error-check in the threaded encoder. If a
492 small memory allocation fails, a .xz file with an invalid
493 Index field would be created. Decompressing such a file would
494 produce the correct output but result in an error at the end.
495 Thus this is a "mild" data corruption bug. Note that while
496 a failed memory allocation can trigger the bug, it cannot
497 cause invalid memory access.
499 - The decoder for .lzma files now supports files that have
500 uncompressed size stored in the header and still use the
501 end of payload marker (end of stream marker) at the end
502 of the LZMA stream. Such files are rare but, according to
503 the documentation in LZMA SDK, they are valid.
504 doc/lzma-file-format.txt was updated too.
506 - Improved 32-bit x86 assembly files:
507 * Support Intel Control-flow Enforcement Technology (CET)
508 * Use non-executable stack on FreeBSD.
510 - Visual Studio: Use non-standard _MSVC_LANG to detect C++
511 standard version in the lzma.h API header. It's used to
512 detect when "noexcept" can be used.
514 * xzgrep:
516 - Fixed arbitrary command injection via a malicious filename
517 (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for
518 this was released to the public on 2022-04-07. A slight
519 robustness improvement has been made since then and, if
520 using GNU or *BSD grep, a new faster method is now used
521 that doesn't use the old sed-based construct at all. This
522 also fixes bad output with GNU grep >= 3.5 (2020-09-27)
523 when xzgrepping binary files.
525 This vulnerability was discovered by:
526 cleemy desu wayo working with Trend Micro Zero Day Initiative
528 - Fixed detection of corrupt .bz2 files.
530 - Improved error handling to fix exit status in some situations
531 and to fix handling of signals: in some situations a signal
532 didn't make xzgrep exit when it clearly should have. It's
533 possible that the signal handling still isn't quite perfect
534 but hopefully it's good enough.
536 - Documented exit statuses on the man page.
538 - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead
539 of the deprecated egrep and fgrep commands.
541 - Fixed parsing of the options -E, -F, -G, -P, and -X. The
542 problem occurred when multiple options were specied in
543 a single argument, for example,
545 echo foo | xzgrep -Fe foo
547 treated foo as a filename because -Fe wasn't correctly
548 split into -F -e.
550 - Added zstd support.
552 * xzdiff/xzcmp:
554 - Fixed wrong exit status. Exit status could be 2 when the
555 correct value is 1.
557 - Documented on the man page that exit status of 2 is used
558 for decompression errors.
560 - Added zstd support.
562 * xzless:
564 - Fix less(1) version detection. It failed if the version number
565 from "less -V" contained a dot.
567 * Translations:
569 - Added new translations: Catalan, Croatian, Esperanto,
570 Korean, Portuguese, Romanian, Serbian, Spanish, Swedish,
571 and Ukrainian
573 - Updated the Brazilian Portuguese translation.
575 - Added French man page translation. This and the existing
576 German translation aren't complete anymore because the
577 English man pages got a few updates and the translators
578 weren't reached so that they could update their work.
580 * Build systems:
582 - Windows: Fix building of resource files when config.h isn't
583 used. CMake + Visual Studio can now build liblzma.dll.
585 - Various fixes to the CMake support. Building static or shared
586 liblzma should work fine in most cases. In contrast, building
587 the command line tools with CMake is still clearly incomplete
588 and experimental and should be used for testing only.
591 5.2.5 (2020-03-17)
593 * liblzma:
595 - Fixed several C99/C11 conformance bugs. Now the code is clean
596 under gcc/clang -fsanitize=undefined. Some of these changes
597 might have a negative effect on performance with old GCC
598 versions or compilers other than GCC and Clang. The configure
599 option --enable-unsafe-type-punning can be used to (mostly)
600 restore the old behavior but it shouldn't normally be used.
602 - Improved API documentation of lzma_properties_decode().
604 - Added a very minor encoder speed optimization.
606 * xz:
608 - Fixed a crash in "xz -dcfv not_an_xz_file". All four options
609 were required to trigger it. The crash occurred in the
610 progress indicator code when xz was in passthru mode where
611 xz works like "cat".
613 - Fixed an integer overflow with 32-bit off_t. It could happen
614 when decompressing a file that has a long run of zero bytes
615 which xz would try to write as a sparse file. Since the build
616 system enables large file support by default, off_t is
617 normally 64-bit even on 32-bit systems.
619 - Fixes for --flush-timeout:
620 * Fix semi-busy-waiting.
621 * Avoid unneeded flushes when no new input has arrived
622 since the previous flush was completed.
624 - Added a special case for 32-bit xz: If --memlimit-compress is
625 used to specify a limit that exceeds 4020 MiB, the limit will
626 be set to 4020 MiB. The values "0" and "max" aren't affected
627 by this and neither is decompression. This hack can be
628 helpful when a 32-bit xz has access to 4 GiB address space
629 but the specified memlimit exceeds 4 GiB. This can happen
630 e.g. with some scripts.
632 - Capsicum sandbox is now enabled by default where available
633 (FreeBSD >= 10). The sandbox debug messages (xz -vv) were
634 removed since they seemed to be more annoying than useful.
636 - DOS build now requires DJGPP 2.05 instead of 2.04beta.
637 A workaround for a locale problem with DJGPP 2.05 was added.
639 * xzgrep and other scripts:
641 - Added a configure option --enable-path-for-scripts=PREFIX.
642 It is disabled by default except on Solaris where the default
643 is /usr/xpg4/bin. See INSTALL for details.
645 - Added a workaround for a POSIX shell detection problem on
646 Solaris.
648 * Build systems:
650 - Added preliminary build instructions for z/OS. See INSTALL
651 section 1.2.9.
653 - Experimental CMake support was added. It should work to build
654 static liblzma on a few operating systems. It may or may not
655 work to build shared liblzma. On some platforms it can build
656 xz and xzdec too but those are only for testing. See the
657 comment in the beginning of CMakeLists.txt for details.
659 - Visual Studio project files were updated.
660 WindowsTargetPlatformVersion was removed from VS2017 files
661 and set to "10.0" in the added VS2019 files. In the future
662 the VS project files will be removed when CMake support is
663 good enough.
665 - New #defines in config.h: HAVE___BUILTIN_ASSUME_ALIGNED,
666 HAVE___BUILTIN_BSWAPXX, and TUKLIB_USE_UNSAFE_TYPE_PUNNING.
668 - autogen.sh has a new optional dependency on po4a and a new
669 option --no-po4a to skip that step. This matters only if one
670 wants to remake the build files. po4a is used to update the
671 translated man pages but as long as the man pages haven't
672 been modified, there's nothing to update and one can use
673 --no-po4a to avoid the dependency on po4a.
675 * Translations:
677 - XZ Utils translations are now handled by the Translation
678 Project: https://translationproject.org/domain/xz.html
680 - All man pages are now included in German too.
682 - New xz translations: Brazilian Portuguese, Finnish,
683 Hungarian, Chinese (simplified), Chinese (traditional),
684 and Danish (partial translation)
686 - Updated xz translations: French, German, Italian, and Polish
688 - Unfortunately a few new xz translations weren't included due
689 to technical problems like too long lines in --help output or
690 misaligned column headings in tables. In the future, many of
691 these strings will be split and e.g. the table column
692 alignment will be handled in software. This should make the
693 strings easier to translate.
696 5.2.4 (2018-04-29)
698 * liblzma:
700 - Allow 0 as memory usage limit instead of returning
701 LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified,
702 which effectively is the same as 0.
704 - Use "noexcept" keyword instead of "throw()" in the public
705 headers when a C++11 (or newer standard) compiler is used.
707 - Added a portability fix for recent Intel C Compilers.
709 - Microsoft Visual Studio build files have been moved under
710 windows/vs2013 and windows/vs2017.
712 * xz:
714 - Fix "xz --list --robot missing_or_bad_file.xz" which would
715 try to print an uninitialized string and thus produce garbage
716 output. Since the exit status is non-zero, most uses of such
717 a command won't try to interpret the garbage output.
719 - "xz --list foo.xz" could print "Internal error (bug)" in a
720 corner case where a specific memory usage limit had been set.
723 5.2.3 (2016-12-30)
725 * xz:
727 - Always close a file before trying to delete it to avoid
728 problems on some operating system and file system combinations.
730 - Fixed copying of file timestamps on Windows.
732 - Added experimental (disabled by default) sandbox support using
733 Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL.
735 * C99/C11 conformance fixes to liblzma. The issues affected at least
736 some builds using link-time optimizations.
738 * Fixed bugs in the rarely-used function lzma_index_dup().
740 * Use of external SHA-256 code is now disabled by default.
741 It can still be enabled by passing --enable-external-sha256
742 to configure. The reasons to disable it by default (see INSTALL
743 for more details):
745 - Some OS-specific SHA-256 implementations conflict with
746 OpenSSL and cause problems in programs that link against both
747 liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0
748 are affected.
750 - The internal SHA-256 is faster than the SHA-256 code in
751 some operating systems.
753 * Changed CPU core count detection to use sched_getaffinity() on
754 GNU/Linux and GNU/kFreeBSD.
756 * Fixes to the build-system and xz to make xz buildable even when
757 encoders, decoders, or threading have been disabled from libilzma
758 using configure options. These fixes added two new #defines to
759 config.h: HAVE_ENCODERS and HAVE_DECODERS.
762 5.2.2 (2015-09-29)
764 * Fixed bugs in QNX-specific code.
766 * Omitted the use of pipe2() even if it is available to avoid
767 portability issues with some old Linux and glibc combinations.
769 * Updated German translation.
771 * Added project files to build static and shared liblzma (not the
772 whole XZ Utils) with Visual Studio 2013 update 2 or later.
774 * Documented that threaded decompression hasn't been implemented
775 yet. A 5.2.0 NEWS entry describing multi-threading support had
776 incorrectly said "decompression" when it should have said
777 "compression".
780 5.2.1 (2015-02-26)
782 * Fixed a compression-ratio regression in fast mode of LZMA1 and
783 LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases.
785 * Fixed a portability problem in xz that affected at least OpenBSD.
787 * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs
788 from most other mktemp implementations.
790 * Changed CPU core count detection to use cpuset_getaffinity() on
791 FreeBSD.
794 5.2.0 (2014-12-21)
796 Since 5.1.4beta:
798 * All fixes from 5.0.8
800 * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset
801 was used.
803 * xzdiff: If mktemp isn't installed, mkdir will be used as
804 a fallback to create a temporary directory. Installing mktemp
805 is still recommended.
807 * Updated French, German, Italian, Polish, and Vietnamese
808 translations.
810 Summary of fixes and new features added in the 5.1.x development
811 releases:
813 * liblzma:
815 - Added support for multi-threaded compression. See the
816 lzma_mt structure, lzma_stream_encoder_mt(), and
817 lzma_stream_encoder_mt_memusage() in <lzma/container.h>,
818 lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads()
819 in <lzma/hardware.h> for details.
821 - Made the uses of lzma_allocator const correct.
823 - Added lzma_block_uncomp_encode() to create uncompressed
824 .xz Blocks using LZMA2 uncompressed chunks.
826 - Added support for LZMA_IGNORE_CHECK.
828 - A few speed optimizations were made.
830 - Added support for symbol versioning. It is enabled by default
831 on GNU/Linux, other GNU-based systems, and FreeBSD.
833 - liblzma (not the whole XZ Utils) should now be buildable
834 with MSVC 2013 update 2 or later using windows/config.h.
836 * xz:
838 - Fixed a race condition in the signal handling. It was
839 possible that e.g. the first SIGINT didn't make xz exit
840 if reading or writing blocked and one had bad luck. The fix
841 is non-trivial, so as of writing it is unknown if it will be
842 backported to the v5.0 branch.
844 - Multi-threaded compression can be enabled with the
845 --threads (-T) option.
846 [Fixed: This originally said "decompression".]
848 - New command line options in xz: --single-stream,
849 --block-size=SIZE, --block-list=SIZES,
850 --flush-timeout=TIMEOUT, and --ignore-check.
852 - xz -lvv now shows the minimum xz version that is required to
853 decompress the file. Currently it is 5.0.0 for all supported
854 .xz files except files with empty LZMA2 streams require 5.0.2.
856 * xzdiff and xzgrep now support .lzo files if lzop is installed.
857 The .tzo suffix is also recognized as a shorthand for .tar.lzo.
860 5.1.4beta (2014-09-14)
862 * All fixes from 5.0.6
864 * liblzma: Fixed the use of presets in threaded encoder
865 initialization.
867 * xz --block-list and --block-size can now be used together
868 in single-threaded mode. Previously the combination only
869 worked in multi-threaded mode.
871 * Added support for LZMA_IGNORE_CHECK to liblzma and made it
872 available in xz as --ignore-check.
874 * liblzma speed optimizations:
876 - Initialization of a new LZMA1 or LZMA2 encoder has been
877 optimized. (The speed of reinitializing an already-allocated
878 encoder isn't affected.) This helps when compressing many
879 small buffers with lzma_stream_buffer_encode() and other
880 similar situations where an already-allocated encoder state
881 isn't reused. This speed-up is visible in xz too if one
882 compresses many small files one at a time instead running xz
883 once and giving all files as command-line arguments.
885 - Buffer comparisons are now much faster when unaligned access
886 is allowed (configured with --enable-unaligned-access). This
887 speeds up encoding significantly. There is arch-specific code
888 for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best
889 results and there's no run-time CPU detection for now).
890 For other archs there is only generic code which probably
891 isn't as optimal as arch-specific solutions could be.
893 - A few speed optimizations were made to the SHA-256 code.
894 (Note that the builtin SHA-256 code isn't used on all
895 operating systems.)
897 * liblzma can now be built with MSVC 2013 update 2 or later
898 using windows/config.h.
900 * Vietnamese translation was added.
903 5.1.3alpha (2013-10-26)
905 * All fixes from 5.0.5
907 * liblzma:
909 - Fixed a deadlock in the threaded encoder.
911 - Made the uses of lzma_allocator const correct.
913 - Added lzma_block_uncomp_encode() to create uncompressed
914 .xz Blocks using LZMA2 uncompressed chunks.
916 - Added support for native threads on Windows and the ability
917 to detect the number of CPU cores.
919 * xz:
921 - Fixed a race condition in the signal handling. It was
922 possible that e.g. the first SIGINT didn't make xz exit
923 if reading or writing blocked and one had bad luck. The fix
924 is non-trivial, so as of writing it is unknown if it will be
925 backported to the v5.0 branch.
927 - Made the progress indicator work correctly in threaded mode.
929 - Threaded encoder now works together with --block-list=SIZES.
931 - Added preliminary support for --flush-timeout=TIMEOUT.
932 It can be useful for (somewhat) real-time streaming. For
933 now the decompression side has to be done with something
934 else than the xz tool due to how xz does buffering, but this
935 should be fixed.
938 5.1.2alpha (2012-07-04)
940 * All fixes from 5.0.3 and 5.0.4
942 * liblzma:
944 - Fixed a deadlock and an invalid free() in the threaded encoder.
946 - Added support for symbol versioning. It is enabled by default
947 on GNU/Linux, other GNU-based systems, and FreeBSD.
949 - Use SHA-256 implementation from the operating system if one is
950 available in libc, libmd, or libutil. liblzma won't use e.g.
951 OpenSSL or libgcrypt to avoid introducing new dependencies.
953 - Fixed liblzma.pc for static linking.
955 - Fixed a few portability bugs.
957 * xz --decompress --single-stream now fixes the input position after
958 successful decompression. Now the following works:
960 echo foo | xz > foo.xz
961 echo bar | xz >> foo.xz
962 ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz
964 Note that it doesn't work if the input is not seekable
965 or if there is Stream Padding between the concatenated
966 .xz Streams.
968 * xz -lvv now shows the minimum xz version that is required to
969 decompress the file. Currently it is 5.0.0 for all supported .xz
970 files except files with empty LZMA2 streams require 5.0.2.
972 * Added an *incomplete* implementation of --block-list=SIZES to xz.
973 It only works correctly in single-threaded mode and when
974 --block-size isn't used at the same time. --block-list allows
975 specifying the sizes of Blocks which can be useful e.g. when
976 creating files for random-access reading.
979 5.1.1alpha (2011-04-12)
981 * All fixes from 5.0.2
983 * liblzma fixes that will also be included in 5.0.3:
985 - A memory leak was fixed.
987 - lzma_stream_buffer_encode() no longer creates an empty .xz
988 Block if encoding an empty buffer. Such an empty Block with
989 LZMA2 data would trigger a bug in 5.0.1 and older (see the
990 first bullet point in 5.0.2 notes). When releasing 5.0.2,
991 I thought that no encoder creates this kind of files but
992 I was wrong.
994 - Validate function arguments better in a few functions. Most
995 importantly, specifying an unsupported integrity check to
996 lzma_stream_buffer_encode() no longer creates a corrupt .xz
997 file. Probably no application tries to do that, so this
998 shouldn't be a big problem in practice.
1000 - Document that lzma_block_buffer_encode(),
1001 lzma_easy_buffer_encode(), lzma_stream_encoder(), and
1002 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
1004 - The return values of the _memusage() functions are now
1005 documented better.
1007 * Support for multithreaded compression was added using the simplest
1008 method, which splits the input data into blocks and compresses
1009 them independently. Other methods will be added in the future.
1010 The current method has room for improvement, e.g. it is possible
1011 to reduce the memory usage.
1013 * Added the options --single-stream and --block-size=SIZE to xz.
1015 * xzdiff and xzgrep now support .lzo files if lzop is installed.
1016 The .tzo suffix is also recognized as a shorthand for .tar.lzo.
1018 * Support for short 8.3 filenames under DOS was added to xz. It is
1019 experimental and may change before it gets into a stable release.
1022 5.0.8 (2014-12-21)
1024 * Fixed an old bug in xzgrep that affected OpenBSD and probably
1025 a few other operating systems too.
1027 * Updated French and German translations.
1029 * Added support for detecting the amount of RAM on AmigaOS/AROS.
1031 * Minor build system updates.
1034 5.0.7 (2014-09-20)
1036 * Fix regressions introduced in 5.0.6:
1038 - Fix building with non-GNU make.
1040 - Fix invalid Libs.private value in liblzma.pc which broke
1041 static linking against liblzma if the linker flags were
1042 taken from pkg-config.
1045 5.0.6 (2014-09-14)
1047 * xzgrep now exits with status 0 if at least one file matched.
1049 * A few minor portability and build system fixes
1052 5.0.5 (2013-06-30)
1054 * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing
1055 .lzma files that have less common settings in the headers
1056 (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed
1057 size greater than 256 GiB). The limitations existed to avoid false
1058 positives when detecting .lzma files. The lc + lp <= 4 limitation
1059 still remains since liblzma's LZMA decoder has that limitation.
1061 NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT
1062 affected by this change. They still consider uncommon .lzma headers
1063 as not being in the .lzma format. Changing this would give way too
1064 many false positives.
1066 * xz:
1068 - Interaction of preset and custom filter chain options was
1069 made less illogical. This affects only certain less typical
1070 uses cases so few people are expected to notice this change.
1072 Now when a custom filter chain option (e.g. --lzma2) is
1073 specified, all preset options (-0 ... -9, -e) earlier are on
1074 the command line are completely forgotten. Similarly, when
1075 a preset option is specified, all custom filter chain options
1076 earlier on the command line are completely forgotten.
1078 Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e"
1079 which is equivalent to "xz -6e". Earlier -e didn't put xz back
1080 into preset mode and thus the example command was equivalent
1081 to "xz --lzma2=preset=5".
1083 Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to
1084 "xz -7". Earlier a custom filter chain option didn't make
1085 xz forget the -e option so the example was equivalent to
1086 "xz -7e".
1088 - Fixes and improvements to error handling.
1090 - Various fixes to the man page.
1092 * xzless: Fixed to work with "less" versions 448 and later.
1094 * xzgrep: Made -h an alias for --no-filename.
1096 * Include the previously missing debug/translation.bash which can
1097 be useful for translators.
1099 * Include a build script for Mac OS X. This has been in the Git
1100 repository since 2010 but due to a mistake in Makefile.am the
1101 script hasn't been included in a release tarball before.
1104 5.0.4 (2012-06-22)
1106 * liblzma:
1108 - Fix lzma_index_init(). It could crash if memory allocation
1109 failed.
1111 - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ
1112 filter is used and the application only provides exactly as
1113 much output space as is the uncompressed size of the file.
1115 - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't
1116 check if the last call to lzma_code() really returned
1117 LZMA_STREAM_END, which made the program think that truncated
1118 files are valid.
1120 - New example programs in doc/examples (old programs are now in
1121 doc/examples_old). These have more comments and more detailed
1122 error handling.
1124 * Fix "xz -lvv foo.xz". It could crash on some corrupted files.
1126 * Fix output of "xz --robot -lv" and "xz --robot -lvv" which
1127 incorrectly printed the filename also in the "foo (x/x)" format.
1129 * Fix exit status of "xzdiff foo.xz bar.xz".
1131 * Fix exit status of "xzgrep foo binary_file".
1133 * Fix portability to EBCDIC systems.
1135 * Fix a configure issue on AIX with the XL C compiler. See INSTALL
1136 for details.
1138 * Update French, German, Italian, and Polish translations.
1141 5.0.3 (2011-05-21)
1143 * liblzma fixes:
1145 - A memory leak was fixed.
1147 - lzma_stream_buffer_encode() no longer creates an empty .xz
1148 Block if encoding an empty buffer. Such an empty Block with
1149 LZMA2 data would trigger a bug in 5.0.1 and older (see the
1150 first bullet point in 5.0.2 notes). When releasing 5.0.2,
1151 I thought that no encoder creates this kind of files but
1152 I was wrong.
1154 - Validate function arguments better in a few functions. Most
1155 importantly, specifying an unsupported integrity check to
1156 lzma_stream_buffer_encode() no longer creates a corrupt .xz
1157 file. Probably no application tries to do that, so this
1158 shouldn't be a big problem in practice.
1160 - Document that lzma_block_buffer_encode(),
1161 lzma_easy_buffer_encode(), lzma_stream_encoder(), and
1162 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK.
1164 - The return values of the _memusage() functions are now
1165 documented better.
1167 * Fix command name detection in xzgrep. xzegrep and xzfgrep now
1168 correctly use egrep and fgrep instead of grep.
1170 * French translation was added.
1173 5.0.2 (2011-04-01)
1175 * LZMA2 decompressor now correctly accepts LZMA2 streams with no
1176 uncompressed data. Previously it considered them corrupt. The
1177 bug can affect applications that use raw LZMA2 streams. It is
1178 very unlikely to affect .xz files because no compressor creates
1179 .xz files with empty LZMA2 streams. (Empty .xz files are a
1180 different thing than empty LZMA2 streams.)
1182 * "xz --suffix=.foo filename.foo" now refuses to compress the
1183 file due to it already having the suffix .foo. It was already
1184 documented on the man page, but the code lacked the test.
1186 * "xzgrep -l foo bar.xz" works now.
1188 * Polish translation was added.
1191 5.0.1 (2011-01-29)
1193 * xz --force now (de)compresses files that have setuid, setgid,
1194 or sticky bit set and files that have multiple hard links.
1195 The man page had it documented this way already, but the code
1196 had a bug.
1198 * gzip and bzip2 support in xzdiff was fixed.
1200 * Portability fixes
1202 * Minor fix to Czech translation
1205 5.0.0 (2010-10-23)
1207 Only the most important changes compared to 4.999.9beta are listed
1208 here. One change is especially important:
1210 * The memory usage limit is now disabled by default. Some scripts
1211 written before this change may have used --memory=max on xz command
1212 line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED
1213 NOW, because they interfere with user's ability to set the memory
1214 usage limit himself. If user-specified limit causes problems to
1215 your script, blame the user.
1217 Other significant changes:
1219 * Added support for XZ_DEFAULTS environment variable. This variable
1220 allows users to set default options for xz, e.g. default memory
1221 usage limit or default compression level. Scripts that use xz
1222 must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT
1223 instead if they need a way to pass options to xz via an
1224 environment variable.
1226 * The compression settings associated with the preset levels
1227 -0 ... -9 have been changed. --extreme was changed a little too.
1228 It is now less likely to make compression worse, but with some
1229 files the new --extreme may compress slightly worse than the old
1230 --extreme.
1232 * If a preset level (-0 ... -9) is specified after a custom filter
1233 chain options have been used (e.g. --lzma2), the custom filter
1234 chain will be forgotten. Earlier the preset options were
1235 completely ignored after custom filter chain options had been
1236 seen.
1238 * xz will create sparse files when decompressing if the uncompressed
1239 data contains long sequences of binary zeros. This is done even
1240 when writing to standard output that is connected to a regular
1241 file and certain additional conditions are met to make it safe.
1243 * Support for "xz --list" was added. Combine with --verbose or
1244 --verbose --verbose (-vv) for detailed output.
1246 * I had hoped that liblzma API would have been stable after
1247 4.999.9beta, but there have been a couple of changes in the
1248 advanced features, which don't affect most applications:
1250 - Index handling code was revised. If you were using the old
1251 API, you will get a compiler error (so it's easy to notice).
1253 - A subtle but important change was made to the Block handling
1254 API. lzma_block.version has to be initialized even for
1255 lzma_block_header_decode(). Code that doesn't do it will work
1256 for now, but might break in the future, which makes this API
1257 change easy to miss.
1259 * The major soname has been bumped to 5.0.0. liblzma API and ABI
1260 are now stable, so the need to recompile programs linking against
1261 liblzma shouldn't arise soon.