Spaces:
Running
Running
| XZ Utils Release Notes | |
| ====================== | |
| 5.4.5 (2023-11-01) | |
| * liblzma: | |
| - Use __attribute__((__no_sanitize_address__)) to avoid address | |
| sanitization with CRC64 CLMUL. It uses 16-byte-aligned reads | |
| which can extend past the bounds of the input buffer and | |
| inherently trigger address sanitization errors. This isn't | |
| a bug. | |
| - Fixed an assertion failure that could be triggered by a large | |
| unpadded_size argument. It was verified that there was no | |
| other bug than the assertion failure. | |
| - Fixed a bug that prevented building with Windows Vista | |
| threading when __attribute__((__constructor__)) is not | |
| supported. | |
| * xz now properly handles special files such as "con" or "nul" on | |
| Windows. Before this fix, the following wrote "foo" to the | |
| console and deleted the input file "con_xz": | |
| echo foo | xz > con_xz | |
| xz --suffix=_xz --decompress con_xz | |
| * Build systems: | |
| - Allow builds with Windows win95 threading and small mode when | |
| __attribute__((__constructor__)) is supported. | |
| - Added a new line to liblzma.pc for MSYS2 (Windows): | |
| Cflags.private: -DLZMA_API_STATIC | |
| When compiling code that will link against static liblzma, | |
| the LZMA_API_STATIC macro needs to be defined on Windows. | |
| - CMake specific changes: | |
| * Fixed a bug that allowed CLOCK_MONOTONIC to be used even | |
| if the check for it failed. | |
| * Fixed a bug where configuring CMake multiple times | |
| resulted in HAVE_CLOCK_GETTIME and HAVE_CLOCK_MONOTONIC | |
| not being set. | |
| * Fixed the build with MinGW-w64-based Clang/LLVM 17. | |
| llvm-windres now has more accurate GNU windres emulation | |
| so the GNU windres workaround from 5.4.1 is needed with | |
| llvm-windres version 17 too. | |
| * The import library on Windows is now properly named | |
| "liblzma.dll.a" instead of "libliblzma.dll.a" | |
| * Fixed a bug causing the Ninja Generator to fail on | |
| UNIX-like systems. This bug was introduced in 5.4.0. | |
| * Added a new option to disable CLMUL CRC64. | |
| * A module-definition (.def) file is now created when | |
| building liblzma.dll with MinGW-w64. | |
| * The pkg-config liblzma.pc file is now installed on all | |
| builds except when using MSVC on Windows. | |
| * Added large file support by default for platforms that | |
| need it to handle files larger than 2 GiB. This includes | |
| MinGW-w64, even 64-bit builds. | |
| * Small fixes and improvements to the tests. | |
| * Updated translations: Chinese (simplified) and Esperanto. | |
| 5.4.4 (2023-08-02) | |
| * liblzma and xzdec can now build against WASI SDK when threading | |
| support is disabled. xz and tests don't build yet. | |
| * CMake: | |
| - Fixed a bug preventing other projects from including liblzma | |
| multiple times using find_package(). | |
| - Don't create broken symlinks in Cygwin and MSYS2 unless | |
| supported by the environment. This prevented building for the | |
| default MSYS2 environment. The problem was introduced in | |
| xz 5.4.0. | |
| * Documentation: | |
| - Small improvements to man pages. | |
| - Small improvements and typo fixes for liblzma API | |
| documentation. | |
| * Tests: | |
| - Added a new section to INSTALL to describe basic test usage | |
| and address recent questions about building the tests when | |
| cross compiling. | |
| - Small fixes and improvements to the tests. | |
| * Translations: | |
| - Fixed a mistake that caused one of the error messages to not | |
| be translated. This only affected versions 5.4.2 and 5.4.3. | |
| - Updated the Chinese (simplified), Croatian, Esperanto, German, | |
| Korean, Polish, Romanian, Spanish, Swedish, Ukrainian, and | |
| Vietnamese translations. | |
| - Updated the German, Korean, Romanian, and Ukrainian man page | |
| translations. | |
| 5.4.3 (2023-05-04) | |
| * All fixes from 5.2.12 | |
| * Features in the CMake build can now be disabled as CMake cache | |
| variables, similar to the Autotools build. | |
| * Minor update to the Croatian translation. | |
| 5.4.2 (2023-03-18) | |
| * All fixes from 5.2.11 that were not included in 5.4.1. | |
| * If xz is built with support for the Capsicum sandbox but running | |
| in an environment that doesn't support Capsicum, xz now runs | |
| normally without sandboxing instead of exiting with an error. | |
| * liblzma: | |
| - Documentation was updated to improve the style, consistency, | |
| and completeness of the liblzma API headers. | |
| - The Doxygen-generated HTML documentation for the liblzma API | |
| header files is now included in the source release and is | |
| installed as part of "make install". All JavaScript is | |
| removed to simplify license compliance and to reduce the | |
| install size. | |
| - Fixed a minor bug in lzma_str_from_filters() that produced | |
| too many filters in the output string instead of reporting | |
| an error if the input array had more than four filters. This | |
| bug did not affect xz. | |
| * Build systems: | |
| - autogen.sh now invokes the doxygen tool via the new wrapper | |
| script doxygen/update-doxygen, unless the command line option | |
| --no-doxygen is used. | |
| - Added microlzma_encoder.c and microlzma_decoder.c to the | |
| VS project files for Windows and to the CMake build. These | |
| should have been included in 5.3.2alpha. | |
| * Tests: | |
| - Added a test to the CMake build that was forgotten in the | |
| previous release. | |
| - Added and refactored a few tests. | |
| * Translations: | |
| - Updated the Brazilian Portuguese translation. | |
| - Added Brazilian Portuguese man page translation. | |
| 5.4.1 (2023-01-11) | |
| * liblzma: | |
| - Fixed the return value of lzma_microlzma_encoder() if the | |
| LZMA options lc/lp/pb are invalid. Invalid lc/lp/pb options | |
| made the function return LZMA_STREAM_END without encoding | |
| anything instead of returning LZMA_OPTIONS_ERROR. | |
| - Windows / Visual Studio: Workaround a possible compiler bug | |
| when targeting 32-bit x86 and compiling the CLMUL version of | |
| the CRC64 code. The CLMUL code isn't enabled by the Windows | |
| project files but it is in the CMake-based builds. | |
| * Build systems: | |
| - Windows-specific CMake changes: | |
| * Don't try to enable CLMUL CRC64 code if _mm_set_epi64x() | |
| isn't available. This fixes CMake-based build with Visual | |
| Studio 2013. | |
| * Created a workaround for a build failure with windres | |
| from GNU binutils. It is used only when the C compiler | |
| is GCC (not Clang). The workaround is incompatible | |
| with llvm-windres, resulting in "XZx20Utils" instead | |
| of "XZ Utils" in the resource file, but without the | |
| workaround llvm-windres works correctly. See the | |
| comment in CMakeLists.txt for details. | |
| * Included the resource files in the xz and xzdec build | |
| rules. Building the command line tools is still | |
| experimental but possible with MinGW-w64. | |
| - Visual Studio: Added stream_decoder_mt.c to the project | |
| files. Now the threaded decompressor lzma_stream_decoder_mt() | |
| gets built. CMake-based build wasn't affected. | |
| - Updated windows/INSTALL-MSVC.txt to mention that CMake-based | |
| build is now the preferred method with Visual Studio. The | |
| project files will probably be removed after 5.4.x releases. | |
| - Changes to #defines in config.h: | |
| * HAVE_DECL_CLOCK_MONOTONIC was replaced by | |
| HAVE_CLOCK_MONOTONIC. The old macro was always defined | |
| in configure-generated config.h to either 0 or 1. The | |
| new macro is defined (to 1) only if the declaration of | |
| CLOCK_MONOTONIC is available. This matches the way most | |
| other config.h macros work and makes things simpler with | |
| other build systems. | |
| * HAVE_DECL_PROGRAM_INVOCATION_NAME was replaced by | |
| HAVE_PROGRAM_INVOCATION_NAME for the same reason. | |
| * Tests: | |
| - Fixed test script compatibility with ancient /bin/sh | |
| versions. Now the five test_compress_* tests should | |
| no longer fail on Solaris 10. | |
| - Added and refactored a few tests. | |
| * Translations: | |
| - Updated the Catalan and Esperanto translations. | |
| - Added Korean and Ukrainian man page translations. | |
| 5.4.0 (2022-12-13) | |
| This bumps the minor version of liblzma because new features were | |
| added. The API and ABI are still backward compatible with liblzma | |
| 5.2.x and 5.0.x. | |
| Since 5.3.5beta: | |
| * All fixes from 5.2.10. | |
| * The ARM64 filter is now stable. The xz option is now --arm64. | |
| Decompression requires XZ Utils 5.4.0. In the future the ARM64 | |
| filter will be supported by XZ for Java, XZ Embedded (including | |
| the version in Linux), LZMA SDK, and 7-Zip. | |
| * Translations: | |
| - Updated Catalan, Croatian, German, Romanian, and Turkish | |
| translations. | |
| - Updated German man page translations. | |
| - Added Romanian man page translations. | |
| Summary of new features added in the 5.3.x development releases: | |
| * liblzma: | |
| - Added threaded .xz decompressor lzma_stream_decoder_mt(). | |
| It can use multiple threads with .xz files that have multiple | |
| Blocks with size information in Block Headers. The threaded | |
| encoder in xz has always created such files. | |
| Single-threaded encoder cannot store the size information in | |
| Block Headers even if one used LZMA_FULL_FLUSH to create | |
| multiple Blocks, so this threaded decoder cannot use multiple | |
| threads with such files. | |
| If there are multiple Streams (concatenated .xz files), one | |
| Stream will be decompressed completely before starting the | |
| next Stream. | |
| - A new decoder flag LZMA_FAIL_FAST was added. It makes the | |
| threaded decompressor report errors soon instead of first | |
| flushing all pending data before the error location. | |
| - New Filter IDs: | |
| * LZMA_FILTER_ARM64 is for ARM64 binaries. | |
| * LZMA_FILTER_LZMA1EXT is for raw LZMA1 streams that don't | |
| necessarily use the end marker. | |
| - Added lzma_str_to_filters(), lzma_str_from_filters(), and | |
| lzma_str_list_filters() to convert a preset or a filter chain | |
| string to a lzma_filter[] and vice versa. These should make | |
| it easier to write applications that allow users to specify | |
| custom compression options. | |
| - Added lzma_filters_free() which can be convenient for freeing | |
| the filter options in a filter chain (an array of lzma_filter | |
| structures). | |
| - lzma_file_info_decoder() to makes it a little easier to get | |
| the Index field from .xz files. This helps in getting the | |
| uncompressed file size but an easy-to-use random access | |
| API is still missing which has existed in XZ for Java for | |
| a long time. | |
| - Added lzma_microlzma_encoder() and lzma_microlzma_decoder(). | |
| It is used by erofs-utils and may be used by others too. | |
| The MicroLZMA format is a raw LZMA stream (without end marker) | |
| whose first byte (always 0x00) has been replaced with | |
| bitwise-negation of the LZMA properties (lc/lp/pb). It was | |
| created for use in EROFS but may be used in other contexts | |
| as well where it is important to avoid wasting bytes for | |
| stream headers or footers. The format is also supported by | |
| XZ Embedded (the XZ Embedded version in Linux got MicroLZMA | |
| support in Linux 5.16). | |
| The MicroLZMA encoder API in liblzma can compress into a | |
| fixed-sized output buffer so that as much data is compressed | |
| as can be fit into the buffer while still creating a valid | |
| MicroLZMA stream. This is needed for EROFS. | |
| - Added lzma_lzip_decoder() to decompress the .lz (lzip) file | |
| format version 0 and the original unextended version 1 files. | |
| Also lzma_auto_decoder() supports .lz files. | |
| - lzma_filters_update() can now be used with the multi-threaded | |
| encoder (lzma_stream_encoder_mt()) to change the filter chain | |
| after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH. | |
| - In lzma_options_lzma, allow nice_len = 2 and 3 with the match | |
| finders that require at least 3 or 4. Now it is internally | |
| rounded up if needed. | |
| - CLMUL-based CRC64 on x86-64 and E2K with runtime processor | |
| detection. On 32-bit x86 it currently isn't available unless | |
| --disable-assembler is used which can make the non-CLMUL | |
| CRC64 slower; this might be fixed in the future. | |
| - Building with --disable-threads --enable-small | |
| is now thread-safe if the compiler supports | |
| __attribute__((__constructor__)). | |
| * xz: | |
| - Using -T0 (--threads=0) will now use multi-threaded encoder | |
| even on a single-core system. This is to ensure that output | |
| from the same xz binary is identical on both single-core and | |
| multi-core systems. | |
| - --threads=+1 or -T+1 is now a way to put xz into | |
| multi-threaded mode while using only one worker thread. | |
| The + is ignored if the number is not 1. | |
| - A default soft memory usage limit is now used for compression | |
| when -T0 is used and no explicit limit has been specified. | |
| This soft limit is used to restrict the number of threads | |
| but if the limit is exceeded with even one thread then xz | |
| will continue with one thread using the multi-threaded | |
| encoder and this limit is ignored. If the number of threads | |
| is specified manually then no default limit will be used; | |
| this affects only -T0. | |
| This change helps on systems that have very many cores and | |
| using all of them for xz makes no sense. Previously xz -T0 | |
| could run out of memory on such systems because it attempted | |
| to reserve memory for too many threads. | |
| This also helps with 32-bit builds which don't have a large | |
| amount of address space that would be required for many | |
| threads. The default soft limit for -T0 is at most 1400 MiB | |
| on all 32-bit platforms. | |
| - Previously a low value in --memlimit-compress wouldn't cause | |
| xz to switch from multi-threaded mode to single-threaded mode | |
| if the limit cannot otherwise be met; xz failed instead. Now | |
| xz can switch to single-threaded mode and then, if needed, | |
| scale down the LZMA2 dictionary size too just like it already | |
| did when it was started in single-threaded mode. | |
| - The option --no-adjust no longer prevents xz from scaling down | |
| the number of threads as that doesn't affect the compressed | |
| output (only performance). Now --no-adjust only prevents | |
| adjustments that affect compressed output, that is, with | |
| --no-adjust xz won't switch from multi-threaded mode to | |
| single-threaded mode and won't scale down the LZMA2 | |
| dictionary size. | |
| - Added a new option --memlimit-mt-decompress=LIMIT. This is | |
| used to limit the number of decompressor threads (possibly | |
| falling back to single-threaded mode) but it will never make | |
| xz refuse to decompress a file. This has a system-specific | |
| default value because without any limit xz could end up | |
| allocating memory for the whole compressed input file, the | |
| whole uncompressed output file, multiple thread-specific | |
| decompressor instances and so on. Basically xz could | |
| attempt to use an insane amount of memory even with fairly | |
| common files. The system-specific default value is currently | |
| the same as the one used for compression with -T0. | |
| The new option works together with the existing option | |
| --memlimit-decompress=LIMIT. The old option sets a hard limit | |
| that must not be exceeded (xz will refuse to decompress) | |
| while the new option only restricts the number of threads. | |
| If the limit set with --memlimit-mt-decompress is greater | |
| than the limit set with --memlimit-compress, then the latter | |
| value is used also for --memlimit-mt-decompress. | |
| - Added new information to the output of xz --info-memory and | |
| new fields to the output of xz --robot --info-memory. | |
| - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders | |
| now that liblzma handles it. | |
| - Don't mention endianness for ARM and ARM-Thumb filters in | |
| --long-help. The filters only work for little endian | |
| instruction encoding but modern ARM processors using | |
| big endian data access still use little endian | |
| instruction encoding. So the help text was misleading. | |
| In contrast, the PowerPC filter is only for big endian | |
| 32/64-bit PowerPC code. Little endian PowerPC would need | |
| a separate filter. | |
| - Added decompression support for the .lz (lzip) file format | |
| version 0 and the original unextended version 1. It is | |
| autodetected by default. See also the option --format on | |
| the xz man page. | |
| - Sandboxing enabled by default: | |
| * Capsicum (FreeBSD) | |
| * pledge(2) (OpenBSD) | |
| * Scripts now support the .lz format using xz. | |
| * A few new tests were added. | |
| * The liblzma-specific tests are now supported in CMake-based | |
| builds too ("make test"). | |
| 5.3.5beta (2022-12-01) | |
| * All fixes from 5.2.9. | |
| * liblzma: | |
| - Added new LZMA_FILTER_LZMA1EXT for raw encoder and decoder to | |
| handle raw LZMA1 streams that don't have end of payload marker | |
| (EOPM) alias end of stream (EOS) marker. It can be used in | |
| filter chains, for example, with the x86 BCJ filter. | |
| - Added lzma_str_to_filters(), lzma_str_from_filters(), and | |
| lzma_str_list_filters() to make it easier for applications | |
| to get custom compression options from a user and convert | |
| it to an array of lzma_filter structures. | |
| - Added lzma_filters_free(). | |
| - lzma_filters_update() can now be used with the multi-threaded | |
| encoder (lzma_stream_encoder_mt()) to change the filter chain | |
| after LZMA_FULL_BARRIER or LZMA_FULL_FLUSH. | |
| - In lzma_options_lzma, allow nice_len = 2 and 3 with the match | |
| finders that require at least 3 or 4. Now it is internally | |
| rounded up if needed. | |
| - ARM64 filter was modified. It is still experimental. | |
| - Fixed LTO build with Clang if -fgnuc-version=10 or similar | |
| was used to make Clang look like GCC >= 10. Now it uses | |
| __has_attribute(__symver__) which should be reliable. | |
| * xz: | |
| - --threads=+1 or -T+1 is now a way to put xz into multi-threaded | |
| mode while using only one worker thread. | |
| - In --lzma2=nice=NUMBER allow 2 and 3 with all match finders | |
| now that liblzma handles it. | |
| * Updated translations: Chinese (simplified), Korean, and Turkish. | |
| 5.3.4alpha (2022-11-15) | |
| * All fixes from 5.2.7 and 5.2.8. | |
| * liblzma: | |
| - Minor improvements to the threaded decoder. | |
| - Added CRC64 implementation that uses SSSE3, SSE4.1, and CLMUL | |
| instructions on 32/64-bit x86 and E2K. On 32-bit x86 it's | |
| not enabled unless --disable-assembler is used but then | |
| the non-CLMUL code might be slower. Processor support is | |
| detected at runtime so this is built by default on x86-64 | |
| and E2K. On these platforms, if compiler flags indicate | |
| unconditional CLMUL support (-msse4.1 -mpclmul) then the | |
| generic version is not built, making liblzma 8-9 KiB smaller | |
| compared to having both versions included. | |
| With extremely compressible files this can make decompression | |
| up to twice as fast but with typical files 5 % improvement | |
| is a more realistic expectation. | |
| The CLMUL version is slower than the generic version with | |
| tiny inputs (especially at 1-8 bytes per call, but up to | |
| 16 bytes). In normal use in xz this doesn't matter at all. | |
| - Added an experimental ARM64 filter. This is *not* the final | |
| version! Files created with this experimental version won't | |
| be supported in the future versions! The filter design is | |
| a compromise where improving one use case makes some other | |
| cases worse. | |
| - Added decompression support for the .lz (lzip) file format | |
| version 0 and the original unextended version 1. See the | |
| API docs of lzma_lzip_decoder() for details. Also | |
| lzma_auto_decoder() supports .lz files. | |
| - Building with --disable-threads --enable-small | |
| is now thread-safe if the compiler supports | |
| __attribute__((__constructor__)) | |
| * xz: | |
| - Added support for OpenBSD's pledge(2) as a sandboxing method. | |
| - Don't mention endianness for ARM and ARM-Thumb filters in | |
| --long-help. The filters only work for little endian | |
| instruction encoding but modern ARM processors using | |
| big endian data access still use little endian | |
| instruction encoding. So the help text was misleading. | |
| In contrast, the PowerPC filter is only for big endian | |
| 32/64-bit PowerPC code. Little endian PowerPC would need | |
| a separate filter. | |
| - Added --experimental-arm64. This will be renamed once the | |
| filter is finished. Files created with this experimental | |
| filter will not be supported in the future! | |
| - Added new fields to the output of xz --robot --info-memory. | |
| - Added decompression support for the .lz (lzip) file format | |
| version 0 and the original unextended version 1. It is | |
| autodetected by default. See also the option --format on | |
| the xz man page. | |
| * Scripts now support the .lz format using xz. | |
| * Build systems: | |
| - New #defines in config.h: HAVE_ENCODER_ARM64, | |
| HAVE_DECODER_ARM64, HAVE_LZIP_DECODER, HAVE_CPUID_H, | |
| HAVE_FUNC_ATTRIBUTE_CONSTRUCTOR, HAVE_USABLE_CLMUL | |
| - New configure options: --disable-clmul-crc, | |
| --disable-microlzma, --disable-lzip-decoder, and | |
| 'pledge' is now an option in --enable-sandbox (but | |
| it's autodetected by default anyway). | |
| - INSTALL was updated to document the new configure options. | |
| - PACKAGERS now lists also --disable-microlzma and | |
| --disable-lzip-decoder as configure options that must | |
| not be used in builds for non-embedded use. | |
| * Tests: | |
| - Fix some of the tests so that they skip instead of fail if | |
| certain features have been disabled with configure options. | |
| It's still not perfect. | |
| - Other improvements to tests. | |
| * Updated translations: Croatian, Finnish, Hungarian, Polish, | |
| Romanian, Spanish, Swedish, and Ukrainian. | |
| 5.3.3alpha (2022-08-22) | |
| * All fixes from 5.2.6. | |
| * liblzma: | |
| - Fixed 32-bit build. | |
| - Added threaded .xz decompressor lzma_stream_decoder_mt(). | |
| It can use multiple threads with .xz files that have multiple | |
| Blocks with size information in Block Headers. The threaded | |
| encoder in xz has always created such files. | |
| Single-threaded encoder cannot store the size information in | |
| Block Headers even if one used LZMA_FULL_FLUSH to create | |
| multiple Blocks, so this threaded decoder cannot use multiple | |
| threads with such files. | |
| If there are multiple Streams (concatenated .xz files), one | |
| Stream will be decompressed completely before starting the | |
| next Stream. | |
| - A new decoder flag LZMA_FAIL_FAST was added. It makes the | |
| threaded decompressor report errors soon instead of first | |
| flushing all pending data before the error location. | |
| * xz: | |
| - Using -T0 (--threads=0) will now use multi-threaded encoder | |
| even on a single-core system. This is to ensure that output | |
| from the same xz binary is identical on both single-core and | |
| multi-core systems. | |
| - A default soft memory usage limit is now used for compression | |
| when -T0 is used and no explicit limit has been specified. | |
| This soft limit is used to restrict the number of threads | |
| but if the limit is exceeded with even one thread then xz | |
| will continue with one thread using the multi-threaded | |
| encoder and this limit is ignored. If the number of threads | |
| is specified manually then no default limit will be used; | |
| this affects only -T0. | |
| This change helps on systems that have very many cores and | |
| using all of them for xz makes no sense. Previously xz -T0 | |
| could run out of memory on such systems because it attempted | |
| to reserve memory for too many threads. | |
| This also helps with 32-bit builds which don't have a large | |
| amount of address space that would be required for many | |
| threads. The default limit is 1400 MiB on all 32-bit | |
| platforms with -T0. | |
| Now xz -T0 should just work. It might use too few threads | |
| in some cases but at least it shouldn't easily run out of | |
| memory. It's possible that this will be tweaked before 5.4.0. | |
| - Changes to --memlimit-compress and --no-adjust: | |
| In single-threaded mode, --memlimit-compress can make xz | |
| scale down the LZMA2 dictionary size to meet the memory usage | |
| limit. This obviously affects the compressed output. However, | |
| if xz was in threaded mode, --memlimit-compress could make xz | |
| reduce the number of threads but it wouldn't make xz switch | |
| from multi-threaded mode to single-threaded mode or scale | |
| down the LZMA2 dictionary size. This seemed illogical. | |
| Now --memlimit-compress can make xz switch to single-threaded | |
| mode if one thread in multi-threaded mode uses too much | |
| memory. If memory usage is still too high, then the LZMA2 | |
| dictionary size can be scaled down too. | |
| The option --no-adjust was also changed so that it no longer | |
| prevents xz from scaling down the number of threads as that | |
| doesn't affect compressed output (only performance). After | |
| this commit --no-adjust only prevents adjustments that affect | |
| compressed output, that is, with --no-adjust xz won't switch | |
| from multithreaded mode to single-threaded mode and won't | |
| scale down the LZMA2 dictionary size. | |
| - Added a new option --memlimit-mt-decompress=LIMIT. This is | |
| used to limit the number of decompressor threads (possibly | |
| falling back to single-threaded mode) but it will never make | |
| xz refuse to decompress a file. This has a system-specific | |
| default value because without any limit xz could end up | |
| allocating memory for the whole compressed input file, the | |
| whole uncompressed output file, multiple thread-specific | |
| decompressor instances and so on. Basically xz could | |
| attempt to use an insane amount of memory even with fairly | |
| common files. | |
| The new option works together with the existing option | |
| --memlimit-decompress=LIMIT. The old option sets a hard limit | |
| that must not be exceeded (xz will refuse to decompress) | |
| while the new option only restricts the number of threads. | |
| If the limit set with --memlimit-mt-decompress is greater | |
| than the limit set with --memlimit-compress, then the latter | |
| value is used also for --memlimit-mt-decompress. | |
| * Tests: | |
| - Added a few more tests. | |
| - Added tests/code_coverage.sh to create a code coverage report | |
| of the tests. | |
| * Build systems: | |
| - Automake's parallel test harness is now used to make tests | |
| finish faster. | |
| - Added the CMake files to the distribution tarball. These were | |
| supposed to be in 5.2.5 already. | |
| - Added liblzma tests to the CMake build. | |
| - Windows: Fix building of liblzma.dll with the included | |
| Visual Studio project files. | |
| 5.3.2alpha (2021-10-28) | |
| This release was made on short notice so that recent erofs-utils can | |
| be built with LZMA support without needing a snapshot from xz.git. | |
| Thus many pending things were not included, not even updated | |
| translations (which would need to be updated for the new --list | |
| strings anyway). | |
| * All fixes from 5.2.5. | |
| * xz: | |
| - When copying metadata from the source file to the destination | |
| file, don't try to set the group (GID) if it is already set | |
| correctly. This avoids a failure on OpenBSD (and possibly on | |
| a few other OSes) where files may get created so that their | |
| group doesn't belong to the user, and fchown(2) can fail even | |
| if it needs to do nothing. | |
| - The --keep option now accepts symlinks, hardlinks, and | |
| setuid, setgid, and sticky files. Previously this required | |
| using --force. | |
| - Split the long strings used in --list and --info-memory modes | |
| to make them much easier for translators. | |
| - If built with sandbox support and enabling the sandbox fails, | |
| xz will now immediately exit with exit status of 1. Previously | |
| it would only display a warning if -vv was used. | |
| - Cap --memlimit-compress to 2000 MiB on MIPS32 because on | |
| MIPS32 userspace processes are limited to 2 GiB of address | |
| space. | |
| * liblzma: | |
| - Added lzma_microlzma_encoder() and lzma_microlzma_decoder(). | |
| The API is in lzma/container.h. | |
| The MicroLZMA format is a raw LZMA stream (without end marker) | |
| whose first byte (always 0x00) has been replaced with | |
| bitwise-negation of the LZMA properties (lc/lp/pb). It was | |
| created for use in EROFS but may be used in other contexts | |
| as well where it is important to avoid wasting bytes for | |
| stream headers or footers. The format is also supported by | |
| XZ Embedded. | |
| The MicroLZMA encoder API in liblzma can compress into a | |
| fixed-sized output buffer so that as much data is compressed | |
| as can be fit into the buffer while still creating a valid | |
| MicroLZMA stream. This is needed for EROFS. | |
| - Added fuzzing support. | |
| - Support Intel Control-flow Enforcement Technology (CET) in | |
| 32-bit x86 assembly files. | |
| - Visual Studio: Use non-standard _MSVC_LANG to detect C++ | |
| standard version in the lzma.h API header. It's used to | |
| detect when "noexcept" can be used. | |
| * Scripts: | |
| - Fix exit status of xzdiff/xzcmp. Exit status could be 2 when | |
| the correct value is 1. | |
| - Fix exit status of xzgrep. | |
| - Detect corrupt .bz2 files in xzgrep. | |
| - Add zstd support to xzgrep and xzdiff/xzcmp. | |
| - Fix less(1) version detection in xzless. It failed if the | |
| version number from "less -V" contained a dot. | |
| * Fix typos and technical issues in man pages. | |
| * Build systems: | |
| - Windows: Fix building of resource files when config.h isn't | |
| used. CMake + Visual Studio can now build liblzma.dll. | |
| - Various fixes to the CMake support. It might still need a few | |
| more fixes even for liblzma-only builds. | |
| 5.3.1alpha (2018-04-29) | |
| * All fixes from 5.2.4. | |
| * Add lzma_file_info_decoder() into liblzma and use it in xz to | |
| implement the --list feature. | |
| * Capsicum sandbox support is enabled by default where available | |
| (FreeBSD >= 10). | |
| 5.2.10 (2022-12-13) | |
| * xz: Don't modify argv[] when parsing the --memlimit* and | |
| --block-list command line options. This fixes confusing | |
| arguments in process listing (like "ps auxf"). | |
| * GNU/Linux only: Use __has_attribute(__symver__) to detect if | |
| that attribute is supported. This fixes build on Mandriva where | |
| Clang is patched to define __GNUC__ to 11 by default (instead | |
| of 4 as used by Clang upstream). | |
| 5.2.9 (2022-11-30) | |
| * liblzma: | |
| - Fixed an infinite loop in LZMA encoder initialization | |
| if dict_size >= 2 GiB. (The encoder only supports up | |
| to 1536 MiB.) | |
| - Fixed two cases of invalid free() that can happen if | |
| a tiny allocation fails in encoder re-initialization | |
| or in lzma_filters_update(). These bugs had some | |
| similarities with the bug fixed in 5.2.7. | |
| - Fixed lzma_block_encoder() not allowing the use of | |
| LZMA_SYNC_FLUSH with lzma_code() even though it was | |
| documented to be supported. The sync-flush code in | |
| the Block encoder was already used internally via | |
| lzma_stream_encoder(), so this was just a missing flag | |
| in the lzma_block_encoder() API function. | |
| - GNU/Linux only: Don't put symbol versions into static | |
| liblzma as it breaks things in some cases (and even if | |
| it didn't break anything, symbol versions in static | |
| libraries are useless anyway). The downside of the fix | |
| is that if the configure options --with-pic or --without-pic | |
| are used then it's not possible to build both shared and | |
| static liblzma at the same time on GNU/Linux anymore; | |
| with those options --disable-static or --disable-shared | |
| must be used too. | |
| * New email address for bug reports is <[email protected]> which | |
| forwards messages to Lasse Collin and Jia Tan. | |
| 5.2.8 (2022-11-13) | |
| * xz: | |
| - If xz cannot remove an input file when it should, this | |
| is now treated as a warning (exit status 2) instead of | |
| an error (exit status 1). This matches GNU gzip and it | |
| is more logical as at that point the output file has | |
| already been successfully closed. | |
| - Fix handling of .xz files with an unsupported check type. | |
| Previously such printed a warning message but then xz | |
| behaved as if an error had occurred (didn't decompress, | |
| exit status 1). Now a warning is printed, decompression | |
| is done anyway, and exit status is 2. This used to work | |
| slightly before 5.0.0. In practice this bug matters only | |
| if xz has been built with some check types disabled. As | |
| instructed in PACKAGERS, such builds should be done in | |
| special situations only. | |
| - Fix "xz -dc --single-stream tests/files/good-0-empty.xz" | |
| which failed with "Internal error (bug)". That is, | |
| --single-stream was broken if the first .xz stream in | |
| the input file didn't contain any uncompressed data. | |
| - Fix displaying file sizes in the progress indicator when | |
| working in passthru mode and there are multiple input files. | |
| Just like "gzip -cdf", "xz -cdf" works like "cat" when the | |
| input file isn't a supported compressed file format. In | |
| this case the file size counters weren't reset between | |
| files so with multiple input files the progress indicator | |
| displayed an incorrect (too large) value. | |
| * liblzma: | |
| - API docs in lzma/container.h: | |
| * Update the list of decoder flags in the decoder | |
| function docs. | |
| * Explain LZMA_CONCATENATED behavior with .lzma files | |
| in lzma_auto_decoder() docs. | |
| - OpenBSD: Use HW_NCPUONLINE to detect the number of | |
| available hardware threads in lzma_physmem(). | |
| - Fix use of wrong macro to detect x86 SSE2 support. | |
| __SSE2_MATH__ was used with GCC/Clang but the correct | |
| one is __SSE2__. The first one means that SSE2 is used | |
| for floating point math which is irrelevant here. | |
| The affected SSE2 code isn't used on x86-64 so this affects | |
| only 32-bit x86 builds that use -msse2 without -mfpmath=sse | |
| (there is no runtime detection for SSE2). It improves LZMA | |
| compression speed (not decompression). | |
| - Fix the build with Intel C compiler 2021 (ICC, not ICX) | |
| on Linux. It defines __GNUC__ to 10 but doesn't support | |
| the __symver__ attribute introduced in GCC 10. | |
| * Scripts: Ignore warnings from xz by using --quiet --no-warn. | |
| This is needed if the input .xz files use an unsupported | |
| check type. | |
| * Translations: | |
| - Updated Croatian and Turkish translations. | |
| - One new translations wasn't included because it needed | |
| technical fixes. It will be in upcoming 5.4.0. No new | |
| translations will be added to the 5.2.x branch anymore. | |
| - Renamed the French man page translation file from | |
| fr_FR.po to fr.po and thus also its install directory | |
| (like /usr/share/man/fr_FR -> .../fr). | |
| - Man page translations for upcoming 5.4.0 are now handled | |
| in the Translation Project. | |
| * Update doc/faq.txt a little so it's less out-of-date. | |
| 5.2.7 (2022-09-30) | |
| * liblzma: | |
| - Made lzma_filters_copy() to never modify the destination | |
| array if an error occurs. lzma_stream_encoder() and | |
| lzma_stream_encoder_mt() already assumed this. Before this | |
| change, if a tiny memory allocation in lzma_filters_copy() | |
| failed it would lead to a crash (invalid free() or invalid | |
| memory reads) in the cleanup paths of these two encoder | |
| initialization functions. | |
| - Added missing integer overflow check to lzma_index_append(). | |
| This affects xz --list and other applications that decode | |
| the Index field from .xz files using lzma_index_decoder(). | |
| Normal decompression of .xz files doesn't call this code | |
| and thus most applications using liblzma aren't affected | |
| by this bug. | |
| - Single-threaded .xz decoder (lzma_stream_decoder()): If | |
| lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible | |
| to use lzma_memlimit_set() to increase the limit and continue | |
| decoding. This was supposed to work from the beginning | |
| but there was a bug. With other decoders (.lzma or | |
| threaded .xz decoder) this already worked correctly. | |
| - Fixed accumulation of integrity check type statistics in | |
| lzma_index_cat(). This bug made lzma_index_checks() return | |
| only the type of the integrity check of the last Stream | |
| when multiple lzma_indexes were concatenated. Most | |
| applications don't use these APIs but in xz it made | |
| xz --list not list all check types from concatenated .xz | |
| files. In xz --list --verbose only the per-file "Check:" | |
| lines were affected and in xz --robot --list only the "file" | |
| line was affected. | |
| - Added ABI compatibility with executables that were linked | |
| against liblzma in RHEL/CentOS 7 or other liblzma builds | |
| that had copied the problematic patch from RHEL/CentOS 7 | |
| (xz-5.2.2-compat-libs.patch). For the details, see the | |
| comment at the top of src/liblzma/validate_map.sh. | |
| WARNING: This uses __symver__ attribute with GCC >= 10. | |
| In other cases the traditional __asm__(".symver ...") | |
| is used. Using link-time optimization (LTO, -flto) with | |
| GCC versions older than 10 can silently result in | |
| broken liblzma.so.5 (incorrect symbol versions)! If you | |
| want to use -flto with GCC, you must use GCC >= 10. | |
| LTO with Clang seems to work even with the traditional | |
| __asm__(".symver ...") method. | |
| * xzgrep: Fixed compatibility with old shells that break if | |
| comments inside command substitutions have apostrophes ('). | |
| This problem was introduced in 5.2.6. | |
| * Build systems: | |
| - New #define in config.h: HAVE_SYMBOL_VERSIONS_LINUX | |
| - Windows: Fixed liblzma.dll build with Visual Studio project | |
| files. It broke in 5.2.6 due to a change that was made to | |
| improve CMake support. | |
| - Windows: Building liblzma with UNICODE defined should now | |
| work. | |
| - CMake files are now actually included in the release tarball. | |
| They should have been in 5.2.5 already. | |
| - Minor CMake fixes and improvements. | |
| * Added a new translation: Turkish | |
| 5.2.6 (2022-08-12) | |
| * xz: | |
| - The --keep option now accepts symlinks, hardlinks, and | |
| setuid, setgid, and sticky files. Previously this required | |
| using --force. | |
| - When copying metadata from the source file to the destination | |
| file, don't try to set the group (GID) if it is already set | |
| correctly. This avoids a failure on OpenBSD (and possibly on | |
| a few other OSes) where files may get created so that their | |
| group doesn't belong to the user, and fchown(2) can fail even | |
| if it needs to do nothing. | |
| - Cap --memlimit-compress to 2000 MiB instead of 4020 MiB on | |
| MIPS32 because on MIPS32 userspace processes are limited | |
| to 2 GiB of address space. | |
| * liblzma: | |
| - Fixed a missing error-check in the threaded encoder. If a | |
| small memory allocation fails, a .xz file with an invalid | |
| Index field would be created. Decompressing such a file would | |
| produce the correct output but result in an error at the end. | |
| Thus this is a "mild" data corruption bug. Note that while | |
| a failed memory allocation can trigger the bug, it cannot | |
| cause invalid memory access. | |
| - The decoder for .lzma files now supports files that have | |
| uncompressed size stored in the header and still use the | |
| end of payload marker (end of stream marker) at the end | |
| of the LZMA stream. Such files are rare but, according to | |
| the documentation in LZMA SDK, they are valid. | |
| doc/lzma-file-format.txt was updated too. | |
| - Improved 32-bit x86 assembly files: | |
| * Support Intel Control-flow Enforcement Technology (CET) | |
| * Use non-executable stack on FreeBSD. | |
| - Visual Studio: Use non-standard _MSVC_LANG to detect C++ | |
| standard version in the lzma.h API header. It's used to | |
| detect when "noexcept" can be used. | |
| * xzgrep: | |
| - Fixed arbitrary command injection via a malicious filename | |
| (CVE-2022-1271, ZDI-CAN-16587). A standalone patch for | |
| this was released to the public on 2022-04-07. A slight | |
| robustness improvement has been made since then and, if | |
| using GNU or *BSD grep, a new faster method is now used | |
| that doesn't use the old sed-based construct at all. This | |
| also fixes bad output with GNU grep >= 3.5 (2020-09-27) | |
| when xzgrepping binary files. | |
| This vulnerability was discovered by: | |
| cleemy desu wayo working with Trend Micro Zero Day Initiative | |
| - Fixed detection of corrupt .bz2 files. | |
| - Improved error handling to fix exit status in some situations | |
| and to fix handling of signals: in some situations a signal | |
| didn't make xzgrep exit when it clearly should have. It's | |
| possible that the signal handling still isn't quite perfect | |
| but hopefully it's good enough. | |
| - Documented exit statuses on the man page. | |
| - xzegrep and xzfgrep now use "grep -E" and "grep -F" instead | |
| of the deprecated egrep and fgrep commands. | |
| - Fixed parsing of the options -E, -F, -G, -P, and -X. The | |
| problem occurred when multiple options were specified in | |
| a single argument, for example, | |
| echo foo | xzgrep -Fe foo | |
| treated foo as a filename because -Fe wasn't correctly | |
| split into -F -e. | |
| - Added zstd support. | |
| * xzdiff/xzcmp: | |
| - Fixed wrong exit status. Exit status could be 2 when the | |
| correct value is 1. | |
| - Documented on the man page that exit status of 2 is used | |
| for decompression errors. | |
| - Added zstd support. | |
| * xzless: | |
| - Fix less(1) version detection. It failed if the version number | |
| from "less -V" contained a dot. | |
| * Translations: | |
| - Added new translations: Catalan, Croatian, Esperanto, | |
| Korean, Portuguese, Romanian, Serbian, Spanish, Swedish, | |
| and Ukrainian | |
| - Updated the Brazilian Portuguese translation. | |
| - Added French man page translation. This and the existing | |
| German translation aren't complete anymore because the | |
| English man pages got a few updates and the translators | |
| weren't reached so that they could update their work. | |
| * Build systems: | |
| - Windows: Fix building of resource files when config.h isn't | |
| used. CMake + Visual Studio can now build liblzma.dll. | |
| - Various fixes to the CMake support. Building static or shared | |
| liblzma should work fine in most cases. In contrast, building | |
| the command line tools with CMake is still clearly incomplete | |
| and experimental and should be used for testing only. | |
| 5.2.5 (2020-03-17) | |
| * liblzma: | |
| - Fixed several C99/C11 conformance bugs. Now the code is clean | |
| under gcc/clang -fsanitize=undefined. Some of these changes | |
| might have a negative effect on performance with old GCC | |
| versions or compilers other than GCC and Clang. The configure | |
| option --enable-unsafe-type-punning can be used to (mostly) | |
| restore the old behavior but it shouldn't normally be used. | |
| - Improved API documentation of lzma_properties_decode(). | |
| - Added a very minor encoder speed optimization. | |
| * xz: | |
| - Fixed a crash in "xz -dcfv not_an_xz_file". All four options | |
| were required to trigger it. The crash occurred in the | |
| progress indicator code when xz was in passthru mode where | |
| xz works like "cat". | |
| - Fixed an integer overflow with 32-bit off_t. It could happen | |
| when decompressing a file that has a long run of zero bytes | |
| which xz would try to write as a sparse file. Since the build | |
| system enables large file support by default, off_t is | |
| normally 64-bit even on 32-bit systems. | |
| - Fixes for --flush-timeout: | |
| * Fix semi-busy-waiting. | |
| * Avoid unneeded flushes when no new input has arrived | |
| since the previous flush was completed. | |
| - Added a special case for 32-bit xz: If --memlimit-compress is | |
| used to specify a limit that exceeds 4020 MiB, the limit will | |
| be set to 4020 MiB. The values "0" and "max" aren't affected | |
| by this and neither is decompression. This hack can be | |
| helpful when a 32-bit xz has access to 4 GiB address space | |
| but the specified memlimit exceeds 4 GiB. This can happen | |
| e.g. with some scripts. | |
| - Capsicum sandbox is now enabled by default where available | |
| (FreeBSD >= 10). The sandbox debug messages (xz -vv) were | |
| removed since they seemed to be more annoying than useful. | |
| - DOS build now requires DJGPP 2.05 instead of 2.04beta. | |
| A workaround for a locale problem with DJGPP 2.05 was added. | |
| * xzgrep and other scripts: | |
| - Added a configure option --enable-path-for-scripts=PREFIX. | |
| It is disabled by default except on Solaris where the default | |
| is /usr/xpg4/bin. See INSTALL for details. | |
| - Added a workaround for a POSIX shell detection problem on | |
| Solaris. | |
| * Build systems: | |
| - Added preliminary build instructions for z/OS. See INSTALL | |
| section 1.2.9. | |
| - Experimental CMake support was added. It should work to build | |
| static liblzma on a few operating systems. It may or may not | |
| work to build shared liblzma. On some platforms it can build | |
| xz and xzdec too but those are only for testing. See the | |
| comment in the beginning of CMakeLists.txt for details. | |
| - Visual Studio project files were updated. | |
| WindowsTargetPlatformVersion was removed from VS2017 files | |
| and set to "10.0" in the added VS2019 files. In the future | |
| the VS project files will be removed when CMake support is | |
| good enough. | |
| - New #defines in config.h: HAVE___BUILTIN_ASSUME_ALIGNED, | |
| HAVE___BUILTIN_BSWAPXX, and TUKLIB_USE_UNSAFE_TYPE_PUNNING. | |
| - autogen.sh has a new optional dependency on po4a and a new | |
| option --no-po4a to skip that step. This matters only if one | |
| wants to remake the build files. po4a is used to update the | |
| translated man pages but as long as the man pages haven't | |
| been modified, there's nothing to update and one can use | |
| --no-po4a to avoid the dependency on po4a. | |
| * Translations: | |
| - XZ Utils translations are now handled by the Translation | |
| Project: https://translationproject.org/domain/xz.html | |
| - All man pages are now included in German too. | |
| - New xz translations: Brazilian Portuguese, Finnish, | |
| Hungarian, Chinese (simplified), Chinese (traditional), | |
| and Danish (partial translation) | |
| - Updated xz translations: French, German, Italian, and Polish | |
| - Unfortunately a few new xz translations weren't included due | |
| to technical problems like too long lines in --help output or | |
| misaligned column headings in tables. In the future, many of | |
| these strings will be split and e.g. the table column | |
| alignment will be handled in software. This should make the | |
| strings easier to translate. | |
| 5.2.4 (2018-04-29) | |
| * liblzma: | |
| - Allow 0 as memory usage limit instead of returning | |
| LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified, | |
| which effectively is the same as 0. | |
| - Use "noexcept" keyword instead of "throw()" in the public | |
| headers when a C++11 (or newer standard) compiler is used. | |
| - Added a portability fix for recent Intel C Compilers. | |
| - Microsoft Visual Studio build files have been moved under | |
| windows/vs2013 and windows/vs2017. | |
| * xz: | |
| - Fix "xz --list --robot missing_or_bad_file.xz" which would | |
| try to print an uninitialized string and thus produce garbage | |
| output. Since the exit status is non-zero, most uses of such | |
| a command won't try to interpret the garbage output. | |
| - "xz --list foo.xz" could print "Internal error (bug)" in a | |
| corner case where a specific memory usage limit had been set. | |
| 5.2.3 (2016-12-30) | |
| * xz: | |
| - Always close a file before trying to delete it to avoid | |
| problems on some operating system and file system combinations. | |
| - Fixed copying of file timestamps on Windows. | |
| - Added experimental (disabled by default) sandbox support using | |
| Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL. | |
| * C99/C11 conformance fixes to liblzma. The issues affected at least | |
| some builds using link-time optimizations. | |
| * Fixed bugs in the rarely-used function lzma_index_dup(). | |
| * Use of external SHA-256 code is now disabled by default. | |
| It can still be enabled by passing --enable-external-sha256 | |
| to configure. The reasons to disable it by default (see INSTALL | |
| for more details): | |
| - Some OS-specific SHA-256 implementations conflict with | |
| OpenSSL and cause problems in programs that link against both | |
| liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0 | |
| are affected. | |
| - The internal SHA-256 is faster than the SHA-256 code in | |
| some operating systems. | |
| * Changed CPU core count detection to use sched_getaffinity() on | |
| GNU/Linux and GNU/kFreeBSD. | |
| * Fixes to the build-system and xz to make xz buildable even when | |
| encoders, decoders, or threading have been disabled from libilzma | |
| using configure options. These fixes added two new #defines to | |
| config.h: HAVE_ENCODERS and HAVE_DECODERS. | |
| 5.2.2 (2015-09-29) | |
| * Fixed bugs in QNX-specific code. | |
| * Omitted the use of pipe2() even if it is available to avoid | |
| portability issues with some old Linux and glibc combinations. | |
| * Updated German translation. | |
| * Added project files to build static and shared liblzma (not the | |
| whole XZ Utils) with Visual Studio 2013 update 2 or later. | |
| * Documented that threaded decompression hasn't been implemented | |
| yet. A 5.2.0 NEWS entry describing multi-threading support had | |
| incorrectly said "decompression" when it should have said | |
| "compression". | |
| 5.2.1 (2015-02-26) | |
| * Fixed a compression-ratio regression in fast mode of LZMA1 and | |
| LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases. | |
| * Fixed a portability problem in xz that affected at least OpenBSD. | |
| * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs | |
| from most other mktemp implementations. | |
| * Changed CPU core count detection to use cpuset_getaffinity() on | |
| FreeBSD. | |
| 5.2.0 (2014-12-21) | |
| Since 5.1.4beta: | |
| * All fixes from 5.0.8 | |
| * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset | |
| was used. | |
| * xzdiff: If mktemp isn't installed, mkdir will be used as | |
| a fallback to create a temporary directory. Installing mktemp | |
| is still recommended. | |
| * Updated French, German, Italian, Polish, and Vietnamese | |
| translations. | |
| Summary of fixes and new features added in the 5.1.x development | |
| releases: | |
| * liblzma: | |
| - Added support for multi-threaded compression. See the | |
| lzma_mt structure, lzma_stream_encoder_mt(), and | |
| lzma_stream_encoder_mt_memusage() in <lzma/container.h>, | |
| lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads() | |
| in <lzma/hardware.h> for details. | |
| - Made the uses of lzma_allocator const correct. | |
| - Added lzma_block_uncomp_encode() to create uncompressed | |
| .xz Blocks using LZMA2 uncompressed chunks. | |
| - Added support for LZMA_IGNORE_CHECK. | |
| - A few speed optimizations were made. | |
| - Added support for symbol versioning. It is enabled by default | |
| on GNU/Linux, other GNU-based systems, and FreeBSD. | |
| - liblzma (not the whole XZ Utils) should now be buildable | |
| with MSVC 2013 update 2 or later using windows/config.h. | |
| * xz: | |
| - Fixed a race condition in the signal handling. It was | |
| possible that e.g. the first SIGINT didn't make xz exit | |
| if reading or writing blocked and one had bad luck. The fix | |
| is non-trivial, so as of writing it is unknown if it will be | |
| backported to the v5.0 branch. | |
| - Multi-threaded compression can be enabled with the | |
| --threads (-T) option. | |
| [Fixed: This originally said "decompression".] | |
| - New command line options in xz: --single-stream, | |
| --block-size=SIZE, --block-list=SIZES, | |
| --flush-timeout=TIMEOUT, and --ignore-check. | |
| - xz -lvv now shows the minimum xz version that is required to | |
| decompress the file. Currently it is 5.0.0 for all supported | |
| .xz files except files with empty LZMA2 streams require 5.0.2. | |
| * xzdiff and xzgrep now support .lzo files if lzop is installed. | |
| The .tzo suffix is also recognized as a shorthand for .tar.lzo. | |
| 5.1.4beta (2014-09-14) | |
| * All fixes from 5.0.6 | |
| * liblzma: Fixed the use of presets in threaded encoder | |
| initialization. | |
| * xz --block-list and --block-size can now be used together | |
| in single-threaded mode. Previously the combination only | |
| worked in multi-threaded mode. | |
| * Added support for LZMA_IGNORE_CHECK to liblzma and made it | |
| available in xz as --ignore-check. | |
| * liblzma speed optimizations: | |
| - Initialization of a new LZMA1 or LZMA2 encoder has been | |
| optimized. (The speed of reinitializing an already-allocated | |
| encoder isn't affected.) This helps when compressing many | |
| small buffers with lzma_stream_buffer_encode() and other | |
| similar situations where an already-allocated encoder state | |
| isn't reused. This speed-up is visible in xz too if one | |
| compresses many small files one at a time instead running xz | |
| once and giving all files as command-line arguments. | |
| - Buffer comparisons are now much faster when unaligned access | |
| is allowed (configured with --enable-unaligned-access). This | |
| speeds up encoding significantly. There is arch-specific code | |
| for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best | |
| results and there's no run-time CPU detection for now). | |
| For other archs there is only generic code which probably | |
| isn't as optimal as arch-specific solutions could be. | |
| - A few speed optimizations were made to the SHA-256 code. | |
| (Note that the builtin SHA-256 code isn't used on all | |
| operating systems.) | |
| * liblzma can now be built with MSVC 2013 update 2 or later | |
| using windows/config.h. | |
| * Vietnamese translation was added. | |
| 5.1.3alpha (2013-10-26) | |
| * All fixes from 5.0.5 | |
| * liblzma: | |
| - Fixed a deadlock in the threaded encoder. | |
| - Made the uses of lzma_allocator const correct. | |
| - Added lzma_block_uncomp_encode() to create uncompressed | |
| .xz Blocks using LZMA2 uncompressed chunks. | |
| - Added support for native threads on Windows and the ability | |
| to detect the number of CPU cores. | |
| * xz: | |
| - Fixed a race condition in the signal handling. It was | |
| possible that e.g. the first SIGINT didn't make xz exit | |
| if reading or writing blocked and one had bad luck. The fix | |
| is non-trivial, so as of writing it is unknown if it will be | |
| backported to the v5.0 branch. | |
| - Made the progress indicator work correctly in threaded mode. | |
| - Threaded encoder now works together with --block-list=SIZES. | |
| - Added preliminary support for --flush-timeout=TIMEOUT. | |
| It can be useful for (somewhat) real-time streaming. For | |
| now the decompression side has to be done with something | |
| else than the xz tool due to how xz does buffering, but this | |
| should be fixed. | |
| 5.1.2alpha (2012-07-04) | |
| * All fixes from 5.0.3 and 5.0.4 | |
| * liblzma: | |
| - Fixed a deadlock and an invalid free() in the threaded encoder. | |
| - Added support for symbol versioning. It is enabled by default | |
| on GNU/Linux, other GNU-based systems, and FreeBSD. | |
| - Use SHA-256 implementation from the operating system if one is | |
| available in libc, libmd, or libutil. liblzma won't use e.g. | |
| OpenSSL or libgcrypt to avoid introducing new dependencies. | |
| - Fixed liblzma.pc for static linking. | |
| - Fixed a few portability bugs. | |
| * xz --decompress --single-stream now fixes the input position after | |
| successful decompression. Now the following works: | |
| echo foo | xz > foo.xz | |
| echo bar | xz >> foo.xz | |
| ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz | |
| Note that it doesn't work if the input is not seekable | |
| or if there is Stream Padding between the concatenated | |
| .xz Streams. | |
| * xz -lvv now shows the minimum xz version that is required to | |
| decompress the file. Currently it is 5.0.0 for all supported .xz | |
| files except files with empty LZMA2 streams require 5.0.2. | |
| * Added an *incomplete* implementation of --block-list=SIZES to xz. | |
| It only works correctly in single-threaded mode and when | |
| --block-size isn't used at the same time. --block-list allows | |
| specifying the sizes of Blocks which can be useful e.g. when | |
| creating files for random-access reading. | |
| 5.1.1alpha (2011-04-12) | |
| * All fixes from 5.0.2 | |
| * liblzma fixes that will also be included in 5.0.3: | |
| - A memory leak was fixed. | |
| - lzma_stream_buffer_encode() no longer creates an empty .xz | |
| Block if encoding an empty buffer. Such an empty Block with | |
| LZMA2 data would trigger a bug in 5.0.1 and older (see the | |
| first bullet point in 5.0.2 notes). When releasing 5.0.2, | |
| I thought that no encoder creates this kind of files but | |
| I was wrong. | |
| - Validate function arguments better in a few functions. Most | |
| importantly, specifying an unsupported integrity check to | |
| lzma_stream_buffer_encode() no longer creates a corrupt .xz | |
| file. Probably no application tries to do that, so this | |
| shouldn't be a big problem in practice. | |
| - Document that lzma_block_buffer_encode(), | |
| lzma_easy_buffer_encode(), lzma_stream_encoder(), and | |
| lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. | |
| - The return values of the _memusage() functions are now | |
| documented better. | |
| * Support for multithreaded compression was added using the simplest | |
| method, which splits the input data into blocks and compresses | |
| them independently. Other methods will be added in the future. | |
| The current method has room for improvement, e.g. it is possible | |
| to reduce the memory usage. | |
| * Added the options --single-stream and --block-size=SIZE to xz. | |
| * xzdiff and xzgrep now support .lzo files if lzop is installed. | |
| The .tzo suffix is also recognized as a shorthand for .tar.lzo. | |
| * Support for short 8.3 filenames under DOS was added to xz. It is | |
| experimental and may change before it gets into a stable release. | |
| 5.0.8 (2014-12-21) | |
| * Fixed an old bug in xzgrep that affected OpenBSD and probably | |
| a few other operating systems too. | |
| * Updated French and German translations. | |
| * Added support for detecting the amount of RAM on AmigaOS/AROS. | |
| * Minor build system updates. | |
| 5.0.7 (2014-09-20) | |
| * Fix regressions introduced in 5.0.6: | |
| - Fix building with non-GNU make. | |
| - Fix invalid Libs.private value in liblzma.pc which broke | |
| static linking against liblzma if the linker flags were | |
| taken from pkg-config. | |
| 5.0.6 (2014-09-14) | |
| * xzgrep now exits with status 0 if at least one file matched. | |
| * A few minor portability and build system fixes | |
| 5.0.5 (2013-06-30) | |
| * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing | |
| .lzma files that have less common settings in the headers | |
| (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed | |
| size greater than 256 GiB). The limitations existed to avoid false | |
| positives when detecting .lzma files. The lc + lp <= 4 limitation | |
| still remains since liblzma's LZMA decoder has that limitation. | |
| NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT | |
| affected by this change. They still consider uncommon .lzma headers | |
| as not being in the .lzma format. Changing this would give way too | |
| many false positives. | |
| * xz: | |
| - Interaction of preset and custom filter chain options was | |
| made less illogical. This affects only certain less typical | |
| uses cases so few people are expected to notice this change. | |
| Now when a custom filter chain option (e.g. --lzma2) is | |
| specified, all preset options (-0 ... -9, -e) earlier are on | |
| the command line are completely forgotten. Similarly, when | |
| a preset option is specified, all custom filter chain options | |
| earlier on the command line are completely forgotten. | |
| Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e" | |
| which is equivalent to "xz -6e". Earlier -e didn't put xz back | |
| into preset mode and thus the example command was equivalent | |
| to "xz --lzma2=preset=5". | |
| Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to | |
| "xz -7". Earlier a custom filter chain option didn't make | |
| xz forget the -e option so the example was equivalent to | |
| "xz -7e". | |
| - Fixes and improvements to error handling. | |
| - Various fixes to the man page. | |
| * xzless: Fixed to work with "less" versions 448 and later. | |
| * xzgrep: Made -h an alias for --no-filename. | |
| * Include the previously missing debug/translation.bash which can | |
| be useful for translators. | |
| * Include a build script for Mac OS X. This has been in the Git | |
| repository since 2010 but due to a mistake in Makefile.am the | |
| script hasn't been included in a release tarball before. | |
| 5.0.4 (2012-06-22) | |
| * liblzma: | |
| - Fix lzma_index_init(). It could crash if memory allocation | |
| failed. | |
| - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ | |
| filter is used and the application only provides exactly as | |
| much output space as is the uncompressed size of the file. | |
| - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't | |
| check if the last call to lzma_code() really returned | |
| LZMA_STREAM_END, which made the program think that truncated | |
| files are valid. | |
| - New example programs in doc/examples (old programs are now in | |
| doc/examples_old). These have more comments and more detailed | |
| error handling. | |
| * Fix "xz -lvv foo.xz". It could crash on some corrupted files. | |
| * Fix output of "xz --robot -lv" and "xz --robot -lvv" which | |
| incorrectly printed the filename also in the "foo (x/x)" format. | |
| * Fix exit status of "xzdiff foo.xz bar.xz". | |
| * Fix exit status of "xzgrep foo binary_file". | |
| * Fix portability to EBCDIC systems. | |
| * Fix a configure issue on AIX with the XL C compiler. See INSTALL | |
| for details. | |
| * Update French, German, Italian, and Polish translations. | |
| 5.0.3 (2011-05-21) | |
| * liblzma fixes: | |
| - A memory leak was fixed. | |
| - lzma_stream_buffer_encode() no longer creates an empty .xz | |
| Block if encoding an empty buffer. Such an empty Block with | |
| LZMA2 data would trigger a bug in 5.0.1 and older (see the | |
| first bullet point in 5.0.2 notes). When releasing 5.0.2, | |
| I thought that no encoder creates this kind of files but | |
| I was wrong. | |
| - Validate function arguments better in a few functions. Most | |
| importantly, specifying an unsupported integrity check to | |
| lzma_stream_buffer_encode() no longer creates a corrupt .xz | |
| file. Probably no application tries to do that, so this | |
| shouldn't be a big problem in practice. | |
| - Document that lzma_block_buffer_encode(), | |
| lzma_easy_buffer_encode(), lzma_stream_encoder(), and | |
| lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. | |
| - The return values of the _memusage() functions are now | |
| documented better. | |
| * Fix command name detection in xzgrep. xzegrep and xzfgrep now | |
| correctly use egrep and fgrep instead of grep. | |
| * French translation was added. | |
| 5.0.2 (2011-04-01) | |
| * LZMA2 decompressor now correctly accepts LZMA2 streams with no | |
| uncompressed data. Previously it considered them corrupt. The | |
| bug can affect applications that use raw LZMA2 streams. It is | |
| very unlikely to affect .xz files because no compressor creates | |
| .xz files with empty LZMA2 streams. (Empty .xz files are a | |
| different thing than empty LZMA2 streams.) | |
| * "xz --suffix=.foo filename.foo" now refuses to compress the | |
| file due to it already having the suffix .foo. It was already | |
| documented on the man page, but the code lacked the test. | |
| * "xzgrep -l foo bar.xz" works now. | |
| * Polish translation was added. | |
| 5.0.1 (2011-01-29) | |
| * xz --force now (de)compresses files that have setuid, setgid, | |
| or sticky bit set and files that have multiple hard links. | |
| The man page had it documented this way already, but the code | |
| had a bug. | |
| * gzip and bzip2 support in xzdiff was fixed. | |
| * Portability fixes | |
| * Minor fix to Czech translation | |
| 5.0.0 (2010-10-23) | |
| Only the most important changes compared to 4.999.9beta are listed | |
| here. One change is especially important: | |
| * The memory usage limit is now disabled by default. Some scripts | |
| written before this change may have used --memory=max on xz command | |
| line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED | |
| NOW, because they interfere with user's ability to set the memory | |
| usage limit himself. If user-specified limit causes problems to | |
| your script, blame the user. | |
| Other significant changes: | |
| * Added support for XZ_DEFAULTS environment variable. This variable | |
| allows users to set default options for xz, e.g. default memory | |
| usage limit or default compression level. Scripts that use xz | |
| must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT | |
| instead if they need a way to pass options to xz via an | |
| environment variable. | |
| * The compression settings associated with the preset levels | |
| -0 ... -9 have been changed. --extreme was changed a little too. | |
| It is now less likely to make compression worse, but with some | |
| files the new --extreme may compress slightly worse than the old | |
| --extreme. | |
| * If a preset level (-0 ... -9) is specified after a custom filter | |
| chain options have been used (e.g. --lzma2), the custom filter | |
| chain will be forgotten. Earlier the preset options were | |
| completely ignored after custom filter chain options had been | |
| seen. | |
| * xz will create sparse files when decompressing if the uncompressed | |
| data contains long sequences of binary zeros. This is done even | |
| when writing to standard output that is connected to a regular | |
| file and certain additional conditions are met to make it safe. | |
| * Support for "xz --list" was added. Combine with --verbose or | |
| --verbose --verbose (-vv) for detailed output. | |
| * I had hoped that liblzma API would have been stable after | |
| 4.999.9beta, but there have been a couple of changes in the | |
| advanced features, which don't affect most applications: | |
| - Index handling code was revised. If you were using the old | |
| API, you will get a compiler error (so it's easy to notice). | |
| - A subtle but important change was made to the Block handling | |
| API. lzma_block.version has to be initialized even for | |
| lzma_block_header_decode(). Code that doesn't do it will work | |
| for now, but might break in the future, which makes this API | |
| change easy to miss. | |
| * The major soname has been bumped to 5.0.0. liblzma API and ABI | |
| are now stable, so the need to recompile programs linking against | |
| liblzma shouldn't arise soon. | |