diff options
| author | Xin LI <delphij@FreeBSD.org> | 2022-11-16 05:53:08 +0000 |
|---|---|---|
| committer | Xin LI <delphij@FreeBSD.org> | 2022-11-16 05:53:08 +0000 |
| commit | bbf0e6b90032784aebc8e88024d495a250b0dad5 (patch) | |
| tree | 2a77f49c97c49066a31e20695d3126fe941981ec | |
| parent | b89a9714932b4fb6d8aec10dc79b7ecf9a9ce536 (diff) | |
Vendor import of xz-5.2.8 (trimmed).vendor/xz/5.2.8
44 files changed, 2939 insertions, 220 deletions
@@ -3,7 +3,7 @@ Authors of XZ Utils =================== XZ Utils is developed and maintained by Lasse Collin - <lasse.collin@tukaani.org>. + <lasse.collin@tukaani.org> and Jia Tan <jiat0218@gmail.com>. Major parts of liblzma are based on code written by Igor Pavlov, specifically the LZMA SDK <http://7-zip.org/sdk.html>. Without diff --git a/ChangeLog b/ChangeLog index 37c0d95c4238..de9c32fb2187 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,2123 @@ +commit 5476089d9c42b9b04e92b80e1800b384a98265cb +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-13 19:58:47 +0200 + + Bump version and soname for 5.2.8. + + src/liblzma/Makefile.am | 2 +- + src/liblzma/api/lzma/version.h | 2 +- + 2 files changed, 2 insertions(+), 2 deletions(-) + +commit f9994f395d04d37b75e879f90397258b86b466cc +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-13 19:57:26 +0200 + + Add NEWS for 5.2.8. + + NEWS | 79 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 79 insertions(+) + +commit cdf14b28993ef2f3168821372467e3aa7cf1e973 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-11 17:16:03 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 454f567e58bbadea5cff4f31ef450c1305e9853d +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-11 17:15:25 +0200 + + liblzma: Fix building with Intel ICC (the classic compiler). + + It claims __GNUC__ >= 10 but doesn't support __symver__ attribute. + + Thanks to Stephen Sachs. + + src/liblzma/common/common.h | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 2f01169f5a81e21e7a7e5f799c32472c6277b1d5 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-11 14:35:58 +0200 + + liblzma: Fix incorrect #ifdef for x86 SSE2 support. + + __SSE2__ is the correct macro for SSE2 support with GCC, Clang, + and ICC. __SSE2_MATH__ means doing floating point math with SSE2 + instead of 387. Often the latter macro is defined if the first + one is but it was still a bug. + + src/liblzma/common/memcmplen.h | 3 +-- + 1 file changed, 1 insertion(+), 2 deletions(-) + +commit fc1358679ed3994f67e5e8c869c281be82370a32 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-11 12:23:58 +0200 + + Scripts: Ignore warnings from xz. + + In practice this means making the scripts work when + the input files have an unsupported check type which + isn't a problem in practice unless support for + some check types has been disabled at build time. + + src/scripts/xzdiff.in | 5 +++-- + src/scripts/xzgrep.in | 2 +- + src/scripts/xzless.in | 4 ++-- + src/scripts/xzmore.in | 4 ++-- + 4 files changed, 8 insertions(+), 7 deletions(-) + +commit a08be1c420501b1c75770a8f6103e3959486c895 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-09 14:10:52 +0200 + + xz: Add comments about stdin and src_st.st_size. + + "xz -v < regular_file > out.xz" doesn't display the percentage + and estimated remaining time because it doesn't even try to + check the input file size when input is read from stdin. + This could be improved but for now there's just a comment + to remind about it. + + src/xz/coder.c | 9 +++++++++ + src/xz/file_io.c | 4 ++++ + 2 files changed, 13 insertions(+) + +commit 3ee411cd1cd636bdced6ecede4651394bb4f9cb1 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-09 12:48:22 +0200 + + xz: Fix displaying of file sizes in progress indicator in passthru mode. + + It worked for one input file since the counters are zero when + xz starts but they weren't reset when starting a new file in + passthru mode. For example, if files A, B, and C are one byte each, + then "xz -dcvf A B C" would show file sizes as 1, 2, and 3 bytes + instead of 1, 1, and 1 byte. + + src/xz/coder.c | 6 +++++- + 1 file changed, 5 insertions(+), 1 deletion(-) + +commit aa7fa9d960df4bb21f7727bbf7ce0503f3d211b4 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-09 11:27:20 +0200 + + xz: Add a comment why --to-stdout is not in --help. + + It is on the man page still. + + src/xz/message.c | 3 +++ + 1 file changed, 3 insertions(+) + +commit ff49ff84a4ba3fab25ffb6d65af321dee36c1273 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-08 22:26:54 +0200 + + Docs: Update faq.txt a little. + + doc/faq.txt | 66 ++++++++++++++++++++++++++++++++++++++++--------------------- + 1 file changed, 43 insertions(+), 23 deletions(-) + +commit 3489565b75b45ee514cf0a6785bb721c7dc624f3 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-08 14:13:03 +0200 + + liblzma: Update API docs about decoder flags. + + src/liblzma/api/lzma/container.h | 20 ++++++++++++++++---- + 1 file changed, 16 insertions(+), 4 deletions(-) + +commit e4937710805c4b660fc629c15d9a6164b2d97822 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-08 14:00:58 +0200 + + liblzma: Fix a comment in auto_decoder.c. + + src/liblzma/common/auto_decoder.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +commit d4674dfbb7d1df1feb841f5dbce6ae1f0b026879 +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-11-07 16:24:14 +0200 + + xz: Avoid a compiler warning in progress_speed() in message.c. + + This should be smaller too since it avoids the string constants. + + src/xz/message.c | 9 +++------ + 1 file changed, 3 insertions(+), 6 deletions(-) + +commit 4ed56d32a91ff8ff21c71db4082a54a72ae08b3d +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-31 16:26:05 +0200 + + Build: Clarify comment in configure.ac about SSE2. + + configure.ac | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +commit f930638797f1fec05b272367c052d4fc9de35896 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-31 16:16:37 +0200 + + Build: Remove obsolete commented-out lines from configure.ac. + + configure.ac | 4 ---- + 1 file changed, 4 deletions(-) + +commit 6930f14733757592a4f8d92513f642a5dcc9ec9c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-31 13:31:58 +0200 + + Windows: Fix mythread_once() macro with Vista threads. + + Don't call InitOnceComplete() if initialization was already done. + + So far mythread_once() has been needed only when building + with --enable-small. windows/build.bash does this together + with --disable-threads so the Vista-specific mythread_once() + is never needed by those builds. VS project files or + CMake-builds don't support HAVE_SMALL builds at all. + + src/common/mythread.h | 7 ++++--- + 1 file changed, 4 insertions(+), 3 deletions(-) + +commit 1c8cbb5be3a97bf22e933777369c4cc28ef9ed5f +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-31 11:49:47 +0200 + + CMake: Sync tuklib_cpucores.cmake with tuklib_cpucores.m4. + + This was forgotten from commit 2611c4d90535652d3eb7ef4a026a6691276fab43. + + cmake/tuklib_cpucores.cmake | 5 +++++ + 1 file changed, 5 insertions(+) + +commit fa9efb729b1827b25ec71aefcf32f3aceeaf9597 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-25 23:45:03 +0300 + + Build: Use AC_CONFIG_HEADERS instead of the ancient AC_CONFIG_HEADER. + + We require Autoconf >= 2.69 and that has AC_CONFIG_HEADERS. + + There is a warning about AC_PROG_CC_C99 being obsolete but + it cannot be removed because it is needed with Autoconf 2.69. + + configure.ac | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit b10ba4bf39edd9c9fef42cade3e64fc3b9e3fd19 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-25 23:31:44 +0300 + + Build: Update m4/ax_pthread.m4 from Autoconf Archive. + + m4/ax_pthread.m4 | 29 ++++++++++++++++++++++------- + 1 file changed, 22 insertions(+), 7 deletions(-) + +commit 01744b280c6e35d331b9dc4cc64716b82bc8b3db +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-25 23:09:11 +0300 + + xz: Fix --single-stream with an empty .xz Stream. + + Example: + + $ xz -dc --single-stream good-0-empty.xz + xz: good-0-empty.xz: Internal error (bug) + + The code, that is tries to catch some input file issues early, + didn't anticipate LZMA_STREAM_END which is possible in that + code only when --single-stream is used. + + src/xz/coder.c | 9 +++++++++ + 1 file changed, 9 insertions(+) + +commit a3e4606134058241bf1c396243bebcec42c2a780 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-25 19:07:17 +0300 + + xz: Fix decompressor behavior if input uses an unsupported check type. + + Now files with unsupported check will make xz display + a warning, set the exit status to 2 (unless --no-warn is used), + and then decompress the file normally. This is how it was + supposed to work since the beginning but this was broken by + the commit 231c3c7098f1099a56abb8afece76fc9b8699f05, that is, + a little before 5.0.0 was released. The buggy behavior displayed + a message, set exit status 1 (error), and xz didn't attempt to + to decompress the file. + + This doesn't matter today except for special builds that disable + CRC64 or SHA-256 at build time (but such builds should be used + in special situations only). The bug matters if new check type + is added in the future and an old xz version is used to decompress + such a file; however, it's likely that such files would use a new + filter too and an old xz wouldn't be able to decompress the file + anyway. + + The first hunk in the commit is the actual fix. The second hunk + is a cleanup since LZMA_TELL_ANY_CHECK isn't used in xz. + + There is a test file for unsupported check type but it wasn't + used by test_files.sh, perhaps due to different behavior between + xz and the simpler xzdec. + + src/xz/coder.c | 19 +++++++++++++++---- + 1 file changed, 15 insertions(+), 4 deletions(-) + +commit 0b5e8c7e073bce07c59112af1e8a7495e3576e6c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-25 18:36:19 +0300 + + xz: Clarify the man page: input file isn't removed if an error occurs. + + src/xz/xz.1 | 5 +++-- + 1 file changed, 3 insertions(+), 2 deletions(-) + +commit 23b7416d5ba0e3579265467bd546865d2ea4cef0 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-25 18:23:54 +0300 + + xz: If input file cannot be removed, treat it as a warning, not error. + + Treating it as a warning (message + exit status 2) matches gzip + and it seems more logical as at that point the output file has + already been successfully closed. When it's a warning it is + possible to suppress it with --no-warn. + + src/xz/file_io.c | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +commit 5daa40454b63c4eb04ab5caeb339eddc95c94bb1 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-10-20 20:22:50 +0300 + + tuklib_cpucores: Use HW_NCPUONLINE on OpenBSD. + + On OpenBSD the number of cores online is often less + than what HW_NCPU would return because OpenBSD disables + simultaneous multi-threading (SMT) by default. + + Thanks to Christian Weisgerber. + + m4/tuklib_cpucores.m4 | 5 +++++ + src/common/tuklib_cpucores.c | 9 +++++++++ + 2 files changed, 14 insertions(+) + +commit 0af861050f375678eafc4e1136ca589ae47d31c0 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-11 13:25:02 +0200 + + NEWS: Omit the extra copy of 5.2.5 NEWS. + + It was a copy-paste error. + + NEWS | 105 ------------------------------------------------------------------- + 1 file changed, 105 deletions(-) + +commit f0c6a66701afed12c5926c0499b150ecd5b0a63c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-10 12:34:43 +0200 + + Translations: Rename poa4/fr_FR.po to po4a/fr.po. + + That's how it is preferred at the Translation Project. + On my system /usr/share/man/fr_FR doesn't contain any + other man pages than XZ Utils while /usr/share/man/fr + has quite a few, so this will fix that too. + + Thanks to Benno Schulenberg from the Translation Project. + + po4a/{fr_FR.po => fr.po} | 0 + po4a/po4a.conf | 2 +- + 2 files changed, 1 insertion(+), 1 deletion(-) + +commit 6bf8b1f870870a1a0c3f555c34decc9ffb9f8d7a +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-08 16:57:17 +0200 + + Translations: Update Turkish translation. + + po/tr.po | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 9f8e9d3c8189d8c6f587c99b0da1cf83f72330c2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-11-08 14:55:32 +0200 + + Translations: Update Croatian translation. + + po/hr.po | 190 ++++++++++++++++++++++++++++++++------------------------------- + 1 file changed, 96 insertions(+), 94 deletions(-) + +commit d24a57b7fc7e5e9267b84367cb0788d3acf7f569 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-30 16:41:03 +0300 + + Bump version and soname for 5.2.7. + + src/liblzma/Makefile.am | 2 +- + src/liblzma/api/lzma/version.h | 2 +- + 2 files changed, 2 insertions(+), 2 deletions(-) + +commit d2003362dd42004355a5550ee7a60ace3b3ea2d8 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-30 16:40:39 +0300 + + Add NEWS for 5.2.7. + + NEWS | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 74 insertions(+) + +commit 369afb51991e91a5c3a23c4ae3872329a403eb31 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-30 12:06:13 +0300 + + liblzma: Add API doc note about the .xz decoder LZMA_MEMLIMIT_ERROR bug. + + The bug was fixed in 660739f99ab211edec4071de98889fb32ed04e98. + + src/liblzma/api/lzma/base.h | 11 +++++++++++ + 1 file changed, 11 insertions(+) + +commit 166431e9951613481b8136f52b570d7bc30b5a03 +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-09-21 16:15:50 +0800 + + liblzma: Add dest and src NULL checks to lzma_index_cat. + + The documentation states LZMA_PROG_ERROR can be returned from + lzma_index_cat. Previously, lzma_index_cat could not return + LZMA_PROG_ERROR. Now, the validation is similar to + lzma_index_append, which does a NULL check on the index + parameter. + + src/liblzma/common/index.c | 3 +++ + 1 file changed, 3 insertions(+) + +commit 5e53a6c28b87a0bb9b0cfeaf708cc99ef8e9bc66 +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-09-21 20:29:28 +0800 + + Tests: Create a test for the lzma_index_cat bug. + + tests/test_index.c | 43 ++++++++++++++++++++++++++++++++++++++++++- + 1 file changed, 42 insertions(+), 1 deletion(-) + +commit 4ed5fd54c676b735e3f58158f214af68755d59b5 +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-09-21 19:28:53 +0800 + + liblzma: Fix copying of check type statistics in lzma_index_cat(). + + The check type of the last Stream in dest was never copied to + dest->checks (the code tried to copy it but it was done too late). + This meant that the value returned by lzma_index_checks() would + only include the check type of the last Stream when multiple + lzma_indexes had been concatenated. + + In xz --list this meant that the summary would only list the + check type of the last Stream, so in this sense this was only + a visual bug. However, it's possible that some applications + use this information for purposes other than merely showing + it to the users in an informational message. I'm not aware of + such applications though and it's quite possible that such + applications don't exist. + + Regular streamed decompression in xz or any other application + doesn't use lzma_index_cat() and so this bug cannot affect them. + + src/liblzma/common/index.c | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +commit c4476f695291a2d7376d5fa406d32e354f858858 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-28 12:20:41 +0300 + + tuklib_physmem: Fix Unicode builds on Windows. + + Thanks to ArSaCiA Game. + + src/common/tuklib_physmem.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 976f897bbba9890a99f01a8d59c6d59c230142e6 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-28 11:05:15 +0300 + + liblzma: Stream decoder: Fix restarting after LZMA_MEMLIMIT_ERROR. + + If lzma_code() returns LZMA_MEMLIMIT_ERROR it is now possible + to use lzma_memlimit_set() to increase the limit and continue + decoding. This was supposed to work from the beginning but + there was a bug. With other decoders (.lzma or threaded .xz) + this already worked correctly. + + src/liblzma/common/stream_decoder.c | 16 +++++++++++++--- + 1 file changed, 13 insertions(+), 3 deletions(-) + +commit 2caa9580e5f7c5cc6668e9427a2c21f3f63bb684 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-28 11:00:23 +0300 + + liblzma: Stream decoder: Fix comments. + + src/liblzma/common/stream_decoder.c | 12 +++++------- + 1 file changed, 5 insertions(+), 7 deletions(-) + +commit 51882fec5b96cf7760389b670d86c1e7a5a0ced2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-16 14:09:07 +0300 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 974186f7cd839f1dbf937b98e112dff670d2bd5c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-16 14:07:03 +0300 + + xzgrep: Fix compatibility with old shells. + + Running the current xzgrep on Slackware 10.1 with GNU bash 3.00.15: + + xzgrep: line 231: syntax error near unexpected token `;;' + + On SCO OpenServer 5.0.7 with Korn Shell 93r: + + syntax error at line 231 : `;;' unexpected + + Turns out that some old shells don't like apostrophes (') inside + command substitutions. For example, the following fails: + + x=$(echo foo + # asdf'zxcv + echo bar) + printf '%s\n' "$x" + + The problem was introduced by commits + 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 (2022-03-29), + bd7b290f3fe4faeceb7d3497ed9bf2e6ed5e7dc5 (2022-07-18), and + a648978b20495b7aa4a8b029c5a810b5ad9d08ff (2022-07-19). + 5.2.6 is the only stable release that included + this problem. + + Thanks to Kevin R. Bulgrien for reporting the problem + on SCO OpenServer 5.0.7 and for providing the fix. + + src/scripts/xzgrep.in | 6 +++--- + 1 file changed, 3 insertions(+), 3 deletions(-) + +commit f94da15120c3d3c363ca12c2262ac6cb9f321f4f +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-09 13:51:57 +0300 + + liblzma: lzma_filters_copy: Keep dest[] unmodified if an error occurs. + + lzma_stream_encoder() and lzma_stream_encoder_mt() always assumed + this. Before this patch, failing lzma_filters_copy() could result + in free(invalid_pointer) or invalid memory reads in stream_encoder.c + or stream_encoder_mt.c. + + To trigger this, allocating memory for a filter options structure + has to fail. These are tiny allocations so in practice they very + rarely fail. + + Certain badness in the filter chain array could also make + lzma_filters_copy() fail but both stream_encoder.c and + stream_encoder_mt.c validate the filter chain before + trying to copy it, so the crash cannot occur this way. + + src/liblzma/api/lzma/filter.h | 4 +++- + src/liblzma/common/filter_common.c | 18 ++++++++++++------ + 2 files changed, 15 insertions(+), 7 deletions(-) + +commit ea57b9aa2c3e1cdb667f8dd698314b1c36047018 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-16 17:08:53 +0300 + + Tests: Add a test file for lzma_index_append() integer overflow bug. + + This test fails before commit 18d7facd3802b55c287581405c4d49c98708c136. + + test_files.sh now runs xz -l for bad-3-index-uncomp-overflow.xz + because only then the previously-buggy code path gets tested. + Normal decompression doesn't use lzma_index_append() at all. + Instead, lzma_index_hash functions are used and those already + did the overflow check. + + tests/files/README | 10 ++++++++++ + tests/files/bad-3-index-uncomp-overflow.xz | Bin 0 -> 132 bytes + tests/test_files.sh | 8 ++++++++ + 3 files changed, 18 insertions(+) + +commit 72e1645a439a999638a63ec9abb7210671ea5415 +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-09-02 20:18:55 +0800 + + liblzma: lzma_index_append: Add missing integer overflow check. + + The documentation in src/liblzma/api/lzma/index.h suggests that + both the unpadded (compressed) size and the uncompressed size + are checked for overflow, but only the unpadded size was checked. + The uncompressed check is done first since that is more likely to + occur than the unpadded or index field size overflows. + + src/liblzma/common/index.c | 4 ++++ + 1 file changed, 4 insertions(+) + +commit 20d82bc90781aa9d2373efe872d90ba64dc7dc62 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-08 15:11:08 +0300 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 31d80c6b261b24220776dfaeb8a04f80f80e0a24 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-04 23:23:00 +0300 + + liblzma: Vaccinate against an ill patch from RHEL/CentOS 7. + + RHEL/CentOS 7 shipped with 5.1.2alpha, including the threaded + encoder that is behind #ifdef LZMA_UNSTABLE in the API headers. + In 5.1.2alpha these symbols are under XZ_5.1.2alpha in liblzma.map. + API/ABI compatibility tracking isn't done between development + releases so newer releases didn't have XZ_5.1.2alpha anymore. + + Later RHEL/CentOS 7 updated xz to 5.2.2 but they wanted to keep + the exported symbols compatible with 5.1.2alpha. After checking + the ABI changes it turned out that >= 5.2.0 ABI is backward + compatible with the threaded encoder functions from 5.1.2alpha + (but not vice versa as fixes and extensions to these functions + were made between 5.1.2alpha and 5.2.0). + + In RHEL/CentOS 7, XZ Utils 5.2.2 was patched with + xz-5.2.2-compat-libs.patch to modify liblzma.map: + + - XZ_5.1.2alpha was added with lzma_stream_encoder_mt and + lzma_stream_encoder_mt_memusage. This matched XZ Utils 5.1.2alpha. + + - XZ_5.2 was replaced with XZ_5.2.2. It is clear that this was + an error; the intention was to keep using XZ_5.2 (XZ_5.2.2 + has never been used in XZ Utils). So XZ_5.2.2 lists all + symbols that were listed under XZ_5.2 before the patch. + lzma_stream_encoder_mt and _mt_memusage are included too so + they are listed both here and under XZ_5.1.2alpha. + + The patch didn't add any __asm__(".symver ...") lines to the .c + files. Thus the resulting liblzma.so exports the threaded encoder + functions under XZ_5.1.2alpha only. Listing the two functions + also under XZ_5.2.2 in liblzma.map has no effect without + matching .symver lines. + + The lack of XZ_5.2 in RHEL/CentOS 7 means that binaries linked + against unpatched XZ Utils 5.2.x won't run on RHEL/CentOS 7. + This is unfortunate but this alone isn't too bad as the problem + is contained within RHEL/CentOS 7 and doesn't affect users + of other distributions. It could also be fixed internally in + RHEL/CentOS 7. + + The second problem is more serious: In XZ Utils 5.2.2 the API + headers don't have #ifdef LZMA_UNSTABLE for obvious reasons. + This is true in RHEL/CentOS 7 version too. Thus now programs + using new APIs can be compiled without an extra #define. However, + the programs end up depending on symbol version XZ_5.1.2alpha + (and possibly also XZ_5.2.2) instead of XZ_5.2 as they would + with an unpatched XZ Utils 5.2.2. This means that such binaries + won't run on other distributions shipping XZ Utils >= 5.2.0 as + they don't provide XZ_5.1.2alpha or XZ_5.2.2; they only provide + XZ_5.2 (and XZ_5.0). (This includes RHEL/CentOS 8 as the patch + luckily isn't included there anymore with XZ Utils 5.2.4.) + + Binaries built by RHEL/CentOS 7 users get distributed and then + people wonder why they don't run on some other distribution. + Seems that people have found out about the patch and been copying + it to some build scripts, seemingly curing the symptoms but + actually spreading the illness further and outside RHEL/CentOS 7. + + The ill patch seems to be from late 2016 (RHEL 7.3) and in 2017 it + had spread at least to EasyBuild. I heard about the events only + recently. :-( + + This commit splits liblzma.map into two versions: one for + GNU/Linux and another for other OSes that can use symbol versioning + (FreeBSD, Solaris, maybe others). The Linux-specific file and the + matching additions to .c files add full compatibility with binaries + that have been built against a RHEL/CentOS-patched liblzma. Builds + for OSes other than GNU/Linux won't get the vaccine as they should + be immune to the problem (I really hope that no build script uses + the RHEL/CentOS 7 patch outside GNU/Linux). + + The RHEL/CentOS compatibility symbols XZ_5.1.2alpha and XZ_5.2.2 + are intentionally put *after* XZ_5.2 in liblzma_linux.map. This way + if one forgets to #define HAVE_SYMBOL_VERSIONS_LINUX when building, + the resulting liblzma.so.5 will have lzma_stream_encoder_mt@@XZ_5.2 + since XZ_5.2 {...} is the first one that lists that function. + Without HAVE_SYMBOL_VERSIONS_LINUX @XZ_5.1.2alpha and @XZ_5.2.2 + will be missing but that's still a minor problem compared to + only having lzma_stream_encoder_mt@@XZ_5.1.2alpha! + + The "local: *;" line was moved to XZ_5.0 so that it doesn't need + to be moved around. It doesn't matter where it is put. + + Having two similar liblzma_*.map files is a bit silly as it is, + at least for now, easily possible to generate the generic one + from the Linux-specific file. But that adds extra steps and + increases the risk of mistakes when supporting more than one + build system. So I rather maintain two files in parallel and let + validate_map.sh check that they are in sync when "make mydist" + is run. + + This adds .symver lines for lzma_stream_encoder_mt@XZ_5.2.2 and + lzma_stream_encoder_mt_memusage@XZ_5.2.2 even though these + weren't exported by RHEL/CentOS 7 (only @@XZ_5.1.2alpha was + for these two). I added these anyway because someone might + misunderstand the RHEL/CentOS 7 patch and think that @XZ_5.2.2 + (@@XZ_5.2.2) versions were exported too. + + At glance one could suggest using __typeof__ to copy the function + prototypes when making aliases. However, this doesn't work trivially + because __typeof__ won't copy attributes (lzma_nothrow, lzma_pure) + and it won't change symbol visibility from hidden to default (done + by LZMA_API()). Attributes could be copied with __copy__ attribute + but that needs GCC 9 and a fallback method would be needed anyway. + + This uses __symver__ attribute with GCC >= 10 and + __asm__(".symver ...") with everything else. The attribute method + is required for LTO (-flto) support with GCC. Using -flto with + GCC older than 10 is now broken on GNU/Linux and will not be fixed + (can silently result in a broken liblzma build that has dangerously + incorrect symbol versions). LTO builds with Clang seem to work + with the traditional __asm__(".symver ...") method. + + Thanks to Boud Roukema for reporting the problem and discussing + the details and testing the fix. + + configure.ac | 23 ++++- + src/liblzma/Makefile.am | 10 +- + src/liblzma/common/block_buffer_encoder.c | 18 ++++ + src/liblzma/common/common.c | 14 +++ + src/liblzma/common/common.h | 28 ++++++ + src/liblzma/common/hardware_cputhreads.c | 12 +++ + src/liblzma/common/stream_encoder_mt.c | 42 ++++++++ + src/liblzma/{liblzma.map => liblzma_generic.map} | 6 +- + src/liblzma/liblzma_linux.map | 123 +++++++++++++++++++++++ + src/liblzma/validate_map.sh | 113 +++++++++++++++++++-- + 10 files changed, 374 insertions(+), 15 deletions(-) + +commit e7a7ac744eb0f890ef52388de838596ef566c73f +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-09-08 15:07:00 +0300 + + CMake: Clarify a comment about Windows symlinks without file extension. + + CMakeLists.txt | 7 +++---- + 1 file changed, 3 insertions(+), 4 deletions(-) + +commit a273a0cb77c6000e4da627b07f28c5af41f0908c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-08 15:02:41 +0300 + + CMake: Update for liblzma_*.map files and fix wrong common_w32res.rc dep. + + The previous commit split liblzma.map into liblzma_linux.map and + liblzma_generic.map. This commit updates the CMake build for those. + + common_w32res.rc dependency was listed under Linux/FreeBSD while + obviously it belongs to Windows when building a DLL. + + CMakeLists.txt | 24 +++++++++++++++++++----- + 1 file changed, 19 insertions(+), 5 deletions(-) + +commit 5875a45be0ab399e37acc31165725a9b5291f67e +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-31 16:42:04 +0300 + + CMake: Add xz symlinks. + + These are a minor thing especially since the xz build has + some real problems still like lack of large file support + on 32-bit systems but I'll commit this since the code exists. + + Thanks to Jia Tan. + + CMakeLists.txt | 38 +++++++++++++++++++++++++++++++++++++- + 1 file changed, 37 insertions(+), 1 deletion(-) + +commit 3523b6ebb56c447f21736f01bfe91602741aa3e7 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-31 16:29:38 +0300 + + CMake: Put xz man page install under if(UNIX) like is for xzdec. + + Thanks to Jia Tan. + + CMakeLists.txt | 8 +++++--- + 1 file changed, 5 insertions(+), 3 deletions(-) + +commit 5af9e8759f748a11a408bfc5aef829df3811fc23 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-09-16 15:10:07 +0300 + + Translations: Add Turkish translation. + + po/LINGUAS | 1 + + po/tr.po | 977 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 978 insertions(+) + +commit f05a69685e9c7f7157d751abfbd2d8ea6e3bacc7 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-18 17:49:16 +0300 + + Build: Include the CMake files in the distribution. + + This was supposed to be done in 2020 with 5.2.5 release + already but it was noticed only today. 5.2.5 and 5.2.6 + even mention experiemental CMake support in the NEWS entries. + + Thanks to Olivier B. for reporting the problem. + + Makefile.am | 2 ++ + 1 file changed, 2 insertions(+) + +commit ad5ef6d3c30bb749113c51aa0bbe8bcb5e193fbe +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-18 17:38:05 +0300 + + Windows: Fix broken liblzma.dll build with Visual Studio project files. + + The bug was introduced in 352ba2d69af2136bc814aa1df1a132559d445616 + "Windows: Fix building of resource files when config.h isn't used." + + That commit fixed liblzma.dll build with CMake while keeping it + working with Autotools on Windows but the VS project files were + forgotten. + + I haven't tested these changes. + + Thanks to Olivier B. for reporting the bug and for the initial patch. + + windows/vs2013/liblzma_dll.vcxproj | 6 ++++++ + windows/vs2017/liblzma_dll.vcxproj | 6 ++++++ + windows/vs2019/liblzma_dll.vcxproj | 6 ++++++ + 3 files changed, 18 insertions(+) + +commit 8dfed05bdaa4873833ba24279f02ad2db25effea +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-12 14:30:13 +0300 + + Bump version and soname for 5.2.6. + + src/liblzma/Makefile.am | 2 +- + src/liblzma/api/lzma/version.h | 2 +- + 2 files changed, 2 insertions(+), 2 deletions(-) + +commit 09b4af4e04def5777b06124200b64b0b941eea7f +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-12 14:29:28 +0300 + + Add NEWS for 5.2.6. + + NEWS | 226 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 226 insertions(+) + +commit 692de534fae6c38f92416793addcf5f60ffe2bf6 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-08-12 14:28:41 +0300 + + Add Jia Tan to AUTHORS. + + AUTHORS | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 275de376a645f406be7c453bb68427aa9466cf83 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-25 19:11:05 +0300 + + Translations: Change the copyright comment string to use with po4a. + + This affects the second line in po4a/xz-man.pot. The man pages of + xzdiff, xzgrep, and xzmore are from GNU gzip and under GNU GPLv2+ + while the rest of the man pages are in the public domain. + + po4a/update-po | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 76a5a752b8467ff591dd028deb61e9bf2c274c7e +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-07-25 18:30:05 +0300 + + liblzma: Refactor lzma_mf_is_supported() to use a switch-statement. + + src/liblzma/lz/lz_encoder.c | 32 ++++++++++++++------------------ + 1 file changed, 14 insertions(+), 18 deletions(-) + +commit 749b86c2c18ab61a07f19ec8fefc67325da97397 +Author: Jia Tan <jiat0218@gmail.com> +Date: 2022-07-25 18:20:01 +0300 + + Build: Don't allow empty LIST in --enable-match-finders=LIST. + + It's enforced only when a match finder is needed, that is, + when LZMA1 or LZMA2 encoder is enabled. + + configure.ac | 4 ++++ + 1 file changed, 4 insertions(+) + +commit 63e3cdef805a4d9ea94c76c36cbabe4dc76abe36 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-11 23:41:16 +0200 + + xz: Make --keep accept symlinks, hardlinks, and setuid/setgid/sticky. + + Previously this required using --force but that has other + effects too which might be undesirable. Changing the behavior + of --keep has a small risk of breaking existing scripts but + since this is a fairly special corner case I expect the + likehood of breakage to be low enough. + + I think the new behavior is more logical. The only reason for + the old behavior was to be consistent with gzip and bzip2. + + Thanks to Vincent Lefevre and Sebastian Andrzej Siewior. + + src/xz/file_io.c | 9 +++++---- + src/xz/xz.1 | 16 +++++++++++++++- + 2 files changed, 20 insertions(+), 5 deletions(-) + +commit 9055584be0ae47032b7fb3ba8531aa56b9ed572a +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-19 23:19:49 +0300 + + xzgrep man page: Document exit statuses. + + src/scripts/xzgrep.1 | 15 ++++++++++++++- + 1 file changed, 14 insertions(+), 1 deletion(-) + +commit 57e1ccbb7ce30e5298ea7ec3934e462fa8d32237 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-19 23:13:24 +0300 + + xzgrep: Improve error handling, especially signals. + + xzgrep wouldn't exit on SIGPIPE or SIGQUIT when it clearly + should have. It's quite possible that it's not perfect still + but at least it's much better. + + If multiple exit statuses compete, now it tries to pick + the largest of value. + + Some comments were added. + + The exit status handling of signals is still broken if the shell + uses values larger than 255 in $? to indicate that a process + died due to a signal ***and*** their "exit" command doesn't take + this into account. This seems to work well with the ksh and yash + versions I tried. However, there is a report in gzip/zgrep that + OpenSolaris 5.11 (not 5.10) has a problem with "exit" truncating + the argument to 8 bits: + + https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22900#25 + + Such a bug would break xzgrep but I didn't add a workaround + at least for now. 5.11 is old and I don't know if the problem + exists in modern descendants, or if the problem exists in other + ksh implementations in use. + + src/scripts/xzgrep.in | 72 +++++++++++++++++++++++++++++++++++++-------------- + 1 file changed, 53 insertions(+), 19 deletions(-) + +commit 6351ea1afb261ec9be95f808bcbe724e0da4ae09 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-19 00:10:55 +0300 + + xzgrep: Make the fix for ZDI-CAN-16587 more robust. + + I don't know if this can make a difference in the real world + but it looked kind of suspicious (what happens with sed + implementations that cannot process very long lines?). + At least this commit shouldn't make it worse. + + src/scripts/xzgrep.in | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +commit 2c1ff2ed6b9f17b3913866a15d776ebbbeb824d2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-18 21:52:31 +0300 + + xzgrep: Use grep -H --label when available (GNU, *BSDs). + + It avoids the use of sed for prefixing filenames to output lines. + Using sed for that is slower and prone to security bugs so now + the sed method is only used as a fallback. + + This also fixes an actual bug: When grepping a binary file, + GNU grep nowadays prints its diagnostics to stderr instead of + stdout and thus the sed-method for prefixing the filename doesn't + work. So with this commit grepping binary files gives reasonable + output with GNU grep now. + + This was inspired by zgrep but the implementation is different. + + src/scripts/xzgrep.in | 21 +++++++++++++++++++++ + 1 file changed, 21 insertions(+) + +commit 8b0be38a795fa1cdffdada64341a34209e630060 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-18 21:10:25 +0300 + + xzgrep: Use -e to specify the pattern to grep. + + Now we don't need the separate test for adding the -q option + as it can be added directly in the two places where it's needed. + + src/scripts/xzgrep.in | 12 ++++-------- + 1 file changed, 4 insertions(+), 8 deletions(-) + +commit 4a61867a874c60b31448d2d4f3c276efba85a243 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-18 19:18:48 +0300 + + Scripts: Use printf instead of echo in a few places. + + It's a good habbit as echo has some portability corner cases + when the string contents can be anything. + + src/scripts/xzdiff.in | 6 +++--- + src/scripts/xzgrep.in | 4 ++-- + src/scripts/xzless.in | 4 ++-- + src/scripts/xzmore.in | 8 ++++---- + 4 files changed, 11 insertions(+), 11 deletions(-) + +commit 0e222bf7d79a9792025c5ffeaa66c8e14eb47615 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-17 21:36:25 +0300 + + xzgrep: Add more LC_ALL=C to avoid bugs with multibyte characters. + + Also replace one use of expr with printf. + + The rationale for LC_ALL=C was already mentioned in + 69d1b3fc29677af8ade8dc15dba83f0589cb63d6 that fixed a security + issue. However, unrelated uses weren't changed in that commit yet. + + POSIX says that with sed and such tools one should use LC_ALL=C + to ensure predictable behavior when strings contain byte sequences + that aren't valid multibyte characters in the current locale. See + under "Application usage" in here: + + https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html + + With GNU sed invalid multibyte strings would work without this; + it's documented in its Texinfo manual. Some other implementations + aren't so forgiving. + + src/scripts/xzgrep.in | 14 ++++++++------ + 1 file changed, 8 insertions(+), 6 deletions(-) + +commit 62c1d2bc2da66e182c52bb1e3aa791f6d8a9e099 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-17 20:55:16 +0300 + + xzgrep: Fix parsing of certain options. + + Fix handling of "xzgrep -25 foo" (in GNU grep "grep -25 foo" is + an alias for "grep -C25 foo"). xzgrep would treat "foo" as filename + instead of as a pattern. This bug was fixed in zgrep in gzip in 2012. + + Add -E, -F, -G, and -P to the "no argument required" list. + + Add -X to "argument required" list. It is an + intentionally-undocumented GNU grep option so this isn't + an important option for xzgrep but it seems that other grep + implementations (well, those that I checked) don't support -X + so I hope this change is an improvement still. + + grep -d (grep --directories=ACTION) requires an argument. In + contrast to zgrep, I kept -d in the "no argument required" list + because it's not supported in xzgrep (or zgrep). This way + "xzgrep -d" gives an error about option being unsupported instead + of telling that it requires an argument. Both zgrep and xzgrep + tell that it's unsupported if an argument is specified. + + Add comments. + + src/scripts/xzgrep.in | 19 +++++++++++++++++-- + 1 file changed, 17 insertions(+), 2 deletions(-) + +commit 372a0d12c95fad5e5477b4f79646bf66d28e0c31 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-14 20:33:05 +0300 + + Tests: Add the .lzma files to test_files.sh. + + tests/test_files.sh | 20 ++++++++++++++++++++ + 1 file changed, 20 insertions(+) + +commit b8e3d0c45b62c81dfc6b350556d77cf2c7401edd +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-14 19:37:42 +0300 + + Tests: Add .lzma test files. + + tests/files/README | 63 ++++++++++++++++----- + tests/files/bad-too_big_size-with_eopm.lzma | Bin 0 -> 37 bytes + tests/files/bad-too_small_size-without_eopm-1.lzma | Bin 0 -> 31 bytes + tests/files/bad-too_small_size-without_eopm-2.lzma | Bin 0 -> 31 bytes + tests/files/bad-too_small_size-without_eopm-3.lzma | Bin 0 -> 36 bytes + tests/files/bad-unknown_size-without_eopm.lzma | Bin 0 -> 31 bytes + tests/files/good-known_size-with_eopm.lzma | Bin 0 -> 37 bytes + tests/files/good-known_size-without_eopm.lzma | Bin 0 -> 31 bytes + tests/files/good-unknown_size-with_eopm.lzma | Bin 0 -> 37 bytes + 9 files changed, 50 insertions(+), 13 deletions(-) + +commit e96bdf7189916bffec0dcc933cd809fac898587c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-14 18:12:38 +0300 + + liblzma: Rename a variable and improve a comment. + + src/liblzma/lzma/lzma_decoder.c | 13 +++++++++---- + 1 file changed, 9 insertions(+), 4 deletions(-) + +commit 2d54fdf58e80bb2b0852633838576fd1ae7961c5 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-13 22:24:41 +0300 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit ff54b557fe3b06b59a24e4028c1464c35dd5b142 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-13 22:24:07 +0300 + + liblzma: Add optional autodetection of LZMA end marker. + + Turns out that this is needed for .lzma files as the spec in + LZMA SDK says that end marker may be present even if the size + is stored in the header. Such files are rare but exist in the + real world. The code in liblzma is so old that the spec didn't + exist in LZMA SDK back then and I had understood that such + files weren't possible (the lzma tool in LZMA SDK didn't + create such files). + + This modifies the internal API so that LZMA decoder can be told + if EOPM is allowed even when the uncompressed size is known. + It's allowed with .lzma and not with other uses. + + Thanks to Karl Beldan for reporting the problem. + + doc/lzma-file-format.txt | 11 ++++- + src/liblzma/common/alone_decoder.c | 2 +- + src/liblzma/lz/lz_decoder.c | 10 +++- + src/liblzma/lz/lz_decoder.h | 8 +-- + src/liblzma/lzma/lzma2_decoder.c | 2 +- + src/liblzma/lzma/lzma_decoder.c | 99 +++++++++++++++++++++++++++----------- + 6 files changed, 94 insertions(+), 38 deletions(-) + +commit bb795fe835cf2b3ae7ed68b155784fbfc14e2faf +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-14 17:20:49 +0300 + + Tests: Add test file good-1-empty-bcj-lzma2.xz. + + This is from test_bcj_exact_size.c. + It's good to have it as a standalone file. + + tests/files/README | 5 +++++ + tests/files/good-1-empty-bcj-lzma2.xz | Bin 0 -> 52 bytes + 2 files changed, 5 insertions(+) + +commit dbd8b0bf45c10f991635f585a2ea59fb5ae28ff3 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-03-31 00:05:07 +0300 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit bb66a98ded39c40df202da17cb2055356dcd4168 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-03-29 19:19:12 +0300 + + xzgrep: Fix escaping of malicious filenames (ZDI-CAN-16587). + + Malicious filenames can make xzgrep to write to arbitrary files + or (with a GNU sed extension) lead to arbitrary code execution. + + xzgrep from XZ Utils versions up to and including 5.2.5 are + affected. 5.3.1alpha and 5.3.2alpha are affected as well. + This patch works for all of them. + + This bug was inherited from gzip's zgrep. gzip 1.12 includes + a fix for zgrep. + + The issue with the old sed script is that with multiple newlines, + the N-command will read the second line of input, then the + s-commands will be skipped because it's not the end of the + file yet, then a new sed cycle starts and the pattern space + is printed and emptied. So only the last line or two get escaped. + + One way to fix this would be to read all lines into the pattern + space first. However, the included fix is even simpler: All lines + except the last line get a backslash appended at the end. To ensure + that shell command substitution doesn't eat a possible trailing + newline, a colon is appended to the filename before escaping. + The colon is later used to separate the filename from the grep + output so it is fine to add it here instead of a few lines later. + + The old code also wasn't POSIX compliant as it used \n in the + replacement section of the s-command. Using \<newline> is the + POSIX compatible method. + + LC_ALL=C was added to the two critical sed commands. POSIX sed + manual recommends it when using sed to manipulate pathnames + because in other locales invalid multibyte sequences might + cause issues with some sed implementations. In case of GNU sed, + these particular sed scripts wouldn't have such problems but some + other scripts could have, see: + + info '(sed)Locale Considerations' + + This vulnerability was discovered by: + cleemy desu wayo working with Trend Micro Zero Day Initiative + + Thanks to Jim Meyering and Paul Eggert discussing the different + ways to fix this and for coordinating the patch release schedule + with gzip. + + src/scripts/xzgrep.in | 20 ++++++++++++-------- + 1 file changed, 12 insertions(+), 8 deletions(-) + +commit fa3af4e4c6a8f692785b87ba0f165f34d154ae0f +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-03-23 16:34:00 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit f12ce0f23a0496daebc77e8359309bf724ad5e8a +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-03-06 16:54:23 +0200 + + liblzma: Fix docs: lzma_block_decoder() cannot return LZMA_UNSUPPORTED_CHECK. + + If Check is unsupported, it will be silently ignored. + It's the caller's job to handle it. + + src/liblzma/api/lzma/block.h | 3 --- + 1 file changed, 3 deletions(-) + +commit 412566731111edc28f0b68bc966d01ab24d17082 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-03-06 15:18:58 +0200 + + liblzma: Index hash: Change return value type of hash_append() to void. + + src/liblzma/common/index_hash.c | 11 +++++------ + 1 file changed, 5 insertions(+), 6 deletions(-) + +commit 7c3ce02df001701a26c2fa41b2dc8b530743f454 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-22 03:42:57 +0200 + + liblzma: Minor addition to lzma_vli_size() API doc. + + Thanks to Jia Tan. + + src/liblzma/api/lzma/vli.h | 2 ++ + 1 file changed, 2 insertions(+) + +commit b8f667fe0c3e17fd2c559901d2aaf19b1dfd51f2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-22 02:04:18 +0200 + + liblzma: Check the return value of lzma_index_append() in threaded encoder. + + If lzma_index_append() failed (most likely memory allocation failure) + it could have gone unnoticed and the resulting .xz file would have + an incorrect Index. Decompressing such a file would produce the + correct uncompressed data but then an error would occur when + verifying the Index field. + + src/liblzma/common/stream_encoder_mt.c | 7 +++++-- + 1 file changed, 5 insertions(+), 2 deletions(-) + +commit 2356d53edd8f44ff8f411972ca62d63079ff59af +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-22 01:37:39 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 748ef0833821f1ea4ff5d57abc3cab1ae76266d3 +Author: Ed Maste <emaste@FreeBSD.org> +Date: 2022-02-11 15:25:46 +0000 + + liblzma: Use non-executable stack on FreeBSD as on Linux + + src/liblzma/check/crc32_x86.S | 4 ++-- + src/liblzma/check/crc64_x86.S | 4 ++-- + 2 files changed, 4 insertions(+), 4 deletions(-) + +commit 068a6e3286c9ca7d67f13f4c78646c4f4a3c3c5b +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-20 20:36:27 +0200 + + liblzma: Make Block decoder catch certain types of errors better. + + Now it limits the input and output buffer sizes that are + passed to a raw decoder. This way there's no need to check + if the sizes can grow too big or overflow when updating + Compressed Size and Uncompressed Size counts. This also means + that a corrupt file cannot cause the raw decoder to process + useless extra input or output that would exceed the size info + in Block Header (and thus cause LZMA_DATA_ERROR anyway). + + More importantly, now the size information is verified more + carefully in case raw decoder returns LZMA_OK. This doesn't + really matter with the current single-threaded .xz decoder + as the errors would be detected slightly later anyway. But + this helps avoiding corner cases in the upcoming threaded + decompressor, and it might help other Block decoder uses + outside liblzma too. + + The test files bad-1-lzma2-{9,10,11}.xz test these conditions. + With the single-threaded .xz decoder the only difference is + that LZMA_DATA_ERROR is detected in a difference place now. + + src/liblzma/common/block_decoder.c | 79 ++++++++++++++++++++++++++------------ + 1 file changed, 54 insertions(+), 25 deletions(-) + +commit 766df4f62c165ba96d60af8fadbada51349fd140 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-20 19:38:55 +0200 + + Tests: Add bad-1-lzma2-11.xz. + + tests/files/README | 5 +++++ + tests/files/bad-1-lzma2-11.xz | Bin 0 -> 64 bytes + 2 files changed, 5 insertions(+) + +commit 12a6d6ce2a9850e32140e542decfd9c8a33c87fc +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-18 18:51:10 +0200 + + Translations: Fix po4a failure with the French man page translations. + + Thanks to Mario Blättermann for the patch. + + po4a/fr_FR.po | 6 ++++-- + 1 file changed, 4 insertions(+), 2 deletions(-) + +commit 00e6aad836cf301e95575ea53790f5ab96fe8ed6 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-02-07 01:14:37 +0200 + + Translations: Add French translation of man pages. + + This matches xz-utils 5.2.5-2 in Debian. + + The translation was done by "bubu", proofread by the debian-l10n-french + mailing list contributors, and submitted to me on the xz-devel mailing + list by Jean-Pierre Giraud. Thanks to everyone! + + po4a/fr_FR.po | 3541 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + po4a/po4a.conf | 2 +- + 2 files changed, 3542 insertions(+), 1 deletion(-) + +commit e20ce2b12251a246c50fb5b7fa2204c11611b407 +Author: jiat75 <jiat0218@gmail.com> +Date: 2022-01-28 20:47:55 +0800 + + liblzma: Add NULL checks to LZMA and LZMA2 properties encoders. + + Previously lzma_lzma_props_encode() and lzma_lzma2_props_encode() + assumed that the options pointers must be non-NULL because the + with these filters the API says it must never be NULL. It is + good to do these checks anyway. + + src/liblzma/lzma/lzma2_encoder.c | 3 +++ + src/liblzma/lzma/lzma_encoder.c | 3 +++ + 2 files changed, 6 insertions(+) + +commit feb80ace866c3033cefbc39f6b07ada83e316e6a +Author: huangqinjin <huangqinjin@gmail.com> +Date: 2021-12-13 20:49:21 +0800 + + CMake: Keep compatible with Windows 95 for 32-bit build. + + CMakeLists.txt | 12 +++++++++++- + 1 file changed, 11 insertions(+), 1 deletion(-) + +commit 725f2e0522c4ef8dc5fdd69b5017bac6322ebce2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-11-13 21:04:05 +0200 + + xzgrep: Update man page timestamp. + + src/scripts/xzgrep.1 | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 7955669d423724e69a7f2dae1173dae248f7a596 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-11-13 18:23:24 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 671673a7a209e6131491e6a424758bbc13fb7aa1 +Author: Ville Skyttä <ville.skytta@iki.fi> +Date: 2021-11-13 10:11:57 +0200 + + xzgrep: use `grep -E/-F` instead of `egrep` and `fgrep` + + `egrep` and `fgrep` have been deprecated in GNU grep since 2007, and in + current post 3.7 Git they have been made to emit obsolescence warnings: + https://git.savannah.gnu.org/cgit/grep.git/commit/?id=a9515624709865d480e3142fd959bccd1c9372d1 + + src/scripts/xzgrep.1 | 8 ++++---- + src/scripts/xzgrep.in | 4 ++-- + 2 files changed, 6 insertions(+), 6 deletions(-) + +commit 45e538257e85050f67a6d363961e31b71e3ec565 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-10-27 23:27:48 +0300 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit ca21733d24a4f65b46a5efa124a89c859ac5592e +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-10-27 23:23:11 +0300 + + xz: Change the coding style of the previous commit. + + It isn't any better now but it's consistent with + the rest of the code base. + + src/xz/file_io.c | 11 ++++++----- + 1 file changed, 6 insertions(+), 5 deletions(-) + +commit 906b990b151b93750958371e39ad00e8d1daada9 +Author: Alexander Bluhm <alexander.bluhm@gmx.net> +Date: 2021-10-05 23:33:16 +0200 + + xz: Avoid fchown(2) failure. + + OpenBSD does not allow to change the group of a file if the user + does not belong to this group. In contrast to Linux, OpenBSD also + fails if the new group is the same as the old one. Do not call + fchown(2) in this case, it would change nothing anyway. + + This fixes an issue with Perl Alien::Build module. + https://github.com/PerlAlien/Alien-Build/issues/62 + + src/xz/file_io.c | 8 +++++++- + 1 file changed, 7 insertions(+), 1 deletion(-) + +commit ca83df96c48371ce0094c73c6f4582899ae4b62b +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-09-09 22:21:07 +0300 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit d8b294af03840c7c47d6b7b0d4c0d6d2f6357dfa +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-09-09 21:41:51 +0300 + + liblzma: Use _MSVC_LANG to detect when "noexcept" can be used with MSVC. + + By default, MSVC always sets __cplusplus to 199711L. The real + C++ standard version is available in _MSVC_LANG (or one could + use /Zc:__cplusplus to set __cplusplus correctly). + + Fixes <https://sourceforge.net/p/lzmautils/discussion/708858/thread/f6bc3b108a/>. + + Thanks to Dan Weiss. + + src/liblzma/api/lzma.h | 3 ++- + 1 file changed, 2 insertions(+), 1 deletion(-) + +commit c2fde22befe1f5256ad3d7c4cb35794aa8e64c67 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-06-04 19:02:38 +0300 + + xzdiff: Update the man page about the exit status. + + This was forgotten from 194029ffaf74282a81f0c299c07f73caca3232ca. + + src/scripts/xzdiff.1 | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +commit 8d0fd42fbedd01fd3c847b56d19841e603500dea +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-06-04 18:52:48 +0300 + + xzless: Fix less(1) version detection when it contains a dot. + + Sometimes the version number from "less -V" contains a dot, + sometimes not. xzless failed detect the version number when + it does contain a dot. This fixes it. + + Thanks to nick87720z for reporting this. Apparently it had been + reported here <https://bugs.gentoo.org/489362> in 2013. + + src/scripts/xzless.in | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 6e2cab8579fc6f832b5954510b27fd7e74563436 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-12 18:53:04 +0300 + + xz: Document the special memlimit case of 2000 MiB on MIPS32. + + See commit 95806a8a52ae57bddf6c77dfd19cf7938a92e040. + + src/xz/xz.1 | 8 ++++++-- + 1 file changed, 6 insertions(+), 2 deletions(-) + +commit 38b311462bd8974888ba3cd9312de132c76ec2e0 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-04-11 19:58:10 +0300 + + Update THANKS. + + THANKS | 2 ++ + 1 file changed, 2 insertions(+) + +commit 95806a8a52ae57bddf6c77dfd19cf7938a92e040 +Author: Ivan A. Melnikov <iv@altlinux.org> +Date: 2021-04-09 11:45:10 +0300 + + Reduce maximum possible memory limit on MIPS32 + + Due to architectural limitations, address space available to a single + userspace process on MIPS32 is limited to 2 GiB, not 4, even on systems + that have more physical RAM -- e.g. 64-bit systems with 32-bit + userspace, or systems that use XPA (an extension similar to x86's PAE). + + So, for MIPS32, we have to impose stronger memory limits. I've chosen + 2000MiB to give the process some headroom. + + src/xz/hardware.c | 6 ++++++ + 1 file changed, 6 insertions(+) + +commit a79bd30a6fbfe614f47ad77498c46e218dbe2b1d +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-02-13 23:31:27 +0200 + + CMake: Use interface library for better FindLibLZMA compatibility. + + https://www.mail-archive.com/xz-devel@tukaani.org/msg00446.html + + Thanks to Markus Rickert. + + CMakeLists.txt | 11 +++++++++-- + 1 file changed, 9 insertions(+), 2 deletions(-) + +commit 64d98147616c48f2df37fb46b49350f3d5febd34 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-30 18:36:04 +0200 + + CMake: Try to improve compatibility with the FindLibLZMA module. + + The naming conflict with FindLibLZMA module gets worse. + Not avoiding it in the first place was stupid. + + Normally find_package(LibLZMA) will use the module and + find_package(liblzma 5.2.5 REQUIRED CONFIG) will use the config + file even with a case insensitive file system. However, if + CMAKE_FIND_PACKAGE_PREFER_CONFIG is TRUE and the file system + is case insensitive, find_package(LibLZMA) will find our liblzma + config file instead of using FindLibLZMA module. + + One big problem with this is that FindLibLZMA uses + LibLZMA::LibLZMA and we use liblzma::liblzma as the target + name. With target names CMake happens to be case sensitive. + To workaround this, this commit adds + + add_library(LibLZMA::LibLZMA ALIAS liblzma::liblzma) + + to the config file. Then both spellings work. + + To make the behavior consistent between case sensitive and + insensitive file systems, the config and related files are + renamed from liblzmaConfig.cmake to liblzma-config.cmake style. + With this style CMake looks for lowercase version of the package + name so find_package(LiBLzmA 5.2.5 REQUIRED CONFIG) will work + to find our config file. + + There are other differences between our config file and + FindLibLZMA so it's still possible that things break for + reasons other than the spelling of the target name. Hopefully + those situations aren't too common. + + When the config file is available, it should always give as good or + better results as FindLibLZMA so this commit doesn't affect the + recommendation to use find_package(liblzma 5.2.5 REQUIRED CONFIG) + which explicitly avoids FindLibLZMA. + + Thanks to Markus Rickert. + + CMakeLists.txt | 21 +++++++++++++-------- + 1 file changed, 13 insertions(+), 8 deletions(-) + +commit 9e2f9e2d0841dd0417125ca3c7a05418541fb230 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-24 22:32:41 +0200 + + Tests: Add bad-1-lzma2-10.xz and also modify -9.xz. + + tests/files/README | 11 +++++++++-- + tests/files/bad-1-lzma2-10.xz | Bin 0 -> 60 bytes + tests/files/bad-1-lzma2-9.xz | Bin 72 -> 72 bytes + 3 files changed, 9 insertions(+), 2 deletions(-) + +commit 00a4c69bbbf302dd6323fd6970890ee22ddd4f96 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-24 18:51:51 +0200 + + Tests: Add bad-1-lzma2-9.xz. + + tests/files/README | 4 ++++ + tests/files/bad-1-lzma2-9.xz | Bin 0 -> 72 bytes + 2 files changed, 4 insertions(+) + +commit 1da2269b2e9b952d165ad7bec823de9dfe9a9e9e +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-24 17:02:00 +0200 + + Tests: Add bad-1-check-crc32-2.xz. + + tests/files/README | 7 +++++++ + tests/files/bad-1-check-crc32-2.xz | Bin 0 -> 72 bytes + 2 files changed, 7 insertions(+) + +commit 11ceecb5e2999ff96989915120e63a2e7e130057 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-11 23:57:11 +0200 + + Scripts: Add zstd support to xzdiff. + + src/scripts/xzdiff.1 | 6 ++++-- + src/scripts/xzdiff.in | 16 +++++++++++----- + 2 files changed, 15 insertions(+), 7 deletions(-) + +commit d655b8c9cb5a4e596c0fa2290fa91e995b734f97 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-11 23:28:52 +0200 + + Scripts: Fix exit status of xzgrep. + + Omit the -q option from xz, gzip, and bzip2. With xz this shouldn't + matter. With gzip it's important because -q makes gzip replace SIGPIPE + with exit status 2. With bzip2 it's important because with -q bzip2 + is completely silent if input is corrupt while other decompressors + still give an error message. + + Avoiding exit status 2 from gzip is important because bzip2 uses + exit status 2 to indicate corrupt input. Before this commit xzgrep + didn't recognize corrupt .bz2 files because xzgrep was treating + exit status 2 as SIGPIPE for gzip compatibility. + + zstd still needs -q because otherwise it is noisy in normal + operation. + + The code to detect real SIGPIPE didn't check if the exit status + was due to a signal (>= 128) and so could ignore some other exit + status too. + + src/scripts/xzgrep.in | 20 +++++++++++++------- + 1 file changed, 13 insertions(+), 7 deletions(-) + +commit 09c331b03c801414d547fc25711daf099cef6fb5 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2021-01-11 22:01:51 +0200 + + Scripts: Fix exit status of xzdiff/xzcmp. + + This is a minor fix since this affects only the situation when + the files differ and the exit status is something else than 0. + In such case there could be SIGPIPE from a decompression tool + and that would result in exit status of 2 from xzdiff/xzcmp + while the correct behavior would be to return 1 or whatever + else diff or cmp may have returned. + + This commit omits the -q option from xz/gzip/bzip2/lzop arguments. + I'm not sure why the -q was used in the first place, perhaps it + hides warnings in some situation that I cannot see at the moment. + Hopefully the removal won't introduce a new bug. + + With gzip the -q option was harmful because it made gzip return 2 + instead of >= 128 with SIGPIPE. Ignoring exit status 2 (warning + from gzip) isn't practical because bzip2 uses exit status 2 to + indicate corrupt input file. It's better if SIGPIPE results in + exit status >= 128. + + With bzip2 the removal of -q seems to be good because with -q + it prints nothing if input is corrupt. The other tools aren't + silent in this situation even with -q. On the other hand, if + zstd support is added, it will need -q since otherwise it's + noisy in normal situations. + + Thanks to Étienne Mollier and Sebastian Andrzej Siewior. + + src/scripts/xzdiff.in | 35 +++++++++++++++++++++-------------- + 1 file changed, 21 insertions(+), 14 deletions(-) + +commit b33a345cbab90e18b601d26cc3a0dea083b7d8d9 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-12-23 17:15:49 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit c01e29a933baba737217a3549eaaced7e499edf5 +Author: H.J. Lu <hjl.tools@gmail.com> +Date: 2020-12-23 06:49:04 -0800 + + liblzma: Enable Intel CET in x86 CRC assembly codes + + When Intel CET is enabled, we need to include <cet.h> in assembly codes + to mark Intel CET support and add _CET_ENDBR to indirect jump targets. + + Tested on Intel Tiger Lake under CET enabled Linux. + + src/liblzma/check/crc32_x86.S | 9 +++++++++ + src/liblzma/check/crc64_x86.S | 9 +++++++++ + 2 files changed, 18 insertions(+) + +commit 0983682f87ae1a9efcbe4e45579d53c9bca16500 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-12-16 18:33:29 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 880596e4b8731b24f6c19cb737ed7d06fe92dda2 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-12-16 18:30:14 +0200 + + Build: Don't build bundles on Apple OSes. + + Thanks to Daniel Packard. + + CMakeLists.txt | 3 +++ + 1 file changed, 3 insertions(+) + +commit 29050c79f1f83bf73714146a44dfbb4d26e27920 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-12-05 22:44:03 +0200 + + Update THANKS. + + THANKS | 1 + + 1 file changed, 1 insertion(+) + +commit 94fd72474973978f8369be7c23f77c3b741953a6 +Author: Adam Borowski <kilobyte@angband.pl> +Date: 2020-09-25 03:35:18 +0200 + + Scripts: Add zstd support to xzgrep. + + Thanks to Adam Borowski. + + src/scripts/xzgrep.1 | 9 ++++++--- + src/scripts/xzgrep.in | 1 + + 2 files changed, 7 insertions(+), 3 deletions(-) + +commit 13c58ac13e1a84c30de24191931af425286d0550 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-11-17 20:51:48 +0200 + + CMake: Fix compatibility with CMake 3.13. + + The syntax "if(DEFINED CACHE{FOO})" requires CMake 3.14. + In some other places the code treats the cache variables + like normal variables already (${FOO} or if(FOO) is used, + not ${CACHE{FOO}). + + Thanks to ygrek for reporting the bug on IRC. + + CMakeLists.txt | 2 +- + cmake/tuklib_cpucores.cmake | 4 ++-- + cmake/tuklib_physmem.cmake | 4 ++-- + 3 files changed, 5 insertions(+), 5 deletions(-) + +commit 8f7d3345a7c31a20a544dc04232715b1bcdc91f7 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-11-01 22:56:43 +0200 + + Update THANKS. + + THANKS | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +commit ca7bcdb30f14e7cd66fe357d10046442eee648a0 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-11-01 22:34:25 +0200 + + xz: Avoid unneeded \f escapes on the man page. + + I don't want to use \c in macro arguments but groff_man(7) + suggests that \f has better portability. \f would be needed + for the .TP strings for portability reasons anyway. + + Thanks to Bjarni Ingi Gislason. + + src/xz/xz.1 | 31 ++++++++++++++++++++++--------- + 1 file changed, 22 insertions(+), 9 deletions(-) + +commit 3b40a0792ee5ba0dd4d9ab129b4443585bbfdd14 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-11-01 19:09:53 +0200 + + xz: Use non-breaking spaces when intentionally using more than one space. + + This silences some style checker warnings. Seems that spaces + in the beginning of a line don't need this treatment. + + Thanks to Bjarni Ingi Gislason. + + src/xz/xz.1 | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit d85699c36df5f839b82800a6e63151eff25677eb +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-11-01 18:49:37 +0200 + + xz: Protect the ellipsis (...) on the man page with \&. + + This does it only when ... appears outside macro calls. + + Thanks to Bjarni Ingi Gislason. + + src/xz/xz.1 | 4 ++-- + 1 file changed, 2 insertions(+), 2 deletions(-) + +commit d996ae66176206de9b4d485b250a88b2d9ded151 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-11-01 18:41:21 +0200 + + xz: Avoid the abbreviation "e.g." on the man page. + + A few are simply omitted, most are converted to "for example" + and surrounded with commas. Sounds like that this is better + style, for example, man-pages(7) recommends avoiding such + abbreviations except in parenthesis. + + Thanks to Bjarni Ingi Gislason. + + src/xz/xz.1 | 66 ++++++++++++++++++++++++++++++------------------------------- + 1 file changed, 33 insertions(+), 33 deletions(-) + +commit d16d0d198ade6820ed9d39ba6a7068b55b8bf6dc +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-07-12 23:10:03 +0300 + + xz man page: Change \- (minus) to \(en (en-dash) for a numeric range. + + Docs of ancient troff/nroff mention \(em (em-dash) but not \(en + and \- was used for both minus and en-dash. I don't know how + portable \(en is nowadays but it can be changed back if someone + complains. At least GNU groff and OpenBSD's mandoc support it. + + Thanks to Bjarni Ingi Gislason for the patch. + + src/xz/xz.1 | 16 ++++++++-------- + 1 file changed, 8 insertions(+), 8 deletions(-) + +commit 3acf1adfc768c7d2290e138bcb1109145e4d8160 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-07-12 20:46:24 +0300 + + Windows: Fix building of resource files when config.h isn't used. + + Now CMake + Visual Studio works for building liblzma.dll. + + Thanks to Markus Rickert. + + src/common/common_w32res.rc | 4 +++- + 1 file changed, 3 insertions(+), 1 deletion(-) + +commit adba06e649875b45b5b671f5ab45f694144938d4 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-04-06 19:31:50 +0300 + + src/scripts/xzgrep.1: Filenames to xzgrep are optional. + + xzgrep --help was correct already. + + src/scripts/xzgrep.1 | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 7be1dcf8585e150bffb57a6ad499a63edd0d3494 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-12 17:59:41 +0300 + + Translations: Add Portuguese translation. + + Jia Tan made white-space changes and also changed "Language: pt_BR\n" + to pt. The translator wasn't reached so I'm hoping these changes + are OK and will commit it without translator's approval. + + Thanks to Pedro Albuquerque and Jia Tan. + + po/LINGUAS | 1 + + po/pt.po | 1001 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 1002 insertions(+) + +commit 3f94d2a568290bf770a762ee69403955fdb0a228 +Author: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> +Date: 2020-03-26 22:17:31 +0000 + + src/script/xzgrep.1: Remove superfluous '.RB' + + Output is from: test-groff -b -e -mandoc -T utf8 -rF0 -t -w w -z + + [ "test-groff" is a developmental version of "groff" ] + + Input file is ./src/scripts/xzgrep.1 + + <src/scripts/xzgrep.1>:20 (macro RB): only 1 argument, but more are expected + <src/scripts/xzgrep.1>:23 (macro RB): only 1 argument, but more are expected + <src/scripts/xzgrep.1>:26 (macro RB): only 1 argument, but more are expected + <src/scripts/xzgrep.1>:29 (macro RB): only 1 argument, but more are expected + <src/scripts/xzgrep.1>:32 (macro RB): only 1 argument, but more are expected + + "abc..." does not mean the same as "abc ...". + + The output from nroff and troff is unchanged except for the space + between "file" and "...". + + Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> + + src/scripts/xzgrep.1 | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) + +commit 725d9791c90b7cef19a3ab4d7469b567101c2a87 +Author: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> +Date: 2020-03-30 21:56:36 +0000 + + xzgrep.1: Delete superfluous '.PP' + + Summary: + + mandoc -T lint xzgrep.1 : + mandoc: xzgrep.1:79:2: WARNING: skipping paragraph macro: PP empty + + There is no change in the output of "nroff" and "troff". + + Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> + + src/scripts/xzgrep.1 | 1 - + 1 file changed, 1 deletion(-) + +commit 55c2555c5d420e9251a97828022e5c8c4d786dac +Author: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> +Date: 2020-03-26 21:16:18 +0000 + + src/xz/xz.1: Correct misused two-fonts macros + + Output is from: test-groff -b -e -mandoc -T utf8 -rF0 -t -w w -z + + [ "test-groff" is a developmental version of "groff" ] + + Input file is ./src/xz/xz.1 + + <src/xz/xz.1>:408 (macro BR): only 1 argument, but more are expected + <src/xz/xz.1>:1009 (macro BR): only 1 argument, but more are expected + <src/xz/xz.1>:1743 (macro BR): only 1 argument, but more are expected + <src/xz/xz.1>:1920 (macro BR): only 1 argument, but more are expected + <src/xz/xz.1>:2213 (macro BR): only 1 argument, but more are expected + + Output from nroff and troff is unchanged, except for a font change of a + full stop (.). + + Signed-off-by: Bjarni Ingi Gislason <bjarniig@rhi.hi.is> + + src/xz/xz.1 | 10 +++++----- + 1 file changed, 5 insertions(+), 5 deletions(-) + +commit 55e3a8e0e437279223c108085dccddc60b06036c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-10 21:16:40 +0300 + + Translations: Add Serbian translation. + + Quite a few white-space changes were made by Jia Tan to make + this look good. Contacting the translator didn't succeed so + I'm committing this without getting translator's approval. + + Thanks to Мирослав Николић (Miroslav Nikolic) and Jia Tan. + + po/LINGUAS | 1 + + po/sr.po | 987 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 988 insertions(+) + +commit 4e04279cbe8c1af25dde42ebf740722deded27a0 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-04 23:51:36 +0300 + + Translations: Add Swedish translation. + + Thanks to Sebastian Rasmussen and Jia Tan. + + po/LINGUAS | 1 + + po/sv.po | 983 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 984 insertions(+) + +commit 27f55e96b0cdae3fc51b409663b6c017a88ea8bc +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-04 23:40:27 +0300 + + Translations: Add Esperanto translation. + + Thanks to Keith Bowes and Jia Tan. + + po/LINGUAS | 1 + + po/eo.po | 984 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 985 insertions(+) + +commit be31be9e8f12cf6fce4be1ab822660ad411e803b +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-07-01 00:22:33 +0300 + + Translations: Add Catalan translation. + + Thanks to Jordi Mas and Jia Tan. + + po/LINGUAS | 1 + + po/ca.po | 1076 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 1077 insertions(+) + +commit 30e3ad040f4e7ba1a0e97179d6fd62a1201c863b +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-30 17:47:08 +0300 + + Translations: Add Ukrainian translation. + + Thanks to Yuri Chornoivan and Jia Tan. + + po/LINGUAS | 1 + + po/uk.po | 996 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 997 insertions(+) + +commit f2e2cce49f7b07e8985ba6add4b4ca79b22eddd0 +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-30 17:45:26 +0300 + + Translators: Add Romanian translation. + + Thanks to Remus-Gabriel Chelu and Jia Tan. + + po/LINGUAS | 1 + + po/ro.po | 1016 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 1017 insertions(+) + +commit a12fd5bad2aa922c9b4cf96d16632696f3e0584e +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-29 18:33:32 +0300 + + Translations: Update Brazilian Portuguese translation. + + One msgstr was changed. The diff is long due to changes + in the source code line numbers in the comments. + + Thanks to Rafael Fontenelle. + + po/pt_BR.po | 186 ++++++++++++++++++++++++++++++------------------------------ + 1 file changed, 92 insertions(+), 94 deletions(-) + +commit dd713288d27a0d64426d74effe9187087496333c +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-29 18:04:44 +0300 + + Translations: Add Croatian translation. + + Thanks to Božidar Putanec and Jia Tan. + + po/LINGUAS | 1 + + po/hr.po | 987 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 988 insertions(+) + +commit 016980f26974623a4bcd889b6ba5a0caa4b2f57f +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-29 17:58:48 +0300 + + Translations: Add Spanish translation. + + Thanks to Cristian Othón Martínez Vera and Jia Tan. + + po/LINGUAS | 1 + + po/es.po | 984 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 985 insertions(+) + +commit 4fe9578c3aab21e6ceb8f0ea470bc87821f5907f +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2022-06-29 17:49:43 +0300 + + Translations: Add Korean translation. + + Thanks to Seong-ho Cho and Jia Tan. + + po/LINGUAS | 1 + + po/ko.po | 972 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 2 files changed, 973 insertions(+) + +commit cf1ec5518e6e3d89f57685ac83b81ab228e48bcd +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-03-23 18:09:40 +0200 + + v5.2-specific typo fix from fossies.org. + + README | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +commit 968bbfea09447a4f52e856bb97301aca80e40dcb +Author: Lasse Collin <lasse.collin@tukaani.org> +Date: 2020-03-23 18:07:50 +0200 + + Typo fixes from fossies.org. + + https://fossies.org/linux/misc/xz-5.2.5.tar.xz/codespell.html + + Makefile.am | 2 +- + doc/examples/01_compress_easy.c | 2 +- + src/liblzma/api/lzma/base.h | 2 +- + src/liblzma/check/crc32_x86.S | 2 +- + src/liblzma/common/index.c | 2 +- + src/xz/xz.1 | 4 ++-- + 6 files changed, 7 insertions(+), 7 deletions(-) + commit 2327a461e1afce862c22269b80d3517801103c1b Author: Lasse Collin <lasse.collin@tukaani.org> Date: 2020-03-17 16:27:42 +0200 @@ -204,7 +204,7 @@ XZ Utils Several strings will change in a future version of xz so if you wish to start a new translation, look at the code in the xz git - repostiory instead of a 5.2.x release. + repository instead of a 5.2.x release. 5. Other implementations of the .xz format @@ -8,27 +8,33 @@ has been important. :-) In alphabetical order: - H. Peter Anvin - Jeff Bastian - Nelson H. F. Beebe + - Karl Beldan - Karl Berry - Anders F. Björklund - Emmanuel Blot - Melanie Blower + - Alexander Bluhm - Martin Blumenstingl - Ben Boeckel - Jakub Bogusz + - Adam Borowski - Maarten Bosmans - Trent W. Buck + - Kevin R. Bulgrien - James Buren - David Burklund - Daniel Mealha Cabrita - Milo Casagrande - Marek Černocký - Tomer Chachamu + - Vitaly Chikunov - Antoine Cœur - Gabi Davar - Chris Donawa - Andrew Dudman - Markus Duft - İsmail Dönmez + - Paul Eggert - Robert Elz - Gilles Espinasse - Denis Excoffier @@ -36,6 +42,7 @@ has been important. :-) In alphabetical order: - Michael Fox - Mike Frysinger - Daniel Richard G. + - Bjarni Ingi Gislason - Bill Glessner - Jason Gorski - Juan Manuel Guerrero @@ -63,10 +70,13 @@ has been important. :-) In alphabetical order: - Xin Li - Eric Lindblad - Lorenzo De Liso + - H.J. Lu - Bela Lubkin - Gregory Margo - Julien Marrec + - Ed Maste - Martin Matuška + - Ivan A. Melnikov - Jim Meyering - Arkadiusz Miskiewicz - Conley Moorhous @@ -77,6 +87,7 @@ has been important. :-) In alphabetical order: - Jonathan Nieder - Andre Noll - Peter O'Gorman + - Daniel Packard - Filip Palian - Peter Pallinger - Rui Paulo @@ -88,12 +99,15 @@ has been important. :-) In alphabetical order: - Trần Ngọc Quân - Pavel Raiskup - Ole André Vadla Ravnås + - Eric S. Raymond - Robert Readman - Bernhard Reutner-Fischer - - Eric S. Raymond + - Markus Rickert - Cristian Rodríguez - Christian von Roques + - Boud Roukema - Torsten Rupp + - Stephen Sachs - Jukka Salmi - Alexandre Sauvé - Benno Schulenberg @@ -102,11 +116,13 @@ has been important. :-) In alphabetical order: - Dan Shechter - Stuart Shelton - Sebastian Andrzej Siewior + - Ville Skyttä - Brad Smith - Bruce Stark - Pippijn van Steenhoven - Jonathan Stott - Dan Stromberg + - Jia Tan - Vincent Torri - Paul Townsend - Mohammed Adnène Trojette @@ -117,6 +133,7 @@ has been important. :-) In alphabetical order: - Adam Walling - Jeffrey Walton - Christian Weisgerber + - Dan Weiss - Bert Wesarg - Fredrik Wikstrom - Jim Wilcoxson diff --git a/src/common/mythread.h b/src/common/mythread.h index be22654240aa..413821836020 100644 --- a/src/common/mythread.h +++ b/src/common/mythread.h @@ -370,10 +370,11 @@ typedef struct { BOOL pending_; \ if (!InitOnceBeginInitialize(&once_, 0, &pending_, NULL)) \ abort(); \ - if (pending_) \ + if (pending_) { \ func(); \ - if (!InitOnceComplete(&once, 0, NULL)) \ - abort(); \ + if (!InitOnceComplete(&once, 0, NULL)) \ + abort(); \ + } \ } while (0) #endif diff --git a/src/common/tuklib_cpucores.c b/src/common/tuklib_cpucores.c index cc968dd25efb..bb3f2f752b1b 100644 --- a/src/common/tuklib_cpucores.c +++ b/src/common/tuklib_cpucores.c @@ -72,7 +72,16 @@ tuklib_cpucores(void) } #elif defined(TUKLIB_CPUCORES_SYSCTL) + // On OpenBSD HW_NCPUONLINE tells the number of processor cores that + // are online so it is preferred over HW_NCPU which also counts cores + // that aren't currently available. The number of cores online is + // often less than HW_NCPU because OpenBSD disables simultaneous + // multi-threading (SMT) by default. +# ifdef HW_NCPUONLINE + int name[2] = { CTL_HW, HW_NCPUONLINE }; +# else int name[2] = { CTL_HW, HW_NCPU }; +# endif int cpus; size_t cpus_size = sizeof(cpus); if (sysctl(name, 2, &cpus, &cpus_size, NULL, 0) != -1 diff --git a/src/common/tuklib_physmem.c b/src/common/tuklib_physmem.c index 4053ad006a64..a1bccb2f6753 100644 --- a/src/common/tuklib_physmem.c +++ b/src/common/tuklib_physmem.c @@ -84,7 +84,7 @@ tuklib_physmem(void) // supports reporting values greater than 4 GiB. To keep the // code working also on older Windows versions, use // GlobalMemoryStatusEx() conditionally. - HMODULE kernel32 = GetModuleHandle("kernel32.dll"); + HMODULE kernel32 = GetModuleHandle(TEXT("kernel32.dll")); if (kernel32 != NULL) { typedef BOOL (WINAPI *gmse_type)(LPMEMORYSTATUSEX); gmse_type gmse = (gmse_type)GetProcAddress( diff --git a/src/liblzma/api/lzma.h b/src/liblzma/api/lzma.h index 122dab80d357..8fbd9a874b32 100644 --- a/src/liblzma/api/lzma.h +++ b/src/liblzma/api/lzma.h @@ -219,7 +219,8 @@ */ #ifndef lzma_nothrow # if defined(__cplusplus) -# if __cplusplus >= 201103L +# if __cplusplus >= 201103L || (defined(_MSVC_LANG) \ + && _MSVC_LANG >= 201103L) # define lzma_nothrow noexcept # else # define lzma_nothrow throw() diff --git a/src/liblzma/api/lzma/base.h b/src/liblzma/api/lzma/base.h index a6005accc93d..51421353ca1d 100644 --- a/src/liblzma/api/lzma/base.h +++ b/src/liblzma/api/lzma/base.h @@ -145,6 +145,12 @@ typedef enum { * specified memory usage limit. To continue decoding, * the memory usage limit has to be increased with * lzma_memlimit_set(). + * + * liblzma 5.2.6 and earlier had a bug in single-threaded .xz + * decoder (lzma_stream_decoder()) which made it impossible + * to continue decoding after LZMA_MEMLIMIT_ERROR even if + * the limit was increased using lzma_memlimit_set(). + * Other decoders worked correctly. */ LZMA_FORMAT_ERROR = 7, @@ -447,7 +453,7 @@ typedef struct lzma_internal_s lzma_internal; * * The lzma_stream structure is used for * - passing pointers to input and output buffers to liblzma; - * - defining custom memory hander functions; and + * - defining custom memory handler functions; and * - holding a pointer to coder-specific internal data structures. * * Typical usage: @@ -649,6 +655,11 @@ extern LZMA_API(uint64_t) lzma_memlimit_get(const lzma_stream *strm) * return LZMA_OK. Later versions treat 0 as if 1 had been specified (so * lzma_memlimit_get() will return 1 even if you specify 0 here). * + * liblzma 5.2.6 and earlier had a bug in single-threaded .xz decoder + * (lzma_stream_decoder()) which made it impossible to continue decoding + * after LZMA_MEMLIMIT_ERROR even if the limit was increased using + * lzma_memlimit_set(). Other decoders worked correctly. + * * \return - LZMA_OK: New memory usage limit successfully set. * - LZMA_MEMLIMIT_ERROR: The new limit is too small. * The limit was not changed. diff --git a/src/liblzma/api/lzma/block.h b/src/liblzma/api/lzma/block.h index 962f38779cc7..082e55833e70 100644 --- a/src/liblzma/api/lzma/block.h +++ b/src/liblzma/api/lzma/block.h @@ -464,9 +464,6 @@ extern LZMA_API(lzma_ret) lzma_block_encoder( * LZMA_FINISH is not required. It is supported only for convenience. * * \return - LZMA_OK: All good, continue with lzma_code(). - * - LZMA_UNSUPPORTED_CHECK: Initialization was successful, but - * the given Check ID is not supported, thus Check will be - * ignored. * - LZMA_PROG_ERROR * - LZMA_MEM_ERROR */ diff --git a/src/liblzma/api/lzma/container.h b/src/liblzma/api/lzma/container.h index 9fbf4df06178..9a9ff1c9a486 100644 --- a/src/liblzma/api/lzma/container.h +++ b/src/liblzma/api/lzma/container.h @@ -526,7 +526,8 @@ extern LZMA_API(lzma_ret) lzma_stream_buffer_encode( * had been specified. * \param flags Bitwise-or of zero or more of the decoder flags: * LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK, - * LZMA_TELL_ANY_CHECK, LZMA_CONCATENATED + * LZMA_TELL_ANY_CHECK, LZMA_IGNORE_CHECK, + * LZMA_CONCATENATED * * \return - LZMA_OK: Initialization was successful. * - LZMA_MEM_ERROR: Cannot allocate memory. @@ -545,13 +546,23 @@ extern LZMA_API(lzma_ret) lzma_stream_decoder( * calls lzma_stream_decoder() or lzma_alone_decoder() once the type * of the input file has been detected. * + * If the flag LZMA_CONCATENATED is used and the input is a .lzma file: + * For historical reasons concatenated .lzma files aren't supported. + * If there is trailing data after one .lzma stream, lzma_code() will + * return LZMA_DATA_ERROR. (lzma_alone_decoder() doesn't have such a check + * as it doesn't support any decoder flags. It will return LZMA_STREAM_END + * after one .lzma stream.) + * * \param strm Pointer to properly prepared lzma_stream * \param memlimit Memory usage limit as bytes. Use UINT64_MAX * to effectively disable the limiter. liblzma * 5.2.3 and earlier don't allow 0 here and return * LZMA_PROG_ERROR; later versions treat 0 as if 1 * had been specified. - * \param flags Bitwise-or of flags, or zero for no flags. + * \param flags Bitwise-or of zero or more of the decoder flags: + * LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK, + * LZMA_TELL_ANY_CHECK, LZMA_IGNORE_CHECK, + * LZMA_CONCATENATED * * \return - LZMA_OK: Initialization was successful. * - LZMA_MEM_ERROR: Cannot allocate memory. @@ -595,8 +606,9 @@ extern LZMA_API(lzma_ret) lzma_alone_decoder( * returned. * \param flags Bitwise-or of zero or more of the decoder flags: * LZMA_TELL_NO_CHECK, LZMA_TELL_UNSUPPORTED_CHECK, - * LZMA_CONCATENATED. Note that LZMA_TELL_ANY_CHECK - * is not allowed and will return LZMA_PROG_ERROR. + * LZMA_IGNORE_CHECK, LZMA_CONCATENATED. Note that + * LZMA_TELL_ANY_CHECK is not allowed and will + * return LZMA_PROG_ERROR. * \param allocator lzma_allocator for custom allocator functions. * Set to NULL to use malloc() and free(). * \param in Beginning of the input buffer diff --git a/src/liblzma/api/lzma/filter.h b/src/liblzma/api/lzma/filter.h index 8c8593147658..04825c655eab 100644 --- a/src/liblzma/api/lzma/filter.h +++ b/src/liblzma/api/lzma/filter.h @@ -108,7 +108,9 @@ extern LZMA_API(lzma_bool) lzma_filter_decoder_is_supported(lzma_vli id) * need to be initialized by the caller in any way. * * If an error occurs, memory possibly already allocated by this function - * is always freed. + * is always freed. liblzma versions older than 5.2.7 may modify the dest + * array and leave its contents in an undefined state if an error occurs. + * liblzma 5.2.7 and newer only modify the dest array when returning LZMA_OK. * * \return - LZMA_OK * - LZMA_MEM_ERROR diff --git a/src/liblzma/api/lzma/version.h b/src/liblzma/api/lzma/version.h index 2bf3eaed24fa..fcae6ea2abc2 100644 --- a/src/liblzma/api/lzma/version.h +++ b/src/liblzma/api/lzma/version.h @@ -22,7 +22,7 @@ */ #define LZMA_VERSION_MAJOR 5 #define LZMA_VERSION_MINOR 2 -#define LZMA_VERSION_PATCH 5 +#define LZMA_VERSION_PATCH 8 #define LZMA_VERSION_STABILITY LZMA_VERSION_STABILITY_STABLE #ifndef LZMA_VERSION_COMMIT diff --git a/src/liblzma/api/lzma/vli.h b/src/liblzma/api/lzma/vli.h index 1b7a952a406d..7a0a803dfc9c 100644 --- a/src/liblzma/api/lzma/vli.h +++ b/src/liblzma/api/lzma/vli.h @@ -159,6 +159,8 @@ extern LZMA_API(lzma_ret) lzma_vli_decode(lzma_vli *vli, size_t *vli_pos, /** * \brief Get the number of bytes required to encode a VLI * + * \param vli Integer whose encoded size is to be determined + * * \return Number of bytes on success (1-9). If vli isn't valid, * zero is returned. */ diff --git a/src/liblzma/check/crc32_x86.S b/src/liblzma/check/crc32_x86.S index 67f68a4145f8..4f395df8122a 100644 --- a/src/liblzma/check/crc32_x86.S +++ b/src/liblzma/check/crc32_x86.S @@ -51,6 +51,14 @@ init_table(void) * extern uint32_t lzma_crc32(const uint8_t *buf, size_t size, uint32_t crc); */ +/* When Intel CET is enabled, include <cet.h> in assembly code to mark + Intel CET support. */ +#ifdef __CET__ +# include <cet.h> +#else +# define _CET_ENDBR +#endif + /* * On some systems, the functions need to be prefixed. The prefix is * usually an underscore. @@ -83,6 +91,7 @@ init_table(void) ALIGN(4, 16) LZMA_CRC32: + _CET_ENDBR /* * Register usage: * %eax crc @@ -195,7 +204,7 @@ LZMA_CRC32: /* * Read the next four bytes, for which the CRC is calculated - * on the next interation of the loop. + * on the next iteration of the loop. */ movl 12(%esi), %ecx @@ -296,9 +305,9 @@ LZMA_CRC32: /* * This is needed to support non-executable stack. It's ugly to - * use __linux__ here, but I don't know a way to detect when + * use __FreeBSD__ and __linux__ here, but I don't know a way to detect when * we are using GNU assembler. */ -#if defined(__ELF__) && defined(__linux__) +#if defined(__ELF__) && (defined(__FreeBSD__) || defined(__linux__)) .section .note.GNU-stack,"",@progbits #endif diff --git a/src/liblzma/check/crc64_x86.S b/src/liblzma/check/crc64_x86.S index f5bb84b97e0a..9aecf5865455 100644 --- a/src/liblzma/check/crc64_x86.S +++ b/src/liblzma/check/crc64_x86.S @@ -41,6 +41,14 @@ init_table(void) * extern uint64_t lzma_crc64(const uint8_t *buf, size_t size, uint64_t crc); */ +/* When Intel CET is enabled, include <cet.h> in assembly code to mark + Intel CET support. */ +#ifdef __CET__ +# include <cet.h> +#else +# define _CET_ENDBR +#endif + /* * On some systems, the functions need to be prefixed. The prefix is * usually an underscore. @@ -73,6 +81,7 @@ init_table(void) ALIGN(4, 16) LZMA_CRC64: + _CET_ENDBR /* * Register usage: * %eax crc LSB @@ -279,9 +288,9 @@ LZMA_CRC64: /* * This is needed to support non-executable stack. It's ugly to - * use __linux__ here, but I don't know a way to detect when + * use __FreeBSD__ and __linux__ here, but I don't know a way to detect when * we are using GNU assembler. */ -#if defined(__ELF__) && defined(__linux__) +#if defined(__ELF__) && (defined(__FreeBSD__) || defined(__linux__)) .section .note.GNU-stack,"",@progbits #endif diff --git a/src/liblzma/common/alone_decoder.c b/src/liblzma/common/alone_decoder.c index 239b230ef19b..a3ea20a27a9c 100644 --- a/src/liblzma/common/alone_decoder.c +++ b/src/liblzma/common/alone_decoder.c @@ -146,7 +146,7 @@ alone_decode(void *coder_ptr, const lzma_allocator *allocator, // Use a hack to set the uncompressed size. lzma_lz_decoder_uncompressed(coder->next.coder, - coder->uncompressed_size); + coder->uncompressed_size, true); coder->sequence = SEQ_CODE; break; diff --git a/src/liblzma/common/auto_decoder.c b/src/liblzma/common/auto_decoder.c index 6895c7ccf7b5..f58ab595ee3c 100644 --- a/src/liblzma/common/auto_decoder.c +++ b/src/liblzma/common/auto_decoder.c @@ -86,8 +86,8 @@ auto_decode(void *coder_ptr, const lzma_allocator *allocator, // Fall through case SEQ_FINISH: - // When LZMA_DECODE_CONCATENATED was used and we were decoding - // LZMA_Alone file, we need to check check that there is no + // When LZMA_CONCATENATED was used and we were decoding + // a LZMA_Alone file, we need to check that there is no // trailing garbage and wait for LZMA_FINISH. if (*in_pos < in_size) return LZMA_DATA_ERROR; diff --git a/src/liblzma/common/block_buffer_encoder.c b/src/liblzma/common/block_buffer_encoder.c index 39e263aa4765..a47342efd0f5 100644 --- a/src/liblzma/common/block_buffer_encoder.c +++ b/src/liblzma/common/block_buffer_encoder.c @@ -325,6 +325,24 @@ lzma_block_buffer_encode(lzma_block *block, const lzma_allocator *allocator, } +#ifdef HAVE_SYMBOL_VERSIONS_LINUX +// This is for compatibility with binaries linked against liblzma that +// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7. +LZMA_SYMVER_API("lzma_block_uncomp_encode@XZ_5.2.2", + lzma_ret, lzma_block_uncomp_encode_522)(lzma_block *block, + const uint8_t *in, size_t in_size, + uint8_t *out, size_t *out_pos, size_t out_size) + lzma_nothrow lzma_attr_warn_unused_result + __attribute__((__alias__("lzma_block_uncomp_encode_52"))); + +LZMA_SYMVER_API("lzma_block_uncomp_encode@@XZ_5.2", + lzma_ret, lzma_block_uncomp_encode_52)(lzma_block *block, + const uint8_t *in, size_t in_size, + uint8_t *out, size_t *out_pos, size_t out_size) + lzma_nothrow lzma_attr_warn_unused_result; + +#define lzma_block_uncomp_encode lzma_block_uncomp_encode_52 +#endif extern LZMA_API(lzma_ret) lzma_block_uncomp_encode(lzma_block *block, const uint8_t *in, size_t in_size, diff --git a/src/liblzma/common/block_decoder.c b/src/liblzma/common/block_decoder.c index 075bd279ff60..4827e0f0469d 100644 --- a/src/liblzma/common/block_decoder.c +++ b/src/liblzma/common/block_decoder.c @@ -40,6 +40,9 @@ typedef struct { /// is unknown. lzma_vli compressed_limit; + /// Maximum allowed Uncompressed Size. + lzma_vli uncompressed_limit; + /// Position when reading the Check field size_t check_pos; @@ -52,21 +55,6 @@ typedef struct { static inline bool -update_size(lzma_vli *size, lzma_vli add, lzma_vli limit) -{ - if (limit > LZMA_VLI_MAX) - limit = LZMA_VLI_MAX; - - if (limit < *size || limit - *size < add) - return true; - - *size += add; - - return false; -} - - -static inline bool is_size_valid(lzma_vli size, lzma_vli reference) { return reference == LZMA_VLI_UNKNOWN || reference == size; @@ -86,21 +74,54 @@ block_decode(void *coder_ptr, const lzma_allocator *allocator, const size_t in_start = *in_pos; const size_t out_start = *out_pos; + // Limit the amount of input and output space that we give + // to the raw decoder based on the information we have + // (or don't have) from Block Header. + const size_t in_stop = *in_pos + (size_t)my_min( + in_size - *in_pos, + coder->compressed_limit - coder->compressed_size); + const size_t out_stop = *out_pos + (size_t)my_min( + out_size - *out_pos, + coder->uncompressed_limit - coder->uncompressed_size); + const lzma_ret ret = coder->next.code(coder->next.coder, - allocator, in, in_pos, in_size, - out, out_pos, out_size, action); + allocator, in, in_pos, in_stop, + out, out_pos, out_stop, action); const size_t in_used = *in_pos - in_start; const size_t out_used = *out_pos - out_start; - // NOTE: We compare to compressed_limit here, which prevents - // the total size of the Block growing past LZMA_VLI_MAX. - if (update_size(&coder->compressed_size, in_used, - coder->compressed_limit) - || update_size(&coder->uncompressed_size, - out_used, - coder->block->uncompressed_size)) - return LZMA_DATA_ERROR; + // Because we have limited the input and output sizes, + // we know that these cannot grow too big or overflow. + coder->compressed_size += in_used; + coder->uncompressed_size += out_used; + + if (ret == LZMA_OK) { + const bool comp_done = coder->compressed_size + == coder->block->compressed_size; + const bool uncomp_done = coder->uncompressed_size + == coder->block->uncompressed_size; + + // If both input and output amounts match the sizes + // in Block Header but we still got LZMA_OK instead + // of LZMA_STREAM_END, the file is broken. + if (comp_done && uncomp_done) + return LZMA_DATA_ERROR; + + // If the decoder has consumed all the input that it + // needs but it still couldn't fill the output buffer + // or return LZMA_STREAM_END, the file is broken. + if (comp_done && *out_pos < out_size) + return LZMA_DATA_ERROR; + + // If the decoder has produced all the output but + // it still didn't return LZMA_STREAM_END or consume + // more input (for example, detecting an end of + // payload marker may need more input but produce + // no output) the file is broken. + if (uncomp_done && *in_pos < in_size) + return LZMA_DATA_ERROR; + } if (!coder->ignore_check) lzma_check_update(&coder->check, coder->block->check, @@ -230,6 +251,14 @@ lzma_block_decoder_init(lzma_next_coder *next, const lzma_allocator *allocator, - lzma_check_size(block->check) : block->compressed_size; + // With Uncompressed Size this is simpler. If Block Header lacks + // the size info, then LZMA_VLI_MAX is the maximum possible + // Uncompressed Size. + coder->uncompressed_limit + = block->uncompressed_size == LZMA_VLI_UNKNOWN + ? LZMA_VLI_MAX + : block->uncompressed_size; + // Initialize the check. It's caller's problem if the Check ID is not // supported, and the Block decoder cannot verify the Check field. // Caller can test lzma_check_is_supported(block->check). diff --git a/src/liblzma/common/common.c b/src/liblzma/common/common.c index cf714e5e43b4..10fc884099a4 100644 --- a/src/liblzma/common/common.c +++ b/src/liblzma/common/common.c @@ -366,6 +366,20 @@ lzma_end(lzma_stream *strm) } +#ifdef HAVE_SYMBOL_VERSIONS_LINUX +// This is for compatibility with binaries linked against liblzma that +// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7. +LZMA_SYMVER_API("lzma_get_progress@XZ_5.2.2", + void, lzma_get_progress_522)(lzma_stream *strm, + uint64_t *progress_in, uint64_t *progress_out) lzma_nothrow + __attribute__((__alias__("lzma_get_progress_52"))); + +LZMA_SYMVER_API("lzma_get_progress@@XZ_5.2", + void, lzma_get_progress_52)(lzma_stream *strm, + uint64_t *progress_in, uint64_t *progress_out) lzma_nothrow; + +#define lzma_get_progress lzma_get_progress_52 +#endif extern LZMA_API(void) lzma_get_progress(lzma_stream *strm, uint64_t *progress_in, uint64_t *progress_out) diff --git a/src/liblzma/common/common.h b/src/liblzma/common/common.h index b3d3b7a059b1..33928c3d2b33 100644 --- a/src/liblzma/common/common.h +++ b/src/liblzma/common/common.h @@ -34,6 +34,34 @@ #include "lzma.h" +#ifdef HAVE_SYMBOL_VERSIONS_LINUX +// To keep link-time optimization (LTO, -flto) working with GCC, +// the __symver__ attribute must be used instead of __asm__(".symver ..."). +// Otherwise the symbol versions may be lost, resulting in broken liblzma +// that has wrong default versions in the exported symbol list! +// The attribute was added in GCC 10; LTO with older GCC is not supported. +// +// To keep -Wmissing-prototypes happy, use LZMA_SYMVER_API only with function +// declarations (including those with __alias__ attribute) and LZMA_API with +// the function definitions. This means a little bit of silly copy-and-paste +// between declarations and definitions though. +// +// As of GCC 12.2, the __symver__ attribute supports only @ and @@ but the +// very convenient @@@ isn't supported (it's supported by GNU assembler +// since 2000). When using @@ instead of @@@, the internal name must not be +// the same as the external name to avoid problems in some situations. This +// is why "#define foo_52 foo" is needed for the default symbol versions. +# if TUKLIB_GNUC_REQ(10, 0) && !defined(__INTEL_COMPILER) +# define LZMA_SYMVER_API(extnamever, type, intname) \ + extern __attribute__((__symver__(extnamever))) \ + LZMA_API(type) intname +# else +# define LZMA_SYMVER_API(extnamever, type, intname) \ + __asm__(".symver " #intname "," extnamever); \ + extern LZMA_API(type) intname +# endif +#endif + // These allow helping the compiler in some often-executed branches, whose // result is almost always the same. #ifdef __GNUC__ diff --git a/src/liblzma/common/filter_common.c b/src/liblzma/common/filter_common.c index 9ad5d5d8e2af..590be7303feb 100644 --- a/src/liblzma/common/filter_common.c +++ b/src/liblzma/common/filter_common.c @@ -122,12 +122,16 @@ static const struct { extern LZMA_API(lzma_ret) -lzma_filters_copy(const lzma_filter *src, lzma_filter *dest, +lzma_filters_copy(const lzma_filter *src, lzma_filter *real_dest, const lzma_allocator *allocator) { - if (src == NULL || dest == NULL) + if (src == NULL || real_dest == NULL) return LZMA_PROG_ERROR; + // Use a temporary destination so that the real destination + // will never be modied if an error occurs. + lzma_filter dest[LZMA_FILTERS_MAX + 1]; + lzma_ret ret; size_t i; for (i = 0; src[i].id != LZMA_VLI_UNKNOWN; ++i) { @@ -173,18 +177,20 @@ lzma_filters_copy(const lzma_filter *src, lzma_filter *dest, } // Terminate the filter array. - assert(i <= LZMA_FILTERS_MAX + 1); + assert(i < LZMA_FILTERS_MAX + 1); dest[i].id = LZMA_VLI_UNKNOWN; dest[i].options = NULL; + // Copy it to the caller-supplied array now that we know that + // no errors occurred. + memcpy(real_dest, dest, (i + 1) * sizeof(lzma_filter)); + return LZMA_OK; error: // Free the options which we have already allocated. - while (i-- > 0) { + while (i-- > 0) lzma_free(dest[i].options, allocator); - dest[i].options = NULL; - } return ret; } diff --git a/src/liblzma/common/hardware_cputhreads.c b/src/liblzma/common/hardware_cputhreads.c index f468366a6045..5d246d2cc083 100644 --- a/src/liblzma/common/hardware_cputhreads.c +++ b/src/liblzma/common/hardware_cputhreads.c @@ -15,6 +15,18 @@ #include "tuklib_cpucores.h" +#ifdef HAVE_SYMBOL_VERSIONS_LINUX +// This is for compatibility with binaries linked against liblzma that +// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7. +LZMA_SYMVER_API("lzma_cputhreads@XZ_5.2.2", + uint32_t, lzma_cputhreads_522)(void) lzma_nothrow + __attribute__((__alias__("lzma_cputhreads_52"))); + +LZMA_SYMVER_API("lzma_cputhreads@@XZ_5.2", + uint32_t, lzma_cputhreads_52)(void) lzma_nothrow; + +#define lzma_cputhreads lzma_cputhreads_52 +#endif extern LZMA_API(uint32_t) lzma_cputhreads(void) { diff --git a/src/liblzma/common/index.c b/src/liblzma/common/index.c index a41e8f33083b..24ec3c10c2e9 100644 --- a/src/liblzma/common/index.c +++ b/src/liblzma/common/index.c @@ -656,6 +656,10 @@ lzma_index_append(lzma_index *i, const lzma_allocator *allocator, const uint32_t index_list_size_add = lzma_vli_size(unpadded_size) + lzma_vli_size(uncompressed_size); + // Check that uncompressed size will not overflow. + if (uncompressed_base + uncompressed_size > LZMA_VLI_MAX) + return LZMA_DATA_ERROR; + // Check that the file size will stay within limits. if (index_file_size(s->node.compressed_base, compressed_base + unpadded_size, s->record_count + 1, @@ -767,6 +771,9 @@ extern LZMA_API(lzma_ret) lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src, const lzma_allocator *allocator) { + if (dest == NULL || src == NULL) + return LZMA_PROG_ERROR; + const lzma_vli dest_file_size = lzma_index_file_size(dest); // Check that we don't exceed the file size limits. @@ -835,6 +842,11 @@ lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src, } } + // dest->checks includes the check types of all except the last Stream + // in dest. Set the bit for the check type of the last Stream now so + // that it won't get lost when Stream(s) from src are appended to dest. + dest->checks = lzma_index_checks(dest); + // Add all the Streams from src to dest. Update the base offsets // of each Stream from src. const index_cat_info info = { @@ -851,7 +863,7 @@ lzma_index_cat(lzma_index *restrict dest, lzma_index *restrict src, dest->total_size += src->total_size; dest->record_count += src->record_count; dest->index_list_size += src->index_list_size; - dest->checks = lzma_index_checks(dest) | src->checks; + dest->checks |= src->checks; // There's nothing else left in src than the base structure. lzma_free(src, allocator); @@ -1226,7 +1238,7 @@ lzma_index_iter_locate(lzma_index_iter *iter, lzma_vli target) // Use binary search to locate the exact Record. It is the first // Record whose uncompressed_sum is greater than target. - // This is because we want the rightmost Record that fullfills the + // This is because we want the rightmost Record that fulfills the // search criterion. It is possible that there are empty Blocks; // we don't want to return them. size_t left = 0; diff --git a/src/liblzma/common/index_hash.c b/src/liblzma/common/index_hash.c index d7a0344b76c3..5c9d314039da 100644 --- a/src/liblzma/common/index_hash.c +++ b/src/liblzma/common/index_hash.c @@ -122,7 +122,7 @@ lzma_index_hash_size(const lzma_index_hash *index_hash) /// Updates the sizes and the hash without any validation. -static lzma_ret +static void hash_append(lzma_index_hash_info *info, lzma_vli unpadded_size, lzma_vli uncompressed_size) { @@ -136,7 +136,7 @@ hash_append(lzma_index_hash_info *info, lzma_vli unpadded_size, lzma_check_update(&info->check, LZMA_CHECK_BEST, (const uint8_t *)(sizes), sizeof(sizes)); - return LZMA_OK; + return; } @@ -152,8 +152,7 @@ lzma_index_hash_append(lzma_index_hash *index_hash, lzma_vli unpadded_size, return LZMA_PROG_ERROR; // Update the hash. - return_if_error(hash_append(&index_hash->blocks, - unpadded_size, uncompressed_size)); + hash_append(&index_hash->blocks, unpadded_size, uncompressed_size); // Validate the properties of *info are still in allowed limits. if (index_hash->blocks.blocks_size > LZMA_VLI_MAX @@ -239,9 +238,9 @@ lzma_index_hash_decode(lzma_index_hash *index_hash, const uint8_t *in, index_hash->sequence = SEQ_UNCOMPRESSED; } else { // Update the hash. - return_if_error(hash_append(&index_hash->records, + hash_append(&index_hash->records, index_hash->unpadded_size, - index_hash->uncompressed_size)); + index_hash->uncompressed_size); // Verify that we don't go over the known sizes. Note // that this validation is simpler than the one used diff --git a/src/liblzma/common/memcmplen.h b/src/liblzma/common/memcmplen.h index dcfd8d6f89d1..b76a0b6327ed 100644 --- a/src/liblzma/common/memcmplen.h +++ b/src/liblzma/common/memcmplen.h @@ -80,8 +80,7 @@ lzma_memcmplen(const uint8_t *buf1, const uint8_t *buf2, #elif defined(TUKLIB_FAST_UNALIGNED_ACCESS) \ && defined(HAVE__MM_MOVEMASK_EPI8) \ - && ((defined(__GNUC__) && defined(__SSE2_MATH__)) \ - || (defined(__INTEL_COMPILER) && defined(__SSE2__)) \ + && (defined(__SSE2__) \ || (defined(_MSC_VER) && defined(_M_IX86_FP) \ && _M_IX86_FP >= 2)) // NOTE: Like above, this will use 128-bit unaligned access which diff --git a/src/liblzma/common/stream_decoder.c b/src/liblzma/common/stream_decoder.c index fdd8ff2f9a3e..6885d0c7c8cf 100644 --- a/src/liblzma/common/stream_decoder.c +++ b/src/liblzma/common/stream_decoder.c @@ -18,15 +18,14 @@ typedef struct { enum { SEQ_STREAM_HEADER, SEQ_BLOCK_HEADER, - SEQ_BLOCK, + SEQ_BLOCK_INIT, + SEQ_BLOCK_RUN, SEQ_INDEX, SEQ_STREAM_FOOTER, SEQ_STREAM_PADDING, } sequence; - /// Block or Metadata decoder. This takes little memory and the same - /// data structure can be used to decode every Block Header, so it's - /// a good idea to have a separate lzma_next_coder structure for it. + /// Block decoder lzma_next_coder block_decoder; /// Block options decoded by the Block Header decoder and used by @@ -63,9 +62,9 @@ typedef struct { /// If true, we will decode concatenated Streams that possibly have /// Stream Padding between or after them. LZMA_STREAM_END is returned - /// once the application isn't giving us any new input, and we aren't - /// in the middle of a Stream, and possible Stream Padding is a - /// multiple of four bytes. + /// once the application isn't giving us any new input (LZMA_FINISH), + /// and we aren't in the middle of a Stream, and possible + /// Stream Padding is a multiple of four bytes. bool concatenated; /// When decoding concatenated Streams, this is true as long as we @@ -187,6 +186,15 @@ stream_decode(void *coder_ptr, const lzma_allocator *allocator, return LZMA_OK; coder->pos = 0; + coder->sequence = SEQ_BLOCK_INIT; + } + + // Fall through + + case SEQ_BLOCK_INIT: { + // Checking memusage and doing the initialization needs + // its own sequence point because we need to be able to + // retry if we return LZMA_MEMLIMIT_ERROR. // Version 1 is needed to support the .ignore_check option. coder->block_options.version = 1; @@ -240,17 +248,17 @@ stream_decode(void *coder_ptr, const lzma_allocator *allocator, coder->block_options.filters = NULL; - // Check if memory usage calculation and Block enocoder + // Check if memory usage calculation and Block decoder // initialization succeeded. if (ret != LZMA_OK) return ret; - coder->sequence = SEQ_BLOCK; + coder->sequence = SEQ_BLOCK_RUN; } // Fall through - case SEQ_BLOCK: { + case SEQ_BLOCK_RUN: { const lzma_ret ret = coder->block_decoder.code( coder->block_decoder.coder, allocator, in, in_pos, in_size, out, out_pos, out_size, diff --git a/src/liblzma/common/stream_encoder_mt.c b/src/liblzma/common/stream_encoder_mt.c index 01e40339750a..819b22733b68 100644 --- a/src/liblzma/common/stream_encoder_mt.c +++ b/src/liblzma/common/stream_encoder_mt.c @@ -715,6 +715,10 @@ stream_encode_mt(void *coder_ptr, const lzma_allocator *allocator, ret = lzma_index_append(coder->index, allocator, unpadded_size, uncompressed_size); + if (ret != LZMA_OK) { + threads_stop(coder, false); + return ret; + } // If we didn't fill the output buffer yet, // try to read more data. Maybe the next @@ -724,8 +728,7 @@ stream_encode_mt(void *coder_ptr, const lzma_allocator *allocator, } if (ret != LZMA_OK) { - // coder->thread_error was set or - // lzma_index_append() failed. + // coder->thread_error was set. threads_stop(coder, false); return ret; } @@ -1075,6 +1078,31 @@ stream_encoder_mt_init(lzma_next_coder *next, const lzma_allocator *allocator, } +#ifdef HAVE_SYMBOL_VERSIONS_LINUX +// These are for compatibility with binaries linked against liblzma that +// has been patched with xz-5.2.2-compat-libs.patch from RHEL/CentOS 7. +// Actually that patch didn't create lzma_stream_encoder_mt@XZ_5.2.2 +// but it has been added here anyway since someone might misread the +// RHEL patch and think both @XZ_5.1.2alpha and @XZ_5.2.2 exist. +LZMA_SYMVER_API("lzma_stream_encoder_mt@XZ_5.1.2alpha", + lzma_ret, lzma_stream_encoder_mt_512a)( + lzma_stream *strm, const lzma_mt *options) + lzma_nothrow lzma_attr_warn_unused_result + __attribute__((__alias__("lzma_stream_encoder_mt_52"))); + +LZMA_SYMVER_API("lzma_stream_encoder_mt@XZ_5.2.2", + lzma_ret, lzma_stream_encoder_mt_522)( + lzma_stream *strm, const lzma_mt *options) + lzma_nothrow lzma_attr_warn_unused_result + __attribute__((__alias__("lzma_stream_encoder_mt_52"))); + +LZMA_SYMVER_API("lzma_stream_encoder_mt@@XZ_5.2", + lzma_ret, lzma_stream_encoder_mt_52)( + lzma_stream *strm, const lzma_mt *options) + lzma_nothrow lzma_attr_warn_unused_result; + +#define lzma_stream_encoder_mt lzma_stream_encoder_mt_52 +#endif extern LZMA_API(lzma_ret) lzma_stream_encoder_mt(lzma_stream *strm, const lzma_mt *options) { @@ -1090,6 +1118,23 @@ lzma_stream_encoder_mt(lzma_stream *strm, const lzma_mt *options) } +#ifdef HAVE_SYMBOL_VERSIONS_LINUX +LZMA_SYMVER_API("lzma_stream_encoder_mt_memusage@XZ_5.1.2alpha", + uint64_t, lzma_stream_encoder_mt_memusage_512a)( + const lzma_mt *options) lzma_nothrow lzma_attr_pure + __attribute__((__alias__("lzma_stream_encoder_mt_memusage_52"))); + +LZMA_SYMVER_API("lzma_stream_encoder_mt_memusage@XZ_5.2.2", + uint64_t, lzma_stream_encoder_mt_memusage_522)( + const lzma_mt *options) lzma_nothrow lzma_attr_pure + __attribute__((__alias__("lzma_stream_encoder_mt_memusage_52"))); + +LZMA_SYMVER_API("lzma_stream_encoder_mt_memusage@@XZ_5.2", + uint64_t, lzma_stream_encoder_mt_memusage_52)( + const lzma_mt *options) lzma_nothrow lzma_attr_pure; + +#define lzma_stream_encoder_mt_memusage lzma_stream_encoder_mt_memusage_52 +#endif // This function name is a monster but it's consistent with the older // monster names. :-( 31 chars is the max that C99 requires so in that // sense it's not too long. ;-) diff --git a/src/liblzma/liblzma.map b/src/liblzma/liblzma_generic.map index f53a4ea30a3c..8cca05bdeead 100644 --- a/src/liblzma/liblzma.map +++ b/src/liblzma/liblzma_generic.map @@ -93,6 +93,9 @@ global: lzma_vli_decode; lzma_vli_encode; lzma_vli_size; + +local: + *; }; XZ_5.2 { @@ -102,7 +105,4 @@ global: lzma_get_progress; lzma_stream_encoder_mt; lzma_stream_encoder_mt_memusage; - -local: - *; } XZ_5.0; diff --git a/src/liblzma/liblzma_linux.map b/src/liblzma/liblzma_linux.map new file mode 100644 index 000000000000..4be882cc2914 --- /dev/null +++ b/src/liblzma/liblzma_linux.map @@ -0,0 +1,123 @@ +XZ_5.0 { +global: + lzma_alone_decoder; + lzma_alone_encoder; + lzma_auto_decoder; + lzma_block_buffer_bound; + lzma_block_buffer_decode; + lzma_block_buffer_encode; + lzma_block_compressed_size; + lzma_block_decoder; + lzma_block_encoder; + lzma_block_header_decode; + lzma_block_header_encode; + lzma_block_header_size; + lzma_block_total_size; + lzma_block_unpadded_size; + lzma_check_is_supported; + lzma_check_size; + lzma_code; + lzma_crc32; + lzma_crc64; + lzma_easy_buffer_encode; + lzma_easy_decoder_memusage; + lzma_easy_encoder; + lzma_easy_encoder_memusage; + lzma_end; + lzma_filter_decoder_is_supported; + lzma_filter_encoder_is_supported; + lzma_filter_flags_decode; + lzma_filter_flags_encode; + lzma_filter_flags_size; + lzma_filters_copy; + lzma_filters_update; + lzma_get_check; + lzma_index_append; + lzma_index_block_count; + lzma_index_buffer_decode; + lzma_index_buffer_encode; + lzma_index_cat; + lzma_index_checks; + lzma_index_decoder; + lzma_index_dup; + lzma_index_encoder; + lzma_index_end; + lzma_index_file_size; + lzma_index_hash_append; + lzma_index_hash_decode; + lzma_index_hash_end; + lzma_index_hash_init; + lzma_index_hash_size; + lzma_index_init; + lzma_index_iter_init; + lzma_index_iter_locate; + lzma_index_iter_next; + lzma_index_iter_rewind; + lzma_index_memusage; + lzma_index_memused; + lzma_index_size; + lzma_index_stream_count; + lzma_index_stream_flags; + lzma_index_stream_padding; + lzma_index_stream_size; + lzma_index_total_size; + lzma_index_uncompressed_size; + lzma_lzma_preset; + lzma_memlimit_get; + lzma_memlimit_set; + lzma_memusage; + lzma_mf_is_supported; + lzma_mode_is_supported; + lzma_physmem; + lzma_properties_decode; + lzma_properties_encode; + lzma_properties_size; + lzma_raw_buffer_decode; + lzma_raw_buffer_encode; + lzma_raw_decoder; + lzma_raw_decoder_memusage; + lzma_raw_encoder; + lzma_raw_encoder_memusage; + lzma_stream_buffer_bound; + lzma_stream_buffer_decode; + lzma_stream_buffer_encode; + lzma_stream_decoder; + lzma_stream_encoder; + lzma_stream_flags_compare; + lzma_stream_footer_decode; + lzma_stream_footer_encode; + lzma_stream_header_decode; + lzma_stream_header_encode; + lzma_version_number; + lzma_version_string; + lzma_vli_decode; + lzma_vli_encode; + lzma_vli_size; + +local: + *; +}; + +XZ_5.2 { +global: + lzma_block_uncomp_encode; + lzma_cputhreads; + lzma_get_progress; + lzma_stream_encoder_mt; + lzma_stream_encoder_mt_memusage; +} XZ_5.0; + +XZ_5.1.2alpha { +global: + lzma_stream_encoder_mt; + lzma_stream_encoder_mt_memusage; +} XZ_5.0; + +XZ_5.2.2 { +global: + lzma_block_uncomp_encode; + lzma_cputhreads; + lzma_get_progress; + lzma_stream_encoder_mt; + lzma_stream_encoder_mt_memusage; +} XZ_5.1.2alpha; diff --git a/src/liblzma/lz/lz_decoder.c b/src/liblzma/lz/lz_decoder.c index 09b574388f59..ab6af0dd99b3 100644 --- a/src/liblzma/lz/lz_decoder.c +++ b/src/liblzma/lz/lz_decoder.c @@ -304,8 +304,14 @@ lzma_lz_decoder_memusage(size_t dictionary_size) extern void -lzma_lz_decoder_uncompressed(void *coder_ptr, lzma_vli uncompressed_size) +lzma_lz_decoder_uncompressed(void *coder_ptr, lzma_vli uncompressed_size, + bool allow_eopm) { lzma_coder *coder = coder_ptr; - coder->lz.set_uncompressed(coder->lz.coder, uncompressed_size); + + if (uncompressed_size == LZMA_VLI_UNKNOWN) + allow_eopm = true; + + coder->lz.set_uncompressed(coder->lz.coder, uncompressed_size, + allow_eopm); } diff --git a/src/liblzma/lz/lz_decoder.h b/src/liblzma/lz/lz_decoder.h index 754ccf37c6a4..e6d7ab2a4107 100644 --- a/src/liblzma/lz/lz_decoder.h +++ b/src/liblzma/lz/lz_decoder.h @@ -62,8 +62,10 @@ typedef struct { void (*reset)(void *coder, const void *options); - /// Set the uncompressed size - void (*set_uncompressed)(void *coder, lzma_vli uncompressed_size); + /// Set the uncompressed size. If uncompressed_size == LZMA_VLI_UNKNOWN + /// then allow_eopm will always be true. + void (*set_uncompressed)(void *coder, lzma_vli uncompressed_size, + bool allow_eopm); /// Free allocated resources void (*end)(void *coder, const lzma_allocator *allocator); @@ -91,7 +93,7 @@ extern lzma_ret lzma_lz_decoder_init(lzma_next_coder *next, extern uint64_t lzma_lz_decoder_memusage(size_t dictionary_size); extern void lzma_lz_decoder_uncompressed( - void *coder, lzma_vli uncompressed_size); + void *coder, lzma_vli uncompressed_size, bool allow_eopm); ////////////////////// diff --git a/src/liblzma/lz/lz_encoder.c b/src/liblzma/lz/lz_encoder.c index 9a74b7c47ce8..e6a49fb19e31 100644 --- a/src/liblzma/lz/lz_encoder.c +++ b/src/liblzma/lz/lz_encoder.c @@ -585,32 +585,28 @@ lzma_lz_encoder_init(lzma_next_coder *next, const lzma_allocator *allocator, extern LZMA_API(lzma_bool) lzma_mf_is_supported(lzma_match_finder mf) { - bool ret = false; - + switch (mf) { #ifdef HAVE_MF_HC3 - if (mf == LZMA_MF_HC3) - ret = true; + case LZMA_MF_HC3: + return true; #endif - #ifdef HAVE_MF_HC4 - if (mf == LZMA_MF_HC4) - ret = true; + case LZMA_MF_HC4: + return true; #endif - #ifdef HAVE_MF_BT2 - if (mf == LZMA_MF_BT2) - ret = true; + case LZMA_MF_BT2: + return true; #endif - #ifdef HAVE_MF_BT3 - if (mf == LZMA_MF_BT3) - ret = true; + case LZMA_MF_BT3: + return true; #endif - #ifdef HAVE_MF_BT4 - if (mf == LZMA_MF_BT4) - ret = true; + case LZMA_MF_BT4: + return true; #endif - - return ret; + default: + return false; + } } diff --git a/src/liblzma/lzma/lzma2_decoder.c b/src/liblzma/lzma/lzma2_decoder.c index cf1b5110aca8..105a28dc97ed 100644 --- a/src/liblzma/lzma/lzma2_decoder.c +++ b/src/liblzma/lzma/lzma2_decoder.c @@ -139,7 +139,7 @@ lzma2_decode(void *coder_ptr, lzma_dict *restrict dict, coder->uncompressed_size += in[(*in_pos)++] + 1U; coder->sequence = SEQ_COMPRESSED_0; coder->lzma.set_uncompressed(coder->lzma.coder, - coder->uncompressed_size); + coder->uncompressed_size, false); break; case SEQ_COMPRESSED_0: diff --git a/src/liblzma/lzma/lzma2_encoder.c b/src/liblzma/lzma/lzma2_encoder.c index 63588ee30c6b..6914f2793b26 100644 --- a/src/liblzma/lzma/lzma2_encoder.c +++ b/src/liblzma/lzma/lzma2_encoder.c @@ -378,6 +378,9 @@ lzma_lzma2_encoder_memusage(const void *options) extern lzma_ret lzma_lzma2_props_encode(const void *options, uint8_t *out) { + if (options == NULL) + return LZMA_PROG_ERROR; + const lzma_options_lzma *const opt = options; uint32_t d = my_max(opt->dict_size, LZMA_DICT_SIZE_MIN); diff --git a/src/liblzma/lzma/lzma_decoder.c b/src/liblzma/lzma/lzma_decoder.c index e605a0a916b2..f56ccaf77fd3 100644 --- a/src/liblzma/lzma/lzma_decoder.c +++ b/src/liblzma/lzma/lzma_decoder.c @@ -238,6 +238,11 @@ typedef struct { /// payload marker is expected. lzma_vli uncompressed_size; + /// True if end of payload marker (EOPM) is allowed even when + /// uncompressed_size is known; false if EOPM must not be present. + /// This is ignored if uncompressed_size == LZMA_VLI_UNKNOWN. + bool allow_eopm; + //////////////////////////////// // State of incomplete symbol // //////////////////////////////// @@ -343,12 +348,24 @@ lzma_decode(void *coder_ptr, lzma_dict *restrict dictptr, lzma_ret ret = LZMA_OK; - // If uncompressed size is known, there must be no end of payload - // marker. - const bool no_eopm = coder->uncompressed_size - != LZMA_VLI_UNKNOWN; - if (no_eopm && coder->uncompressed_size < dict.limit - dict.pos) + // This is true when the next LZMA symbol is allowed to be EOPM. + // That is, if this is false, then EOPM is considered + // an invalid symbol and we will return LZMA_DATA_ERROR. + // + // EOPM is always required (not just allowed) when + // the uncompressed size isn't known. When uncompressed size + // is known, eopm_is_valid may be set to true later. + bool eopm_is_valid = coder->uncompressed_size == LZMA_VLI_UNKNOWN; + + // If uncompressed size is known and there is enough output space + // to decode all the data, limit the available buffer space so that + // the main loop won't try to decode past the end of the stream. + bool might_finish_without_eopm = false; + if (coder->uncompressed_size != LZMA_VLI_UNKNOWN + && coder->uncompressed_size <= dict.limit - dict.pos) { dict.limit = dict.pos + (size_t)(coder->uncompressed_size); + might_finish_without_eopm = true; + } // The main decoder loop. The "switch" is used to restart the decoder at // correct location. Once restarted, the "switch" is no longer used. @@ -361,8 +378,32 @@ lzma_decode(void *coder_ptr, lzma_dict *restrict dictptr, case SEQ_NORMALIZE: case SEQ_IS_MATCH: - if (unlikely(no_eopm && dict.pos == dict.limit)) - break; + if (unlikely(might_finish_without_eopm + && dict.pos == dict.limit)) { + // In rare cases there is a useless byte that needs + // to be read anyway. + rc_normalize(SEQ_NORMALIZE); + + // If the range decoder state is such that we can + // be at the end of the LZMA stream, then the + // decoding is finished. + if (rc_is_finished(rc)) { + ret = LZMA_STREAM_END; + goto out; + } + + // If the caller hasn't allowed EOPM to be present + // together with known uncompressed size, then the + // LZMA stream is corrupt. + if (!coder->allow_eopm) { + ret = LZMA_DATA_ERROR; + goto out; + } + + // Otherwise continue decoding with the expectation + // that the next LZMA symbol is EOPM. + eopm_is_valid = true; + } rc_if_0(coder->is_match[state][pos_state], SEQ_IS_MATCH) { rc_update_0(coder->is_match[state][pos_state]); @@ -658,11 +699,18 @@ lzma_decode(void *coder_ptr, lzma_dict *restrict dictptr, if (rep0 == UINT32_MAX) { // End of payload marker was - // found. It must not be - // present if uncompressed - // size is known. - if (coder->uncompressed_size - != LZMA_VLI_UNKNOWN) { + // found. It may only be + // present if + // - uncompressed size is + // unknown or + // - after known uncompressed + // size amount of bytes has + // been decompressed and + // caller has indicated + // that EOPM might be used + // (it's not allowed in + // LZMA2). + if (!eopm_is_valid) { ret = LZMA_DATA_ERROR; goto out; } @@ -671,7 +719,9 @@ lzma_decode(void *coder_ptr, lzma_dict *restrict dictptr, // LZMA1 stream with // end-of-payload marker. rc_normalize(SEQ_EOPM); - ret = LZMA_STREAM_END; + ret = rc_is_finished(rc) + ? LZMA_STREAM_END + : LZMA_DATA_ERROR; goto out; } } @@ -793,9 +843,6 @@ lzma_decode(void *coder_ptr, lzma_dict *restrict dictptr, } } - rc_normalize(SEQ_NORMALIZE); - coder->sequence = SEQ_IS_MATCH; - out: // Save state @@ -822,24 +869,21 @@ out: if (coder->uncompressed_size != LZMA_VLI_UNKNOWN) { coder->uncompressed_size -= dict.pos - dict_start; - // Since there cannot be end of payload marker if the - // uncompressed size was known, we check here if we - // finished decoding. + // If we have gotten all the output but the decoder wants + // to write more output, the file is corrupt. There are + // three SEQ values where output is produced. if (coder->uncompressed_size == 0 && ret == LZMA_OK - && coder->sequence != SEQ_NORMALIZE) - ret = coder->sequence == SEQ_IS_MATCH - ? LZMA_STREAM_END : LZMA_DATA_ERROR; + && (coder->sequence == SEQ_LITERAL_WRITE + || coder->sequence == SEQ_SHORTREP + || coder->sequence == SEQ_COPY)) + ret = LZMA_DATA_ERROR; } - // We can do an additional check in the range decoder to catch some - // corrupted files. if (ret == LZMA_STREAM_END) { - if (!rc_is_finished(coder->rc)) - ret = LZMA_DATA_ERROR; - // Reset the range decoder so that it is ready to reinitialize // for a new LZMA2 chunk. rc_reset(coder->rc); + coder->sequence = SEQ_IS_MATCH; } return ret; @@ -848,10 +892,12 @@ out: static void -lzma_decoder_uncompressed(void *coder_ptr, lzma_vli uncompressed_size) +lzma_decoder_uncompressed(void *coder_ptr, lzma_vli uncompressed_size, + bool allow_eopm) { lzma_lzma1_decoder *coder = coder_ptr; coder->uncompressed_size = uncompressed_size; + coder->allow_eopm = allow_eopm; } @@ -977,7 +1023,7 @@ lzma_decoder_init(lzma_lz_decoder *lz, const lzma_allocator *allocator, lz, allocator, options, lz_options)); lzma_decoder_reset(lz->coder, options); - lzma_decoder_uncompressed(lz->coder, LZMA_VLI_UNKNOWN); + lzma_decoder_uncompressed(lz->coder, LZMA_VLI_UNKNOWN, true); return LZMA_OK; } diff --git a/src/liblzma/lzma/lzma_encoder.c b/src/liblzma/lzma/lzma_encoder.c index 07d2b87bc65b..c1552f194451 100644 --- a/src/liblzma/lzma/lzma_encoder.c +++ b/src/liblzma/lzma/lzma_encoder.c @@ -658,6 +658,9 @@ lzma_lzma_lclppb_encode(const lzma_options_lzma *options, uint8_t *byte) extern lzma_ret lzma_lzma_props_encode(const void *options, uint8_t *out) { + if (options == NULL) + return LZMA_PROG_ERROR; + const lzma_options_lzma *const opt = options; if (lzma_lzma_lclppb_encode(opt, out)) diff --git a/src/liblzma/validate_map.sh b/src/liblzma/validate_map.sh index 3aee46687c38..2bf6f8b98cbb 100755..100644 --- a/src/liblzma/validate_map.sh +++ b/src/liblzma/validate_map.sh @@ -2,7 +2,79 @@ ############################################################################### # -# Check liblzma.map for certain types of errors +# Check liblzma_*.map for certain types of errors. +# +# liblzma_generic.map is for FreeBSD and Solaris and possibly others +# except GNU/Linux. +# +# liblzma_linux.map is for GNU/Linux only. This and the matching extra code +# in the .c files make liblzma >= 5.2.7 compatible with binaries that were +# linked against ill-patched liblzma in RHEL/CentOS 7. By providing the +# compatibility in official XZ Utils release will hopefully prevent people +# from further copying the broken patch to other places when they want +# compatibility with binaries linked on RHEL/CentOS 7. The long version +# of the story: +# +# RHEL/CentOS 7 shipped with 5.1.2alpha, including the threaded +# encoder that is behind #ifdef LZMA_UNSTABLE in the API headers. +# In 5.1.2alpha these symbols are under XZ_5.1.2alpha in liblzma.map. +# API/ABI compatibility tracking isn't done between development +# releases so newer releases didn't have XZ_5.1.2alpha anymore. +# +# Later RHEL/CentOS 7 updated xz to 5.2.2 but they wanted to keep +# the exported symbols compatible with 5.1.2alpha. After checking +# the ABI changes it turned out that >= 5.2.0 ABI is backward +# compatible with the threaded encoder functions from 5.1.2alpha +# (but not vice versa as fixes and extensions to these functions +# were made between 5.1.2alpha and 5.2.0). +# +# In RHEL/CentOS 7, XZ Utils 5.2.2 was patched with +# xz-5.2.2-compat-libs.patch to modify liblzma.map: +# +# - XZ_5.1.2alpha was added with lzma_stream_encoder_mt and +# lzma_stream_encoder_mt_memusage. This matched XZ Utils 5.1.2alpha. +# +# - XZ_5.2 was replaced with XZ_5.2.2. It is clear that this was +# an error; the intention was to keep using XZ_5.2 (XZ_5.2.2 +# has never been used in XZ Utils). So XZ_5.2.2 lists all +# symbols that were listed under XZ_5.2 before the patch. +# lzma_stream_encoder_mt and _mt_memusage are included too so +# they are listed both here and under XZ_5.1.2alpha. +# +# The patch didn't add any __asm__(".symver ...") lines to the .c +# files. Thus the resulting liblzma.so exports the threaded encoder +# functions under XZ_5.1.2alpha only. Listing the two functions +# also under XZ_5.2.2 in liblzma.map has no effect without +# matching .symver lines. +# +# The lack of XZ_5.2 in RHEL/CentOS 7 means that binaries linked +# against unpatched XZ Utils 5.2.x won't run on RHEL/CentOS 7. +# This is unfortunate but this alone isn't too bad as the problem +# is contained within RHEL/CentOS 7 and doesn't affect users +# of other distributions. It could also be fixed internally in +# RHEL/CentOS 7. +# +# The second problem is more serious: In XZ Utils 5.2.2 the API +# headers don't have #ifdef LZMA_UNSTABLE for obvious reasons. +# This is true in RHEL/CentOS 7 version too. Thus now programs +# using new APIs can be compiled without an extra #define. However, +# the programs end up depending on symbol version XZ_5.1.2alpha +# (and possibly also XZ_5.2.2) instead of XZ_5.2 as they would +# with an unpatched XZ Utils 5.2.2. This means that such binaries +# won't run on other distributions shipping XZ Utils >= 5.2.0 as +# they don't provide XZ_5.1.2alpha or XZ_5.2.2; they only provide +# XZ_5.2 (and XZ_5.0). (This includes RHEL/CentOS 8 as the patch +# luckily isn't included there anymore with XZ Utils 5.2.4.) +# +# Binaries built by RHEL/CentOS 7 users get distributed and then +# people wonder why they don't run on some other distribution. +# Seems that people have found out about the patch and been copying +# it to some build scripts, seemingly curing the symptoms but +# actually spreading the illness further and outside RHEL/CentOS 7. +# Adding compatibility in an official XZ Utils release should work +# as a vaccine against this ill patch and stop it from spreading. +# The vaccine is kept GNU/Linux-only as other OSes should be immune +# (hopefully it hasn't spread via some build script to other OSes). # # Author: Lasse Collin # @@ -18,11 +90,11 @@ STATUS=0 cd "$(dirname "$0")" -# Get the list of symbols that aren't defined in liblzma.map. +# Get the list of symbols that aren't defined in liblzma_generic.map. SYMS=$(sed -n 's/^extern LZMA_API([^)]*) \([a-z0-9_]*\)(.*$/\1;/p' \ api/lzma/*.h \ | sort \ - | grep -Fve "$(sed '/[{}:*]/d;/^$/d;s/^ //' liblzma.map)") + | grep -Fve "$(sed '/[{}:*]/d;/^$/d;s/^ //' liblzma_generic.map)") # Check that there are no old alpha or beta versions listed. VER=$(cd ../.. && sh build-aux/version.sh) @@ -30,21 +102,41 @@ NAMES= case $VER in *alpha | *beta) NAMES=$(sed -n 's/^.*XZ_\([^ ]*\)\(alpha\|beta\) .*$/\1\2/p' \ - liblzma.map | grep -Fv "$VER") + liblzma_generic.map | grep -Fv "$VER") ;; esac # Check for duplicate lines. It can catch missing dependencies. -DUPS=$(sort liblzma.map | sed '/^$/d;/^global:$/d' | uniq -d) +DUPS=$(sort liblzma_generic.map | sed '/^$/d;/^global:$/d' | uniq -d) + +# Check that liblzma_linux.map is in sync with liblzma_generic.map. +# The RHEL/CentOS 7 compatibility symbols are in a fixed location +# so it makes it easy to remove them for comparison with liblzma_generic.map. +# +# NOTE: Putting XZ_5.2 before the compatibility symbols XZ_5.1.2alpha +# and XZ_5.2.2 in liblzma_linux.map is important: If liblzma_linux.map is +# incorrectly used without #define HAVE_SYMBOL_VERSIONS_LINUX, only the first +# occurrence of each function name will be used from liblzma_linux.map; +# the rest are ignored by the linker. Thus having XZ_5.2 before the +# compatibility symbols means that @@XZ_5.2 will be used for the symbols +# listed under XZ_5.2 {...} and the same function names later in +# the file under XZ_5.1.2alpha {...} and XZ_5.2.2 {...} will be +# ignored (@XZ_5.1.2alpha or @XZ_5.2.2 won't be added at all when +# the #define HAVE_SYMBOL_VERSIONS_LINUX isn't used). +IN_SYNC= +if ! sed '109,123d' liblzma_linux.map \ + | cmp -s - liblzma_generic.map; then + IN_SYNC=no +fi # Print error messages if needed. -if test -n "$SYMS$NAMES$DUPS"; then +if test -n "$SYMS$NAMES$DUPS$IN_SYNC"; then echo - echo 'validate_map.sh found problems from liblzma.map:' + echo 'validate_map.sh found problems from liblzma_*.map:' echo if test -n "$SYMS"; then - echo 'liblzma.map lacks the following symbols:' + echo 'liblzma_generic.map lacks the following symbols:' echo "$SYMS" echo fi @@ -61,6 +153,11 @@ if test -n "$SYMS$NAMES$DUPS"; then echo fi + if test -n "$IN_SYNC"; then + echo "liblzma_generic.map and liblzma_linux.map aren't in sync" + echo + fi + STATUS=1 fi diff --git a/src/xz/coder.c b/src/xz/coder.c index 85f954393d8b..4f51e8d5c0e0 100644 --- a/src/xz/coder.c +++ b/src/xz/coder.c @@ -513,8 +513,12 @@ coder_init(file_pair *pair) // is needed, because we don't want to do use // passthru mode with --test. if (opt_mode == MODE_DECOMPRESS - && opt_stdout && opt_force) + && opt_stdout && opt_force) { + // These are needed for progress info. + strm.total_in = 0; + strm.total_out = 0; return CODER_INIT_PASSTHRU; + } ret = LZMA_FORMAT_ERROR; break; @@ -542,10 +546,30 @@ coder_init(file_pair *pair) // memory usage limit in case it happens in the first // Block of the first Stream, which is where it very // probably will happen if it is going to happen. + // + // This will also catch unsupported check type which + // we treat as a warning only. If there are empty + // concatenated Streams with unsupported check type then + // the message can be shown more than once here. The loop + // is used in case there is first a warning about + // unsupported check type and then the first Block + // would exceed the memlimit. if (ret == LZMA_OK && init_format != FORMAT_RAW) { strm.next_out = NULL; strm.avail_out = 0; - ret = lzma_code(&strm, LZMA_RUN); + while ((ret = lzma_code(&strm, LZMA_RUN)) + == LZMA_UNSUPPORTED_CHECK) + message_warning("%s: %s", pair->src_name, + message_strm(ret)); + + // With --single-stream lzma_code won't wait for + // LZMA_FINISH and thus it can return LZMA_STREAM_END + // if the file has no uncompressed data inside. + // So treat LZMA_STREAM_END as LZMA_OK here. + // When lzma_code() is called again in coder_normal() + // it will return LZMA_STREAM_END again. + if (ret == LZMA_STREAM_END) + ret = LZMA_OK; } #endif } @@ -756,9 +780,9 @@ coder_normal(file_pair *pair) } else if (ret != LZMA_OK) { // Determine if the return value indicates that we - // won't continue coding. - const bool stop = ret != LZMA_NO_CHECK - && ret != LZMA_UNSUPPORTED_CHECK; + // won't continue coding. LZMA_NO_CHECK would be + // here too if LZMA_TELL_ANY_CHECK was used. + const bool stop = ret != LZMA_UNSUPPORTED_CHECK; if (stop) { // Write the remaining bytes even if something @@ -907,6 +931,15 @@ coder_run(const char *filename) mytime_set_start_time(); // Initialize the progress indicator. + // + // NOTE: When reading from stdin, fstat() + // isn't called on it and thus src_st.st_size + // is zero. If stdin pointed to a regular + // file, it would still be possible to know + // the file size but then we would also need + // to take into account the current reading + // position since with stdin it isn't + // necessarily at the beginning of the file. const bool is_passthru = init_ret == CODER_INIT_PASSTHRU; const uint64_t in_size diff --git a/src/xz/file_io.c b/src/xz/file_io.c index 0ba8db8fbc4c..41e4c2d893f8 100644 --- a/src/xz/file_io.c +++ b/src/xz/file_io.c @@ -330,14 +330,14 @@ io_unlink(const char *name, const struct stat *known_st) // it is possible that the user has put a new file in place // of the original file, and in that case it obviously // shouldn't be removed. - message_error(_("%s: File seems to have been moved, " + message_warning(_("%s: File seems to have been moved, " "not removing"), name); else #endif // There's a race condition between lstat() and unlink() // but at least we have tried to avoid removing wrong file. if (unlink(name)) - message_error(_("%s: Cannot remove: %s"), + message_warning(_("%s: Cannot remove: %s"), name, strerror(errno)); return; @@ -368,7 +368,14 @@ io_copy_attrs(const file_pair *pair) mode_t mode; - if (fchown(pair->dest_fd, (uid_t)(-1), pair->src_st.st_gid)) { + // With BSD semantics the new dest file may have a group that + // does not belong to the user. If the src file has the same gid + // nothing has to be done. Nevertheless OpenBSD fchown(2) fails + // in this case which seems to be POSIX compliant. As there is + // nothing to do, skip the system call. + if (pair->dest_st.st_gid != pair->src_st.st_gid + && fchown(pair->dest_fd, (uid_t)(-1), + pair->src_st.st_gid)) { message_warning(_("%s: Cannot set the file group: %s"), pair->dest_name, strerror(errno)); // We can still safely copy some additional permissions: @@ -536,8 +543,9 @@ io_open_src_real(file_pair *pair) } // Symlinks are not followed unless writing to stdout or --force - // was used. - const bool follow_symlinks = opt_stdout || opt_force; + // or --keep was used. + const bool follow_symlinks + = opt_stdout || opt_force || opt_keep_original; // We accept only regular files if we are writing the output // to disk too. bzip2 allows overriding this with --force but @@ -674,7 +682,7 @@ io_open_src_real(file_pair *pair) } #ifndef TUKLIB_DOSLIKE - if (reg_files_only && !opt_force) { + if (reg_files_only && !opt_force && !opt_keep_original) { if (pair->src_st.st_mode & (S_ISUID | S_ISGID)) { // gzip rejects setuid and setgid files even // when --force was used. bzip2 doesn't check @@ -683,7 +691,7 @@ io_open_src_real(file_pair *pair) // and setgid bits there. // // We accept setuid and setgid files if - // --force was used. We drop these bits + // --force or --keep was used. We drop these bits // explicitly in io_copy_attr(). message_warning(_("%s: File has setuid or " "setgid bit set, skipping"), @@ -747,6 +755,10 @@ io_open_src(const char *src_name) // a statically allocated structure. static file_pair pair; + // This implicitly also initializes src_st.st_size to zero + // which is expected to be <= 0 by default. fstat() isn't + // called when reading from standard input but src_st.st_size + // is still read. pair = (file_pair){ .src_name = src_name, .dest_name = NULL, diff --git a/src/xz/hardware.c b/src/xz/hardware.c index e746cf91a76f..da3ed3f1cd11 100644 --- a/src/xz/hardware.c +++ b/src/xz/hardware.c @@ -91,7 +91,13 @@ hardware_memlimit_set(uint64_t new_memlimit, // Use a value less than SIZE_MAX so that there's some room // for the xz program and so on. Don't use 4000 MiB because // it could look like someone mixed up base-2 and base-10. +#ifdef __mips__ + // For MIPS32, due to architectural pecularities, + // the limit is even lower. + const uint64_t limit_max = UINT64_C(2000) << 20; +#else const uint64_t limit_max = UINT64_C(4020) << 20; +#endif // UINT64_MAX is a special case for the string "max" so // that has to be handled specially. diff --git a/src/xz/message.c b/src/xz/message.c index aa915d2d60d1..da90420fb897 100644 --- a/src/xz/message.c +++ b/src/xz/message.c @@ -355,11 +355,8 @@ progress_speed(uint64_t uncompressed_pos, uint64_t elapsed) if (elapsed < 3000) return ""; - static const char unit[][8] = { - "KiB/s", - "MiB/s", - "GiB/s", - }; + // The first character of KiB/s, MiB/s, or GiB/s: + static const char unit[] = { 'K', 'M', 'G' }; size_t unit_index = 0; @@ -381,7 +378,7 @@ progress_speed(uint64_t uncompressed_pos, uint64_t elapsed) // - 999 KiB/s // Use big enough buffer to hold e.g. a multibyte decimal point. static char buf[16]; - snprintf(buf, sizeof(buf), "%.*f %s", + snprintf(buf, sizeof(buf), "%.*f %ciB/s", speed > 9.9 ? 0 : 1, speed, unit[unit_index]); return buf; } @@ -1116,6 +1113,9 @@ message_help(bool long_help) " -k, --keep keep (don't delete) input files\n" " -f, --force force overwrite of output file and (de)compress links\n" " -c, --stdout write to standard output and don't delete input files")); + // NOTE: --to-stdout isn't included above because it's not + // the recommended spelling. It was copied from gzip but other + // compressors with gzip-like syntax don't support it. if (long_help) { puts(_( diff --git a/src/xz/xz.1 b/src/xz/xz.1 index 540d1364dccb..066b13a213eb 100644 --- a/src/xz/xz.1 +++ b/src/xz/xz.1 @@ -5,7 +5,7 @@ .\" This file has been put into the public domain. .\" You can do whatever you want with this file. .\" -.TH XZ 1 "2020-02-01" "Tukaani" "XZ Utils" +.TH XZ 1 "2022-10-25" "Tukaani" "XZ Utils" . .SH NAME xz, unxz, xzcat, lzma, unlzma, lzcat \- Compress or decompress .xz and .lzma files @@ -183,7 +183,8 @@ is removed unless was specified. The source .I file -is never removed if the output is written to standard output. +is never removed if the output is written to standard output +or if an error occurs. .PP Sending .B SIGINFO @@ -221,7 +222,7 @@ To prevent uncomfortable surprises, has a built-in memory usage limiter, which is disabled by default. While some operating systems provide ways to limit the memory usage of processes, relying on it -wasn't deemed to be flexible enough (e.g. using +wasn't deemed to be flexible enough (for example, using .BR ulimit (1) to limit virtual memory tends to cripple .BR mmap (2)). @@ -231,19 +232,21 @@ the command line option \fB\-\-memlimit=\fIlimit\fR. Often it is more convenient to enable the limiter by default by setting the environment variable .BR XZ_DEFAULTS , -e.g.\& +for example, .BR XZ_DEFAULTS=\-\-memlimit=150MiB . It is possible to set the limits separately -for compression and decompression -by using \fB\-\-memlimit\-compress=\fIlimit\fR and -\fB\-\-memlimit\-decompress=\fIlimit\fR. +for compression and decompression by using +.BI \-\-memlimit\-compress= limit +and \fB\-\-memlimit\-decompress=\fIlimit\fR. Using these two options outside .B XZ_DEFAULTS is rarely useful because a single run of .B xz cannot do both compression and decompression and .BI \-\-memlimit= limit -(or \fB\-M\fR \fIlimit\fR) +(or +.B \-M +.IR limit ) is shorter to type on the command line. .PP If the specified memory usage limit is exceeded when decompressing, @@ -252,11 +255,13 @@ will display an error and decompressing the file will fail. If the limit is exceeded when compressing, .B xz will try to scale the settings down so that the limit -is no longer exceeded (except when using \fB\-\-format=raw\fR -or \fB\-\-no\-adjust\fR). +is no longer exceeded (except when using +.B \-\-format=raw +or +.BR \-\-no\-adjust ). This way the operation won't fail unless the limit is very small. The scaling of the settings is done in steps that don't -match the compression level presets, e.g. if the limit is +match the compression level presets, for example, if the limit is only slightly less than the amount required for .BR "xz \-9" , the settings will be scaled down only a little, @@ -276,7 +281,7 @@ It is possible to insert padding between the concatenated parts or after the last part. The padding must consist of null bytes and the size of the padding must be a multiple of four bytes. -This can be useful e.g. if the +This can be useful, for example, if the .B .xz file is stored on a medium that measures file sizes in 512-byte blocks. @@ -373,7 +378,7 @@ For even more information, use twice, but note that this may be slow, because getting all the extra information requires many seeks. The width of verbose output exceeds -80 characters, so piping the output to e.g.\& +80 characters, so piping the output to, for example, .B "less\ \-S" may be convenient if the terminal isn't wide enough. .IP "" @@ -388,6 +393,20 @@ should be used. .TP .BR \-k ", " \-\-keep Don't delete the input files. +.IP "" +Since +.B xz +5.2.6, +this option also makes +.B xz +compress or decompress even if the input is +a symbolic link to a regular file, +has more than one hard link, +or has the setuid, setgid, or sticky bit set. +The setuid, setgid, and sticky bits are not copied +to the target file. +In earlier versions this was only done with +.BR \-\-force . .TP .BR \-f ", " \-\-force This option has several effects: @@ -405,7 +424,7 @@ to the target file. .IP \(bu 3 When used with .B \-\-decompress -.BR \-\-stdout +.B \-\-stdout and .B xz cannot recognize the type of the source file, @@ -670,7 +689,7 @@ Good to very good compression while keeping decompressor memory usage reasonable even for old systems. .B \-6 is the default, which is usually a good choice -e.g. for distributing files that need to be decompressible +for distributing files that need to be decompressible even on systems with only 16\ MiB RAM. .RB ( \-5e or @@ -840,7 +859,7 @@ The default .I size is three times the LZMA2 dictionary size or 1 MiB, whichever is more. -Typically a good value is 2\-4 times +Typically a good value is 2\(en4 times the size of the LZMA2 dictionary or at least 1 MiB. Using .I size @@ -1006,7 +1025,7 @@ until the details have been decided. .RE .IP "" For 32-bit -.BR xz +.B xz there is a special case: if the .I limit would be over @@ -1015,6 +1034,9 @@ the .I limit is set to .BR "4020\ MiB" . +On MIPS32 +.B "2000\ MiB" +is used instead. (The values .B 0 and @@ -1022,7 +1044,8 @@ and aren't affected by this. A similar feature doesn't exist for decompression.) This can be helpful when a 32-bit executable has access -to 4\ GiB address space while hopefully doing no harm in other situations. +to 4\ GiB address space (2 GiB on MIPS32) +while hopefully doing no harm in other situations. .IP "" See also the section .BR "Memory usage" . @@ -1042,7 +1065,8 @@ for possible ways to specify the .IR limit . .TP \fB\-M\fR \fIlimit\fR, \fB\-\-memlimit=\fIlimit\fR, \fB\-\-memory=\fIlimit -This is equivalent to specifying \fB\-\-memlimit\-compress=\fIlimit +This is equivalent to specifying +.BI \-\-memlimit\-compress= limit \fB\-\-memlimit\-decompress=\fIlimit\fR. .TP .B \-\-no\-adjust @@ -1088,7 +1112,12 @@ A custom filter chain allows specifying the compression settings in detail instead of relying on the settings associated to the presets. When a custom filter chain is specified, -preset options (\fB\-0\fR ... \fB\-9\fR and \fB\-\-extreme\fR) +preset options +.RB ( \-0 +\&...\& +.B \-9 +and +.BR \-\-extreme ) earlier on the command line are forgotten. If a preset option is specified after one or more custom filter chain options, @@ -1172,7 +1201,10 @@ The integer can be from .B 0 to .BR 9 , -matching the command line options \fB\-0\fR ... \fB\-9\fR. +matching the command line options +.B \-0 +\&...\& +.BR \-9 . The only supported modifier is currently .BR e , which matches @@ -1253,7 +1285,7 @@ The literal coding makes an assumption that the highest .I lc bits of the previous uncompressed byte correlate with the next byte. -E.g. in typical English text, an upper-case letter is +For example, in typical English text, an upper-case letter is often followed by a lower-case letter, and a lower-case letter is usually followed by another lower-case letter. In the US-ASCII character set, the highest three bits are 010 @@ -1268,7 +1300,7 @@ If you want maximum compression, test .BR lc=4 . Sometimes it helps a little, and sometimes it makes compression worse. -If it makes it worse, test e.g.\& +If it makes it worse, test .B lc=2 too. .TP @@ -1294,10 +1326,10 @@ The default means four-byte alignment .RI (2^ pb =2^2=4), which is often a good choice when there's no better guess. .IP "" -When the aligment is known, setting +When the alignment is known, setting .I pb accordingly may reduce the file size a little. -E.g. with text files having one-byte +For example, with text files having one-byte alignment (US-ASCII, ISO-8859-*, UTF-8), setting .B pb=0 can improve compression slightly. @@ -1325,7 +1357,7 @@ The default depends on the .IR preset : 0 uses .BR hc3 , -1\-3 +1\(en3 use .BR hc4 , and the rest use @@ -1441,11 +1473,11 @@ The default is .B fast for .I presets -0\-3 and +0\(en3 and .B normal for .I presets -4\-9. +4\(en9. .IP "" Usually .B fast @@ -1464,7 +1496,7 @@ bytes is found, the algorithm stops looking for possibly better matches. .IP "" .I Nice -can be 2\-273 bytes. +can be 2\(en273 bytes. Higher values tend to give better compression ratio at the expense of speed. The default depends on the @@ -1482,7 +1514,7 @@ and .IP "" Reasonable .I depth -for Hash Chains is 4\-100 and 16\-1000 for Binary Trees. +for Hash Chains is 4\(en100 and 16\(en1000 for Binary Trees. Using very high values for .I depth can make the encoder extremely slow with some files. @@ -1523,7 +1555,7 @@ A BCJ filter converts relative addresses in the machine code to their absolute counterparts. This doesn't change the size of the data, but it increases redundancy, -which can help LZMA2 to produce 0\-15\ % smaller +which can help LZMA2 to produce 0\(en15\ % smaller .B .xz file. The BCJ filters are always reversible, @@ -1551,7 +1583,7 @@ the compression ratio: .RS .IP \(bu 3 Some types of files containing executable code -(e.g. object files, static libraries, and Linux kernel modules) +(for example, object files, static libraries, and Linux kernel modules) have the addresses in the instructions filled with filler values. These BCJ filters will still do the address conversion, which will make the compression worse with these files. @@ -1625,12 +1657,12 @@ The Delta filter can be only used as a non-last filter in the filter chain. .IP "" Currently only simple byte-wise delta calculation is supported. -It can be useful when compressing e.g. uncompressed bitmap images +It can be useful when compressing, for example, uncompressed bitmap images or uncompressed PCM audio. However, special purpose algorithms may give significantly better results than Delta + LZMA2. This is true especially with audio, -which compresses faster and better e.g. with +which compresses faster and better, for example, with .BR flac (1). .IP "" Supported @@ -1642,7 +1674,7 @@ Specify the .I distance of the delta calculation in bytes. .I distance -must be 1\-256. +must be 1\(en256. The default is 1. .IP "" For example, with @@ -1701,7 +1733,7 @@ known and a couple of seconds have already passed since .B xz started processing the file. The time is shown in a less precise format which -never has any colons, e.g. 2 min 30 s. +never has any colons, for example, 2 min 30 s. .RE .IP "" When standard error is not a terminal, @@ -1714,7 +1746,7 @@ on a single line to standard error after compressing or decompressing the file. The speed and elapsed time are included only when the operation took at least a few seconds. -If the operation didn't finish, e.g. due to user interruption, +If the operation didn't finish, for example, due to user interruption, also the completion percentage is printed if the size of the input file is known. .TP @@ -1740,7 +1772,7 @@ See the section .B "ROBOT MODE" for details. .TP -.BR \-\-info\-memory +.B \-\-info\-memory Display, in human-readable format, how much physical memory (RAM) .B xz thinks the system has and the memory usage limits for compression @@ -1916,8 +1948,8 @@ Compressed size of the file .IP 5. 4 Uncompressed size of the file .IP 6. 4 -Compression ratio, for example -.BR 0.123. +Compression ratio, for example, +.BR 0.123 . If ratio is over 9.999, three dashes .RB ( \-\-\- ) are displayed instead of the ratio. @@ -2168,9 +2200,9 @@ This is for passing options to when it is not possible to set the options directly on the .B xz command line. -This is the case e.g. when +This is the case when .B xz -is run by a script or tool, e.g. GNU +is run by a script or tool, for example, GNU .BR tar (1): .RS .RS @@ -2184,11 +2216,12 @@ XZ_OPT=\-2v tar caf foo.tar.xz foo .RE .IP "" Scripts may use -.B XZ_OPT -e.g. to set script-specific default compression options. +.BR XZ_OPT , +for example, to set script-specific default compression options. It is still recommended to allow users to override .B XZ_OPT -if that is reasonable, e.g. in +if that is reasonable. +For example, in .BR sh (1) scripts one may use something like this: .RS @@ -2210,7 +2243,7 @@ is practically a superset of .BR lzma , .BR unlzma , and -.BR lzcat +.B lzcat as found from LZMA Utils 4.32.x. In most cases, it is possible to replace LZMA Utils with XZ Utils without breaking existing scripts. @@ -2284,7 +2317,7 @@ The alternative is to mark that uncompressed size is unknown and use end-of-payload marker to indicate where the decompressor should stop. LZMA Utils uses this method when uncompressed size isn't known, -which is the case for example in pipes. +which is the case, for example, in pipes. .PP .B xz supports decompressing @@ -2480,7 +2513,7 @@ Create with the preset .B \-4e .RB ( "\-4 \-\-extreme" ), -which is slower than e.g. the default +which is slower than the default .BR \-6 , but needs less memory for compression and decompression (48\ MiB and 5\ MiB, respectively): @@ -2590,7 +2623,7 @@ but if a limit has already been set, don't increase it: .PP .nf .ft CW -NEWLIM=$((123 << 20)) # 123 MiB +NEWLIM=$((123 << 20))\ \ # 123 MiB OLDLIM=$(xz \-\-robot \-\-info\-memory | cut \-f3) if [ $OLDLIM \-eq 0 \-o $OLDLIM \-gt $NEWLIM ]; then XZ_OPT="$XZ_OPT \-\-memlimit\-decompress=$NEWLIM" @@ -2634,10 +2667,10 @@ Preset;CompCPU .RE .PP If you know that a file requires -somewhat big dictionary (e.g. 32 MiB) to compress well, +somewhat big dictionary (for example, 32\ MiB) to compress well, but you want to compress it quicker than .B "xz \-8" -would do, a preset with a low CompCPU value (e.g. 1) +would do, a preset with a low CompCPU value (for example, 1) can be modified to use a bigger dictionary: .RS .PP @@ -2687,9 +2720,8 @@ the size of the uncompressed file is waste of memory, so the above command isn't useful for small files. .PP Sometimes the compression time doesn't matter, -but the decompressor memory usage has to be kept low -e.g. to make it possible to decompress the file on -an embedded system. +but the decompressor memory usage has to be kept low, for example, +to make it possible to decompress the file on an embedded system. The following command uses .B \-6e .RB ( "\-6 \-\-extreme" ) @@ -2720,7 +2752,7 @@ might help too, but usually and .I pb are more important. -E.g. a source code archive contains mostly US-ASCII text, +For example, a source code archive contains mostly US-ASCII text, so something like the following might give slightly (like 0.1\ %) smaller file than .B "xz \-6e" @@ -2737,7 +2769,7 @@ xz \-\-lzma2=preset=6e,pb=0,lc=4 source_code.tar .PP Using another filter together with LZMA2 can improve compression with certain file types. -E.g. to compress a x86-32 or x86-64 shared library +For example, to compress a x86-32 or x86-64 shared library using the x86 BCJ filter: .RS .PP @@ -2766,10 +2798,10 @@ which has a few more advanced filters than simple delta but uses Deflate for the actual compression. .PP The image has to be saved in uncompressed format, -e.g. as uncompressed TIFF. +for example, as uncompressed TIFF. The distance parameter of the Delta filter is set to match the number of bytes per pixel in the image. -E.g. 24-bit RGB bitmap needs +For example, 24-bit RGB bitmap needs .BR dist=3 , and it is also good to pass .B pb=0 @@ -2783,7 +2815,7 @@ xz \-\-delta=dist=3 \-\-lzma2=pb=0 foo.tiff .fi .RE .PP -If multiple images have been put into a single archive (e.g.\& +If multiple images have been put into a single archive (for example, .BR .tar ), the Delta filter will work on that too as long as all images have the same number of bytes per pixel. |