aboutsummaryrefslogtreecommitdiff
path: root/usr.bin/sort
Commit message (Collapse)AuthorAgeFilesLines
* sort: Fix message catalogue usageMark Johnston2022-01-282-10/+12
| | | | | | | | | | | | | - Check that catopen() succeeded before calling catclose(). musl will crash in the latter if the catalogue descriptor is -1. - Keep the message catalogue open for most of sort(1)'s actual operation. - Don't use catgets(3) to print error messages if catopen(3) had failed. Reviewed by: arichardson, emaste MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D34081
* sort: Fix random sortMark Johnston2021-10-291-1/+1
| | | | | | | | | | bwsrawdata() is supposed to return the string buffer. PR: 259451 Reported by: sigsys@gmail.com Fixes: d053fb22f6d3 ("usr.bin/sort: Avoid UBSan errors") MFC after: 3 days Sponsored by: The FreeBSD Foundation
* usr.bin/sort: Avoid UBSan errorsAlex Richardson2021-07-063-145/+155
| | | | | | | | | | | | | UBSan complains about out-of-bounds accesses for zero-length arrays. To avoid this we can use flexible array members. However, the C standard does not allow for structures that only contain flexible array members, so we move the length parameters into that structure too. Split out from D28233. Reviewed By: markj MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D31009
* sort: Change default algorithm to mergesortCyril Zhang2021-06-171-2/+2
| | | | | | | | | | | This results in a significant improvement in the runtime of sort(1) when radix sort cannot be used. This comes at the expense of increased memory usage, but this is small relative to sort's overall memory usage. PR: 255551 Reviewed by: markj Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D30319
* sort: Hook NetBSD tests up to the buildMark Johnston2021-05-131-2/+2
| | | | | MFC after: 1 week Sponsored by: The FreeBSD Foundation
* sort: Cache value of MB_CUR_MAXCyril Zhang2021-05-135-34/+45
| | | | | | | | | | | | | | Every usage of MB_CUR_MAX results in a call to __mb_cur_max. This is inefficient and redundant. Caching the value of MB_CUR_MAX in a global variable removes these calls and speeds up the runtime of sort. For numeric sorting, runtime is almost halved in some tests. PR: 255551 PR: 255840 Reviewed by: markj MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D30170
* sort: Stop "fixing" obsolete key syntax after -- flagCyril Zhang2021-05-131-0/+5
| | | | | | | | PR: 255798 Reviewed by: markj MFC after: 1 week Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D30234
* Fix -Wpointer-sign warnings in bwstring.cAlex Richardson2020-09-101-7/+7
| | | | Notes: svn path=/head/; revision=365582
* sort(1): Remove duplicate option checkGordon Bergling2020-09-081-1/+1
| | | | | | | | | | | Reviewed by: lwhsu, emaste Approved by: emaste Obtained from: DragonFlyBSD MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D23892 Notes: svn path=/head/; revision=365458
* sort(1): Fix two wchar-related bugs in radixsortConrad Meyer2020-06-231-4/+33
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sort(1)'s radixsort implementation was broken for multibyte LC_CTYPEs in at least two ways: * In actual radix sort, it would only bucket the least significant byte from each wchar, ignoring the 24 most-significant bits of each unicode character. * In degenerate cases / "fast paths," it would fall back to another sorting algorithm (default: mergesort) with a bogus comparator offset. The string comparison functions in sort(1) take an offset in units of the operating character size. However, radixsort was passing an offset in units of bytes. The byte offset must be divided by sizeof(wchar_t). This revision addresses both discovered issues. Some example testcases: $ (echo 耳 ; echo 脳 ; echo 耳) | \ LC_CTYPE=ja_JP.UTF-8 LC_COLLATE=C LANG=C sort --radixsort --debug $ (echo 耳 ; echo 脳 ; echo 耳) | \ LC_CTYPE=C LC_COLLATE=C LANG=C sort --radixsort --debug $ (for i in $(jot 34); do echo 耳耳耳耳耳; echo 耳耳耳耳脳; echo 耳耳耳耳脴; done) | \ LC_CTYPE=ja_JP.UTF-8 LC_COLLATE=C LANG=C sort --radixsort --debug PR: 247494 Reported by: knu MFC after: I do not intend to, but parties interested in stable might want to Notes: svn path=/head/; revision=362545
* Update Makefile.depend filesSimon J. Gerraty2019-12-111-2/+0
| | | | | | | | | | | | | Update a bunch of Makefile.depend files as a result of adding Makefile.depend.options files Reviewed by: bdrewery MFC after: 1 week Sponsored by: Juniper Networks Differential Revision: https://reviews.freebsd.org/D22494 Notes: svn path=/head/; revision=355617
* Add Makefile.depend.optionsSimon J. Gerraty2019-12-111-0/+8
| | | | | | | | | | | | | | | | | | | | Leaf directories that have dependencies impacted by options need a Makefile.depend.options file to avoid churn in Makefile.depend DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc can be set in local.dirdeps-options.mk which can add to those set in Makefile.depend.options See share/mk/dirdeps-options.mk Reviewed by: bdrewery MFC after: 1 week Sponsored by: Juniper Networks Differential Revision: https://reviews.freebsd.org/D22469 Notes: svn path=/head/; revision=355616
* Adjust history, info source from v1's manualsSevan Janiyan2019-09-041-2/+2
| | | | | | | | | https://www.bell-labs.com/usr/dmr/www/1stEdman.html MFC after: 5 days Notes: svn path=/head/; revision=351811
* sort(1): Memoize MD5 computation to reduce repeated computationConrad Meyer2019-04-133-0/+36
| | | | | | | | | | | | | | | | Experimentally, reduces sort -R time of a 148160 line corpus from about 3.15s to about 0.93s on this particular system. There's probably room for improvement using some digest other than md5, but I don't want to look at sort(1) anymore. Some discussion of other possible improvements in the Test Plan section of the Differential. PR: 230792 Reviewed by: jhb (earlier version) Differential Revision: https://reviews.freebsd.org/D19885 Notes: svn path=/head/; revision=346175
* sort(1): Simplify and bound random seedingConrad Meyer2019-04-111-49/+66
| | | | | | | | | | | | | | | | | | | | | Bound input file processing length to avoid the issue reported in [1]. For simplicity, only allow regular file and character device inputs. For character devices, only allow /dev/random (and /dev/urandom symblink). 32 bytes of random is perfectly sufficient to seed MD5; we don't need any more. Users that want to use large files as seeds are encouraged to truncate those files down to an appropriate input file via tools like sha256(1). (This does not change the sort algorithm of sort -R.) [1]: https://lists.freebsd.org/pipermail/freebsd-hackers/2018-August/053152.html PR: 230792 Reported by: Ali Abdallah <aliovx AT gmail.com> Relnotes: yes Notes: svn path=/head/; revision=346116
* sort(1): Whitespace and style cleanupConrad Meyer2019-04-111-88/+87
| | | | | | | | | No functional change. Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=346110
* sort(1): randomcoll: Skip the memory allocation entirelyConrad Meyer2019-04-041-15/+6
| | | | | | | | | | | | | | | | | | | | | There's no reason to order based on strcmp of ASCII digests instead of memcmp of the raw digests. While here, remove collision fallback. If you collide two MD5s, they're probably the same string anyway. If robustness against MD5 collisions is desired, maybe we shouldn't use MD5. None of the behavior of sort -R is specified by POSIX, so we're free to implement this however we like. E.g., using a 128-bit counter and block cipher to generate unique indices for each line of input. PR: 230792 (2/many) Relnotes: This will change the sort order for a given dataset with a given seed. Other similarly breaking changes are planned. Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=345896
* sort(1): randomcoll: Don't sort on ENOMEMConrad Meyer2019-04-041-19/+9
| | | | | | | | PR: 230792 (1/many) Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=345891
* Don't use absolute path to sed when building usr.bin/joinAlex Richardson2018-08-231-1/+1
| | | | | | | | | This is required to build sort on Linux hosts since sed is in /bin there. Approved By: jhb (mentor) Notes: svn path=/head/; revision=338265
* sort(1): Fix -m when only implicit stdin is used for inputKyle Evans2018-06-203-5/+9
| | | | | | | | | | | | | | | | | | | | | Observe: printf "a\nb\nc\n" > /tmp/foo # Next command results in no output cat /tmp/foo | sort -m # Next command results in proper output cat /tmp/foo | sort -m - # Also works: sort -m /tmp/foo Some const'ification was done to simplify the actual solution of adding "-" explicitly to the file list if we didn't have any file arguments left over. PR: 190099 MFC after: 1 week Notes: svn path=/head/; revision=335404
* sort(1): Add bits to allow easy checking against NetBSD testsKyle Evans2018-06-202-0/+17
| | | | | | | I'm looking at sort(1) failures, for better or worse. Notes: svn path=/head/; revision=335403
* Fix the WITH_SORT_THREADS build.Mark Johnston2018-02-071-12/+12
| | | | | | | | PR: 201664 MFC after: 1 week Notes: svn path=/head/; revision=328995
* various: general adoption of SPDX licensing ID tags.Pedro F. Giffuni2017-11-2714-0/+28
| | | | | | | | | | | | | | | | | Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts. No functional change intended. Notes: svn path=/head/; revision=326276
* DIRDEPS_BUILD: Update dependencies.Bryan Drewery2017-10-311-1/+0
| | | | | | | Sponsored by: Dell EMC Isilon Notes: svn path=/head/; revision=325188
* sort(1): Remove unneeded initializations.Pedro F. Giffuni2017-02-173-6/+1
| | | | | | | Found by: Clang static analyzer Notes: svn path=/head/; revision=313891
* sort - Don't live-loop threads.Pedro F. Giffuni2017-01-231-24/+28
| | | | | | | | | | | Worker threads now use a pthread_cond_t to wait for work instead of burning the cpu up. Obtained from: DragonflyBSD (07774aea0ccf64a48fcfad8899e3bf7c8f18277a) MFC after: 2 weeks Notes: svn path=/head/; revision=312667
* - Use correct offsets into the keys set array. As the elements of thisMarius Strobl2016-12-283-11/+32
| | | | | | | | | | | | | | zero-length array are dynamically sized at run-time based on the use of hints, compilers can't be expected to figure out these offsets on their own. [1] - Fix incorrect comparison in cmp_nans(). [2] PR: 204571 [1], 202301 [2] Submitted by: David Binderman [2] MFC after: 3 days Notes: svn path=/head/; revision=310712
* pages and psize are always assigned, so there is no need to initializeXin LI2016-11-281-2/+0
| | | | | | | | | them as zero. MFC after: 2 weeks Notes: svn path=/head/; revision=309234
* Eliminate variables that are computed, assigned but neverXin LI2016-11-282-11/+1
| | | | | | | | | used. MFC after: 2 weeks Notes: svn path=/head/; revision=309233
* Fix an obvious typo.Xin LI2016-11-281-1/+1
| | | | | | | MFC after: 2 weeks Notes: svn path=/head/; revision=309232
* - Fix typoGabor Kovesdan2016-09-081-1/+1
| | | | | | | | | PR: 211245 Submitted by: Christoph Schonweiler <public2016@hauptsignal.at> MFC after: 5 days Notes: svn path=/head/; revision=305613
* Cleanup unnecessary semicolons from utilities we all love.Pedro F. Giffuni2016-04-153-3/+3
| | | | Notes: svn path=/head/; revision=298089
* Fix some mdoc(7) issuesBaptiste Daroussin2015-10-241-17/+16
| | | | | | | Obtained from: DragonflyBSD Notes: svn path=/head/; revision=289876
* -C and -c allow at most one input file. Ensure this is the case when theGabor Kovesdan2015-10-221-5/+5
| | | | | | | | | | | input files are specified through --files0-from. Submitted by: tim@OpenBSD Obtained from: OpenBSD MFC after: 1 week Notes: svn path=/head/; revision=289751
* new dependsSimon J. Gerraty2015-06-161-1/+0
| | | | Notes: svn path=/head/; revision=284481
* Add META_MODE support.Simon J. Gerraty2015-06-131-0/+21
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | Off by default, build behaves normally. WITH_META_MODE we get auto objdir creation, the ability to start build from anywhere in the tree. Still need to add real targets under targets/ to build packages. Differential Revision: D2796 Reviewed by: brooks imp Notes: svn path=/head/; revision=284345
| * dirdeps.mk now sets DEP_RELDIRSimon J. Gerraty2015-06-081-2/+0
| | | | | | | | Notes: svn path=/projects/bmake/; revision=284172
| * Merge sync of headSimon J. Gerraty2015-05-279-117/+63
| |\ | |/ |/| | | Notes: svn path=/projects/bmake/; revision=283595
| * Merge from head@274682Simon J. Gerraty2014-11-193-4/+6
| |\ | | | | | | | | | Notes: svn path=/projects/bmake/; revision=274683
| * \ Merge head from 7/28Simon J. Gerraty2014-08-191-0/+1
| |\ \ | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=270164
| * | | Updated dependenciesSimon J. Gerraty2014-05-161-1/+0
| | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=266219
| * | | Updated dependenciesSimon J. Gerraty2014-05-101-0/+2
| | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=265802
| * | | Merge from headSimon J. Gerraty2014-05-081-1/+1
| |\| | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=265720
| * | | Merge headSimon J. Gerraty2014-04-2810-22/+22
| |\ \ \ | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=265044
| * \ \ \ Merge from headSimon J. Gerraty2013-09-0515-17/+17
| |\ \ \ \ | | | | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=255263
| * | | | | Updated dependenciesSimon J. Gerraty2013-03-111-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=248169
| * | | | | Updated dependenciesSimon J. Gerraty2013-02-161-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=246868
| * | | | | Sync with HEAD.David E. O'Brien2013-02-082-46/+28
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=246555
| * | | | | | Updated/new Makefile.dependSimon J. Gerraty2012-11-081-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=242788
| * | | | | | Sync from headSimon J. Gerraty2012-11-0411-115/+109
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Notes: svn path=/projects/bmake/; revision=242545
generated by cgit v1.2.3 (git 2.25.1) at 2024-10-20 03:26:25 +0000