Commit Graph

34 Commits

Author SHA1 Message Date
Conrad Meyer
f20b149b45 sort(1): Memoize MD5 computation to reduce repeated computation
Experimentally, reduces sort -R time of a 148160 line corpus from about
3.15s to about 0.93s on this particular system.

There's probably room for improvement using some digest other than md5, but
I don't want to look at sort(1) anymore.  Some discussion of other possible
improvements in the Test Plan section of the Differential.

PR:		230792
Reviewed by:	jhb (earlier version)
Differential Revision:	https://reviews.freebsd.org/D19885
2019-04-13 04:42:17 +00:00
Conrad Meyer
7a590a370a sort(1): Simplify and bound random seeding
Bound input file processing length to avoid the issue reported in [1].  For
simplicity, only allow regular file and character device inputs.  For
character devices, only allow /dev/random (and /dev/urandom symblink).

32 bytes of random is perfectly sufficient to seed MD5; we don't need any
more.  Users that want to use large files as seeds are encouraged to truncate
those files down to an appropriate input file via tools like sha256(1).

(This does not change the sort algorithm of sort -R.)

[1]: https://lists.freebsd.org/pipermail/freebsd-hackers/2018-August/053152.html

PR:		230792
Reported by:	Ali Abdallah <aliovx AT gmail.com>
Relnotes:	yes
2019-04-11 05:08:49 +00:00
Conrad Meyer
74504eefa1 sort(1): Whitespace and style cleanup
No functional change.

Sponsored by:	Dell EMC Isilon
2019-04-11 00:39:06 +00:00
Kyle Evans
7137597e15 sort(1): Fix -m when only implicit stdin is used for input
Observe:

printf "a\nb\nc\n" > /tmp/foo
# Next command results in no output
cat /tmp/foo | sort -m
# Next command results in proper output
cat /tmp/foo | sort -m -
# Also works:
sort -m /tmp/foo

Some const'ification was done to simplify the actual solution of adding "-"
explicitly to the file list if we didn't have any file arguments left over.

PR:		190099
MFC after:	1 week
2018-06-20 03:31:19 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Xin LI
3611de44ef pages and psize are always assigned, so there is no need to initialize
them as zero.

MFC after:	2 weeks
2016-11-28 06:38:41 +00:00
Xin LI
c514c3ed4f Eliminate variables that are computed, assigned but never
used.

MFC after:	2 weeks
2016-11-28 06:36:10 +00:00
Xin LI
665d2db378 Fix an obvious typo.
MFC after:	2 weeks
2016-11-28 06:32:05 +00:00
Gabor Kovesdan
a7bc18929d -C and -c allow at most one input file. Ensure this is the case when the
input files are specified through --files0-from.

Submitted by:	tim@OpenBSD
Obtained from:	OpenBSD
MFC after:	1 week
2015-10-22 10:57:15 +00:00
Pedro F. Giffuni
0f4b9a9057 Remove custom getdelim(3) and fix a small memory leak.
Originally from Andre Smagin.

Obtained from:	OpenBSD
MFC after:	1 week
2015-04-07 01:17:49 +00:00
Pedro F. Giffuni
45e151e97d sort: style knits / cleanups.
Minor cleanups that got accidentally reverted.

Obtained from:	OpenBSD
2015-04-06 03:02:20 +00:00
Pedro F. Giffuni
e5f71a07e4 Revert (partial) r281123, r281125:
sort: style knits / cleanups.

Our style guide(9) specifies that in absence of local variables
an empty line must be inserted.

Pointed out by:	eadler
2015-04-06 02:35:55 +00:00
Pedro F. Giffuni
db8026c7bb sort: style knits / cleanups.
Obtained from:	OpenBSD
2015-04-05 23:06:42 +00:00
Baptiste Daroussin
3e16491d77 Make sure to not skip any argument when converting from deprecated
+POS1, -POS2 to -kPOS1,POS2, so that sort +0n gets translated to sort -k1,1n
as it is expected

PR:		193994
Submitted by:	rodrigo
MFC after:	3 days
2014-10-02 06:29:49 +00:00
Gabor Kovesdan
c859c6dd54 - Update Oleg Moskalenko's email address
Requested by:	Oleg Moskalenko <mom040267@gmail.com>
2013-06-02 09:43:48 +00:00
Gabor Kovesdan
0c2c94e1b7 - Remove forgotten commented out debug code
Submitted by:	Christoph Mallon <christoph.mallon@gmx.de>
Reviewed by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
2013-01-27 19:50:24 +00:00
Gabor Kovesdan
f187ff08ef - Simplify and unify diagnostic and error message handling
Submitted by:	Christoph Mallon <christoph.mallon@gmx.de>
Reviewed by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
2013-01-27 19:49:10 +00:00
Gabor Kovesdan
554442439d - Change the memory heuristics to an actually working one
Submitted by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
Prodded by:	kib
2012-12-20 22:30:40 +00:00
Gabor Kovesdan
ab28d4d3cf - Use unsigned int for values obtained with sysctlbyname(). This fixes
sparc64 performance problems.

Submitted by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
Tested by:	trasz
2012-12-17 09:36:43 +00:00
Gabor Kovesdan
e8da8c744b - Portability changes for ARM
- Allow larger sort memory on 64-bit platforms

Submitted by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
2012-11-01 11:38:34 +00:00
Ed Schouten
bf70becee6 More -Wmissing-variable-declarations fixes.
In addition to adding `static' where possible:

- bin/date: Move `retval' into extern.h to make it visible to date.c.
- bin/ed: Move globally used variables into ed.h.
- sbin/camcontrol: Move `verbose' into camcontrol.h and fix shadow warnings.
- usr.bin/calendar: Remove unneeded variables.
- usr.bin/chat: Make `line' local instead of global.
- usr.bin/elfdump: Comment out unneeded function.
- usr.bin/rlogin: Use _Noreturn instead of __dead2.
- usr.bin/tset: Pull `Ospeed' into extern.h.
- usr.sbin/mfiutil: Put global variables in mfiutil.h.
- usr.sbin/pkg: Remove unused `os_corres'.
- usr.sbin/quotaon, usr.sbin/repquota: Remove unused `qfname'.
2012-10-19 14:49:42 +00:00
Gabor Kovesdan
5d5151ae7e - Change --nthreads parameter to --parallel for GNU compatibility
- Change default sort method to mergesort, which has a better worst case
  performance than qsort

Submitted by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
2012-07-04 16:25:11 +00:00
Gabor Kovesdan
5ca724dc59 - Only use multi-threading for large files
- Do not use mmap() by default; it can be enabled by --mmap
- Add some minor optimizations for -u
- Update manual page according to the changes

Submitted by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
2012-05-25 09:30:16 +00:00
Gabor Kovesdan
f50d9b2ffb - Fix -o option that was broken by my clang compile fix
Submitted by:	Oleg Moskalenko <oleg.moskalenko@citrix.com>
2012-05-17 13:08:30 +00:00
Gabor Kovesdan
ce1e997f54 - Eliminate initializations if global variables. Compilers are not
required to optimize these so it may result in larger binary size.

Pointed out by:	kib
2012-05-14 10:06:49 +00:00
Gabor Kovesdan
8818aa392a - Fix build with clang 2012-05-14 09:53:54 +00:00
Gabor Kovesdan
c66bbc9143 Add a BSD-licensed sort rewrite that was started by me and later completed
with the major functionality and optimizations by Oleg Moskalenko.
It is compatible with the latest version of POSIX and the current GNU sort
version that we have in base.  Beside this, it implements all the
functionality introduced in later versions of GNU sort.  For now, it will
be installed as "bsdsort", keeping GNU sort as the default sort
implementation.
2012-05-11 12:37:16 +00:00
Andrey A. Chernov
50cd73c1a7 Remove old sort files to _actually_ build it from contrib sources
Forgotten by: des
2002-04-06 20:22:06 +00:00
Andrey A. Chernov
b412ae253b Remove __P 2002-04-04 19:18:58 +00:00
Andrey A. Chernov
cac2ce40ff Remove \n from __COPYRIGHT to shut assembler warning 2002-03-31 20:11:10 +00:00
Andrey A. Chernov
903163a139 Remove double SINGL_FLD and SEP_FLAG initialization 2002-03-27 22:41:01 +00:00
Andrey A. Chernov
3c1a6de57e Remove double REC_D assignment
Cast REC_D optarg assignment to (u_char) to prevent out of bounds indexing
2002-03-25 14:22:31 +00:00
Mark Murray
e58067e0f2 Make this compile on FreeBSD. Sort out VCS tags, fix warnings. 2002-03-24 15:59:17 +00:00
Mark Murray
7a0a395027 Vendor import NETBSD's sort(1). This will be a replacement for
our GNU sort, as discussed 6 months or more ago.
2002-03-22 13:54:24 +00:00