17115 Commits

Author SHA1 Message Date
jtl
71d3115e20 Improvements to the MDXFileChunk() template function:
- Remove unneeded fstat()/lseek() calls.
- Return NULL and set errno to EINVAL on negative length.
- Fix small style problems and expand variable names.

After this change, it is possible to use this code for some irregular
files. For example, 'md5 /dev/md0' should now succeed.

Differential Revision:	https://reviews.freebsd.org/D4748
Suggested by:	bde
Reviewed by:	bde, allanjude, delphij
2016-01-14 21:08:23 +00:00
andrew
da6ebcec56 Set -mlong-calls where needed to get a static clang and lldb 3.8.0
linking. These are too large for a branch instruction to branch from an
earlier point in the code to somewhere later.

This will also allow these to be build with Thumb-2 when we get this
infrastructure.

Reviewed by:	dim
Differential Revision:	https://reviews.freebsd.org/D4855
2016-01-14 19:00:13 +00:00
jhb
b499f710a7 Fix building with GCC since PAGE_MASK is signed on i386.
Reviewed by:	ngie
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D4772
2016-01-14 15:51:13 +00:00
ed
3e8ec99afc Remove an unneeded assignment of the return value.
tdelete() is supposed to return the address of the parent node that has
been deleted. We already keep track of this node in the loop between
lines 94-107. The GO_LEFT()/GO_RIGHT() macros are used later on as well,
so we must make sure not to change it to something else.
2016-01-14 07:27:42 +00:00
dteske
2f0e6a8f9a Default to en_US.ISO8859-1 if no locale
MFC after:	3 days
X-MFC-to:	stable/10
2016-01-14 01:59:20 +00:00
kevlo
3ffa934214 - Add the 'restrict' type qualifier to match function prototype.
- Remove sys/types.h.
2016-01-14 01:33:16 +00:00
vangyzen
e39bbe5207 libthr: const-ify two variables
Make the default umutex and urwlock initializers const,
because they can be, and as a microoptimization.

MFC after:	5 days
Sponsored by:	Dell Inc.
2016-01-13 22:34:55 +00:00
brooks
e7eafcf596 Avoid reading pass the end of the source buffer when it is not NUL
terminated.

If this buffer is adjacent to an unmapped page or a version of C with
bounds checked is used this may result in a crash.

PR:		206178
Submitted by:	Alexander Cherepanov <cherepan@mccme.ru>
MFC after:	1 week
2016-01-13 21:50:08 +00:00
brooks
50e4d892f3 Avoid reading pass the end of the source buffer when it is not NUL
terminated.

If this buffer is adjacent to an unmapped page or a version of C with
bounds checked is used this may result in a crash.

PR:		206177
Submitted by:	Alexander Cherepanov <cherepan@mccme.ru>
MFC after:	1 week
2016-01-13 21:49:01 +00:00
andrew
8298ebeec0 Remove the compat code to handle the kernel passing us an unalinged
stackpointer. Userland expects the kernel to pass it an aligned sp and
pass a pointer to the arguments in x0. The kernel side was updated in
r289502, 3 months ago.

Sponsored by:	ABT Systems Ltd
2016-01-13 15:54:17 +00:00
jilles
67473fa318 Update futimens/utimensat for MFC to stable/10:
* Fix __FreeBSD_version check.
* Update history section in man page.

An MFC of this commit to stable/10 will allow using the new system calls
instead of the fallback.

MFC after:	3 days
2016-01-12 20:53:57 +00:00
ngie
1777eef1bc Fix a mismerge from NetBSD in r162194 with xdr_rpcb_entry_list_ptr(..)
This fixes the potential NULL pointer dereference properly, and also fixes
memory leaks encountered in the process of iterating through `*rp`.

MFC after: 1 week
Found by: Valgrind
Reported by: Dan Roberts <dan.roberts@isilon.com>
Submitted by: Miles Ohlrich <miles.ohlrich@isilon.com>
Sponsored by: EMC / Isilon Storage Division
2016-01-12 00:20:57 +00:00
ngie
728878b7d7 Similar to r293704, fix theoretical leak of netconfig(3) resources in
__rpcbind_is_up(..) if getnetconfig(3) is partly successful in allocating
resources, but not completely successful by moving the endnetconfig(3) call
up before we return from the function if nconf == NULL.

MFC after: 1 week
Reported by: Coverity
Submitted by: Miles Ohlrich <miles.ohlrich@isilon.com>
Sponsored by: EMC / Isilon Storage Division
2016-01-11 22:01:33 +00:00
ngie
cf1ded0c14 Fix theoretical leak of netconfig(3) resources in svcunix_create(..)
In the event that the getconfig(3) call in svcunix_create is partly successful,
some of the netconfig(3) resources allocated might be leaked if the call returns
NULL as endnetconfig(3) wasn't called explicitly in that case. Ensure that the
resources are fully cleaned up by going to the `done` label, which will call
endnetconfig(3) for us.

MFC after: 1 week
Reported by: Coverity
Submitted by: Miles Ohlrich <miles.ohlrich@isilon.com>
Sponsored by: EMC / Isilon Storage Division
2016-01-11 21:56:53 +00:00
andrew
6b8d860df3 Use -mlong-calls to build crt1.o and gcrt1.o. This tells the compiler to
generate code to branch based on an address in a register. This allows us
to have binaries larger than the 32MiB limit of a branch instruction.

The main use of this is with clang. Clang 3.8.0 has been shown to be larger
than the above limit.
2016-01-10 23:41:31 +00:00
ae
724790746e Make tohex() work as expected. 2016-01-09 08:04:29 +00:00
ae
258e30c8e4 Fix a typo.
PR:		205722
2016-01-09 08:02:35 +00:00
glebius
aaa09777e1 New sendfile(2) syscall. A joint effort of NGINX and Netflix from 2013 and
up to now.

The new sendfile is the code that Netflix uses to send their multiple tens
of gigabits of data per second. The new implementation features asynchronous
I/O, when I/O operations are launched, but not awaited to be complete. An
explanation of why such behavior is beneficial compared to old one is
going to be too long for a commit message, so we will skip it here.

Additional features of new syscall are extra flags, which provide an
application more control over data sent. The SF_NOCACHE flag tells
kernel that data shouldn't be cached after it was sent. The SF_READAHEAD()
macro allows to specify readahead size in pages.

The new syscalls is a drop in replacement. No modifications are required
to applications. One can take nginx binary for stable/10 and run it
successfully on head. Although SF_NODISKIO lost its original sense, as now
sendfile doesn't block, and now means something completely different (tm),
using the new sendfile the old way is absolutely safe.

Celebrates:	Netflix global launch!
Sponsored by:	Nginx, Inc.
Sponsored by:	Netflix
Relnotes:	yes
2016-01-08 20:34:57 +00:00
dteske
5ee4880e05 Bump copyright (forgotten part of r293340)
MFC after:	3 days
X-MFC-to:	stable/10
X-MFC-with:	r293340
2016-01-07 22:13:17 +00:00
bdrewery
d6b33ca987 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	EMC / Isilon Storage Division
2016-01-07 19:58:23 +00:00
bdrewery
36d4fab170 Don't install /usr/include/stand.h twice after r293040.
Only install it from lib/libstand.

Sponsored by:	EMC / Isilon Storage Division
2016-01-07 19:19:23 +00:00
dteske
8d062e4dae Increase maximum buffer size for `-x cmd' value
MFC after:	3 days
X-MFC-to:	stable/10
2016-01-07 18:50:02 +00:00
bdrewery
0e634907c1 Add in DIRDEPS_BUILD support.
Sponsored by:	EMC / Isilon Storage Division
2016-01-07 00:19:30 +00:00
emaste
23dbf7923c Add fls() to libstand
Although we don't use it in tree yet libstand is installed as user-
facing /usr/liblibstand.a, and some work in progress makes use of it.
Instead of conflicting with ongoing libstand Makefile deduplication,
just add it now.
2016-01-06 17:33:32 +00:00
jilles
14b7155b88 Add sbin and /usr/local directories to _PATH_DEFPATH.
Set _PATH_DEFPATH to
/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin. This is the
path in the default class in the default /etc/login.conf,
excluding ~/bin which would not be expanded properly in a string
constant.

For normal logins, _PATH_DEFPATH is overridden by /etc/login.conf,
~/.login_conf or shell startup files. _PATH_DEFPATH is still used as a
default by execlp(), execvp(), posix_spawnp() and sh if PATH is not set, and
by cron. Especially the latter is a common trap (most recently in PR
204813).

PR:		204813
Reviewed by:	secteam (delphij), alfred
2016-01-05 16:21:20 +00:00
dfr
df4e91dca3 Fix a memory leak in gss_release_oid_set 2016-01-04 17:42:12 +00:00
ngie
ff9e83788d Remove free'ing of an uninitialized variable
Just remove it completely from the test as it's initialized but unused apart
from the free(3) call

Differential Revision: https://reviews.freebsd.org/D4769 (part of larger diff)
MFC after: 5 days
Reported by: cppcheck
Reviewed by: oshogbo
Sponsored by: EMC / Isilon Storage Division
2016-01-04 03:34:22 +00:00
ngie
e324a485ff Use nitems(x) macro instead of using hardcoded numbers for indices into
the nvlists

Convert some of the variables from int to unsigned int to squelch -Wsign-compare
warnings when converting hardcoded values to nitems(..)

Differential Revision: https://reviews.freebsd.org/D4769 (part of larger diff)
MFC after: 5 days
Reviewed by: oshogbo
Sponsored by: EMC / Isilon Storage Division
2016-01-04 03:26:36 +00:00
ngie
749ad0ff5a Convert another string variable to string_arr missed in r293130
Differential Revision: https://reviews.freebsd.org/D4769 (part of larger diff)
MFC after: 5 days
Reviewed by: oshogbo
Sponsored by: EMC / Isilon Storage Division
2016-01-04 03:12:18 +00:00
ngie
f07e33e8a7 Rename nitems and string variables to avoid collisions
Rename the `nitems` variable to `num_items` to avoid collisions with the
macro in sys/param.h for counting elements in an array

Similarly, rename `string` to `string_arr` to avoid future collisions with
potential keywords, as well as make it clear that `string_arr` isn't a char*
value, but instead a char** value.

Differential Revision: https://reviews.freebsd.org/D4769 (part of larger diff)
MFC after: 5 days
Reviewed by: oshogbo
Sponsored by: EMC / Isilon Storage Division
2016-01-04 03:02:44 +00:00
ngie
76a57e8c3b Add sys/types.h for for size_t, etc
stable/10 requires it due to header pollution

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2016-01-03 11:22:15 +00:00
emaste
78ef3d1599 Reduce libstand Makefile duplication
libstand is built in three places (lib/libstand, sys/boot/libstand32,
and sys/boot/userboot/libstand). Reduce Makefile duplication by
.including libstand/Makefile from sys/boot/libstand32/Makefile.

sys/boot/userboot/libstand/Makefile will be addressed later, as it
contains additional differences yet to be handled.

This change also switches libstand32 to use the new uuid_from_string
and uuid_to_string, which was not included in r292473.

Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D4747
2016-01-01 15:30:11 +00:00
ian
7781ba8bcc When building libstand for arm, don't emit instructions that require
relocation fixups unsupported by the self_reloc() code, and don't optimize
memcpy/memset using floating point registers, because in a standalone
environment nothing has initialized the fpu hardware.
2015-12-31 18:29:24 +00:00
jtl
df8d88f0e5 Fix a file descriptor leak in mdXhl.c (which is used by numerous hashing
algorithms.

CID:	1305669,1305611,1305663,1305603,1305584,1305639,1346865,1305601
Differential Revision:	https://reviews.freebsd.org/D4732
Reviewed by:	allanjude, delphij
MFC after:	2 weeks
Sponsored by:	Juniper Networks
2015-12-30 18:04:50 +00:00
imp
95fd2c39d8 Use __alignof__ instead of assuming int64_t to get the right
alignment.

Differential Revision: https://reviews.freebsd.org/D4708
2015-12-30 03:36:22 +00:00
jhb
d4f8a20c6b Document the recently added support for ptrace(2) LWP events. 2015-12-30 00:04:57 +00:00
imp
242eb0dd48 The FILE structure has a mbstate_t in it. This structure needs to be
aligned on a int64_t boundary. However, when we allocate the array of
these structures, we use ALIGNBYTES which defaults to sizeof(int) on
arm, i386 and others. The i386 stuff can handle unaligned accesses
seemlessly. However, arm cannot. Take this into account when creating
the array of FILEs, and add some comments about why.

Differential Revision: https://reviews.freebsd.org/D4708
2015-12-27 23:04:11 +00:00
allanjude
c7c2f2dfab Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.c
cperciva's libmd implementation is 5-30% faster

The same was done for SHA256 previously in r263218

cperciva's implementation was lacking SHA-384 which I implemented, validated against OpenSSL and the NIST documentation

Extend sbin/md5 to create sha384(1)

Chase dependancies on sys/crypto/sha2/sha2.{c,h} and replace them with sha512{c.c,.h}

Reviewed by:	cperciva, des, delphij
Approved by:	secteam, bapt (mentor)
MFC after:	2 weeks
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D3929
2015-12-27 17:33:59 +00:00
dchagin
dad1819732 Verify that tv_sec value specified in settimeofday() and clock_settime()
(CLOCK_REALTIME case) system calls is non negative.
This commit hides a kernel panic in atrtc_settime() as the clock_ts_to_ct()
does not properly convert negative tv_sec.

ps. in my opinion clock_ts_to_ct() should be rewritten to properly handle
negative tv_sec values.

Differential Revision:	https://reviews.freebsd.org/D4714
Reviewed by:		kib

MFC after:	1 week
2015-12-27 15:37:07 +00:00
ed
f846ec41eb Replace implementation of hsearch() by one that scales.
Traditionally the hcreate() function creates a hash table that uses
chaining, using a fixed user-provided size. The problem with this
approach is that this often either wastes memory (table too big) or
yields bad performance (table too small). For applications it may not
always be easy to estimate the right hash table size. A fixed number
only increases performance compared to a linked list by a constant
factor.

This problem can be solved easily by dynamically resizing the hash
table. If the size of the hash table is at least doubled, this has no
negative on the running time complexity. If a dynamically sized hash
table is used, we can also switch to using open addressing instead of
chaining, which has the advantage of just using a single allocation for
the entire table, instead of allocating many small objects.

Finally, a problem with the existing implementation is that its
deterministic algorithm for hashing makes it possible to come up with
fixed patterns to trigger an excessive number of collisions. We can
easily solve this by using FNV-1a as a hashing algorithm in combination
with a randomly generated offset basis.

Measurements have shown that this implementation is about 20-25% faster
than the existing implementation (even if the existing implementation is
given an excessive number of buckets). Though it allocates more memory
through malloc() than the old implementation (between 4-8 pointers per
used entry instead of 3), process memory use is similar to the old
implementation as if the estimated size was underestimated by a factor
10. This is due to the fact that malloc() needs to perform less
bookkeeping.

Reviewed by:	jilles, pfg
Obtained from:	https://github.com/NuxiNL/cloudlibc
Differential Revision:	https://reviews.freebsd.org/D4644
2015-12-27 07:50:11 +00:00
kib
27f78ada40 Typo. 2015-12-27 01:14:42 +00:00
dim
6f44a590da Upgrade our copies of clang and llvm to 3.7.1 release. This is a
bugfix-only release, with no new features.

Please note that from 3.5.0 onwards, clang and llvm require C++11
support to build; see UPDATING for more information.
2015-12-25 21:39:45 +00:00
cperciva
ea20143730 Make libxnet.so a symlink to libc.so. This makes -lxnet a no-op, as
POSIX requires for the c99 compiler.

(In fact, our c99(1) already ignores -lxnet; but our make(1) doesn't set
${CC} correctly, and our cc(1) treats xnet like any other library.)

Reviewed by:	kib
2015-12-25 11:29:18 +00:00
ume
6e0a844f59 Remove _gethostbynisname() and _gethostbynisaddr(). These functions
used to be called from getipnodebyname().

MFC after:	1 week
2015-12-25 10:49:40 +00:00
kib
7659430654 Do not compile ARMv6 instructions on ARMv4/v5. Although clang is fine
with mrrc, gcc is not.  The disabled code is not executed on ARMv4
anyway.

Reported and reviewed by:	ian
Sponsored by:	The FreeBSD Foundation
2015-12-24 22:13:52 +00:00
ngie
d94d9ab649 Increase the timeout for resolv_test from the default (300 seconds) to
450 seconds

This is required on slower network connections, and on older releases
(stable/10 seems to be slower as far as name resolution goes.. not sure
why yet).

Remove an outdated comment in the Makefile from when I was working on
this code over a year ago on github

MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
2015-12-23 18:48:32 +00:00
ngie
8061f8577d Use j instead of a hardcoded index (9) and increment it after
running the NaNs testcases

MFC after: 3 days
Pointyhat to: ngie
Sponsored by: EMC / Isilon Storage Division
2015-12-23 09:11:18 +00:00
emaste
3808bf980c Support a.out format in nlist only on i386
i386 is the only current FreeBSD architecture that ever used a.out
format.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D4687
2015-12-22 20:36:14 +00:00
jhb
b665ac0758 Move the mkioctls script to libsysdecode and use it to generate a
sysdecode_ioctlname() function.  This function matches the behavior
of the truss variant in that it returns a pointer to a string description
for known ioctls.  The caller is responsible for displaying unknown
ioctl requests.  For kdump this meant moving the logic to handle unknown
ioctl requests out of the generated function and into an ioctlname()
function in kdump.c instead.

Differential Revision:	https://reviews.freebsd.org/D4610
2015-12-22 20:33:49 +00:00
ed
4fec3a8161 Let tsearch()/tdelete() use an AVL tree.
The existing implementations of POSIX tsearch() and tdelete() don't
attempt to perform any balancing at all. Testing reveals that inserting
100k nodes into a tree sequentially takes approximately one minute on my
system.

Though most other BSDs also don't use any balanced tree internally, C
libraries like glibc and musl do provide better implementations. glibc
uses a red-black tree and musl uses an AVL tree.

Red-black trees have the advantage over AVL trees that they only require
O(1) rotations after insertion and deletion, but have the disadvantage
that the tree has a maximum depth of 2*log2(n) instead of 1.44*log2(n).
My take is that it's better to focus on having a lower maximum depth,
for the reason that in the case of tsearch() the invocation of the
comparator likely dominates the running time.

This change replaces the tsearch() and tdelete() functions by versions
that create an AVL tree. Compared to musl's implementation, this version
is different in two different ways:

- We don't keep track of heights; just balances. This is sufficient.
  This has the advantage that it reduces the number of nodes that are
  being accessed. Storing heights requires us to also access all of the
  siblings along the path.

- Don't use any recursion at all. We know that the tree cannot 2^64
  elements in size, so the height of the tree can never be larger than
  96. Use a 128-bit bitmask to keep track of the path that is computed.
  This allows us to iterate over the same path twice, meaning we can
  apply rotations from top to bottom.

Inserting 100k nodes into a tree now only takes 0.015 seconds. Insertion
seems to be twice as fast as glibc, whereas deletion has about the same
performance. Unlike glibc, it uses a fixed amount of memory.

I also experimented with both recursive and iterative bottom-up
implementations of the same algorithm. This iterative top-down version
performs similar to the recursive bottom-up version in terms of speed
and code size.

For some reason, the iterative bottom-up algorithm was actually 30%
faster for deletion, but has a quadratic memory complexity to keep track
of all the parent pointers.

Reviewed by:	jilles
Obtained from:	https://github.com/NuxiNL/cloudlibc
Differential Revision:	https://reviews.freebsd.org/D4412
2015-12-22 18:12:11 +00:00