Commit Graph

20511 Commits

Author SHA1 Message Date
Kyle Evans
1dc8556358 libbe(3): Fix destroy of imported BE w/ AUTOORIGIN
Imported BE, much like the activated BE, will not have an origin that we can
fetch/examine for destruction. be_destroy should not return BE_ERR_NOORIGIN
for failure to get the origin property for BE_DESTROY_AUTOORIGIN, because
we don't really know going into it that there's even an origin to be
destroyed.

BE_DESTROY_NEEDORIGIN has been renamed to BE_DESTROY_WANTORIGIN because only
a subset of it *needs* the origin, so 'need' is too strong of verbiage.

This was caught by jenkins and the bectl tests, but kevans failed to run the
bectl tests prior to commit.

Reported by:	lwhsu
2019-10-16 18:33:31 +00:00
Kyle Evans
455d8009b4 libbe(3): add needed bits for be_destroy to auto-destroy some origins
New BEs can be created from either an existing snapshot or an existing BE.
If an existing BE is chosen (either implicitly via 'bectl create' or
explicitly via 'bectl create -e foo bar', for instance), then bectl will
create a snapshot of the current BE or "foo" with be_snapshot, with a name
formatted like: strftime("%F-%T") and a serial added to it.

This commit adds the needed bits for libbe or consumers to determine if a
snapshot names matches one of these auto-created snapshots (with some light
validation of the date/time/serial), and also a be_destroy flag to specify
that the origin should be automatically destroyed if possible.

A future commit to bectl will specify BE_DESTROY_AUTOORIGIN by default so we
clean up the origin in the most common case, non-user-managed snapshots.
2019-10-16 14:43:05 +00:00
Justin Hibbits
6b672ba642 [PowerPC] force applications linked with lib CSU to have .got
Summary:
This forces applications linked with lib CSU to have a .got, fixing binaries
linked with LLD9 after secure-plt was enabled on FreeBSD.

Submitted by:	Alfredo Dal'Ava Junior (alfredo.junior_eldorado.org.br)
Differential Revision:	https://reviews.freebsd.org/D21476
2019-10-12 20:53:40 +00:00
Ruslan Bukin
8694561d5d Install the libopencsd version header (other headers now depend on it).
Sponsored by:	DARPA, AFRL
2019-10-11 09:18:45 +00:00
Ruslan Bukin
fc5020859f Update ARM CoreSight trace decoder library.
Its latest version merged from:
^/vendor/opencsd/a1961c91b02a92f3c6ed8b145c636ac4c5565aca

Sponsored by:	DARPA, AFRL
2019-10-10 13:30:13 +00:00
Ruslan Bukin
85f87cf491 Update Intel Processor Trace decoder library.
Its latest version merged from:
^/vendor/processor-trace/892e12c5a27bda5806d1e63269986bb4171b5a8b

Sponsored by:	DARPA, AFRL
2019-10-10 12:46:34 +00:00
Dimitry Andric
9122aeeaa6 Merge ^/head r353316 through r353350. 2019-10-09 16:40:22 +00:00
Glen Barber
013f1782dd Connect the libucl(3) manual page to the build.
MFC after:	3 days
Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-10-09 15:35:22 +00:00
Mitchell Horne
68b6aaa70d RISC-V: Fix an alignment warning in libthr
Compiling with clang gives a loss-of-alignment error due the cast to
uint8_t *. Since the TLS is always tcb aligned and TP_OFFSET is defined
as sizeof(struct tcb) we can guarantee there is no misalignment. Silence
the error by moving the offset into the inline assembly.

Reviewed by:	br
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21926
2019-10-09 02:02:22 +00:00
Brooks Davis
e593620997 msun: Silence new harmless -Wimplicit-int-float-conversion warnings
Clang from trunk recently added a warning for when implicit int-to-float
conversions cause a loss of precision. The code in question is designed
to be able to handle that, so add explicit casts to silence this.

Submitted by:	James Clarke <jrtc27@jrtc27.com>
Reviewed by:	dim
Obtained from:	CheriBSD
MFC after:	1 week
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D21913
2019-10-08 21:39:51 +00:00
Brooks Davis
f189761366 Fix various -Wpointer-compare warnings
This warning (comparing a pointer against a zero character literal
rather than NULL) has existed since GCC 7.1.0, and was recently added to
Clang trunk.

Almost all of these are harmless, except for fwcontrol's str2node, which
needs to both guard against dereferencing a NULL pointer (though in
practice it appears none of the callers will ever pass one in), as well
as ensure it doesn't parse the empty string as node 0 due to strtol's
awkward interface.

Submitted by:	James Clarke <jtrc27@jrtc27.com>
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D21914
2019-10-08 21:14:09 +00:00
Dimitry Andric
063e3a6dcc Prepare for merging back to head:
* Set tentative merge date
* Add UPDATING entry
* Bump __FreeBSD_version
* Bump FREEBSD_CC_VERSION
* Bump LLD_REVISION
2019-10-08 18:21:33 +00:00
Dimitry Andric
8b3bc70a2b Merge ^/head r352764 through r353315. 2019-10-08 18:17:02 +00:00
Edward Tomasz Napierala
1a13f2e6b4 Introduce stats(3), a flexible statistics gathering API.
This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof).  Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs.  The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by:	sef (earlier version)
Obtained from:	Netflix
Relnotes:	yes
Sponsored by:	Klara Inc, Netflix
Differential Revision:	https://reviews.freebsd.org/D20477
2019-10-07 19:05:05 +00:00
Kyle Evans
036d2e814b Re-add ALLOW_MIPS_SHARED_TEXTREL, sprinkle it around
Diff partially stolen from CheriBSD; these bits need -Wl,-z,notext in order
to build in an LLVM world. They are needed for all flavors/sizes of MIPS.
This will eventually get fixed in LLVM, but it's unclear when.

Reported by:	arichardson, emaste
Differential Revision:	https://reviews.freebsd.org/D21696
2019-10-06 04:19:49 +00:00
Kyle Evans
8495fa081b libusb: LIBUSB_DEBUG environment variable override of libusb_set_debug
The debug level generally just controls verbosity of libusb for debugging
libusb devices/usage. We allow the environment to set the debug level
independent of the application, but the application will always override
this if it explicitly sets the debug level.

Changing the environment is easy, but patching the software to change the
debug level isn't necessarily easy or possible. Further, there's this
write-only debug_fixed variable that would seem to imply that the debug
level should be fixed, but it isn't currently used. Change the logic to use
strtol() so we can detect real 0 vs. conversion failure, then honor
debug_fixed in libusb_set_debug.

Reviewed by:	hselasky
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D21877
2019-10-02 15:19:39 +00:00
Pawel Biernacki
c1dd36cfca libsysdecode: decode PROT_MAX flags
Extend libsysdecode to pretty-print PROT_MAX flags and fix decoding of regular protection flags broken since r349240.

before:
truss:
mmap(0x0,40960,0x30000,MAP_PRIVATE|MAP_ANON|MAP_NOCORE,-1,0x0) = 34366234624 (0x800632000)
kdump:
11439 protmax  CALL  mmap(0,0xa000,0x30000<><invalid>196608,0x21002<MAP_PRIVATE|MAP_ANON|MAP_NOCORE>,0xffffffff,0)

after:
truss:
mmap(0x0,40960,PROT_MAX(PROT_READ|PROT_WRITE)|PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON|MAP_NOCORE,-1,0x0) = 34366234624 (0x800632000)
kdump:
11439 protmax  CALL  mmap(0,0xa000,0x30000<PROT_MAX(PROT_READ|PROT_WRITE)|PROT_READ|PROT_WRITE>,0x21002<MAP_PRIVATE|MAP_ANON|MAP_NOCORE>,0xffffffff,0)

Reviewed by:	kib (mentor)
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D21706
2019-09-30 19:03:43 +00:00
Kyle Evans
142c5c8c36 memfd_create(3): Don't actually force hugetlb size with MFD_HUGETLB
The size flags are only required to select a size on systems that support
multiple sizes. MFD_HUGETLB by itself is valid.
2019-09-29 17:30:10 +00:00
Warner Losh
ab311b7f12 Revert the mode_t -> int changes and add a warning in the BUGS section instead.
While FreeBSD's implementation of these expect an int inside of libc, that's an
implementation detail that we can hide from the user as it's the natural
promotion of the current mode_t type and before it is used in the kernel, it's
converted back to the narrower type that's the current definition of mode_t. As
such, documenting int is at best confusing and at worst misleading. Instead add
a note that these args are variadic and as such calling conventions may differ
from non-variadic arguments.
2019-09-28 17:15:48 +00:00
Warner Losh
4470d73996 Document varadic args as int, since you can't have short varadic args (they are
promoted to ints).

- `mode_t` is `uint16_t` (`sys/sys/_types.h`)
- `openat` takes variadic args
- variadic args cannot be 16-bit, and indeed the code uses int
- the manpage currently kinda implies the argument is 16-bit by saying `mode_t`

Prompted by Rust things: https://github.com/tailhook/openat/issues/21
Submitted by: Greg V at unrelenting
Differential Revision: https://reviews.freebsd.org/D21816
2019-09-27 16:11:47 +00:00
Dimitry Andric
668ee10168 Merge ^/head r352587 through r352763. 2019-09-26 18:25:54 +00:00
Dimitry Andric
6b3555c38e Enable OpenMP for powerpc64
Summary: When powerpc64 switches to LLVM, use this patch to enable
OpenMP as well. OpenMP on PPC is only for 64-bits, so don't make a
32-bit libomp. A change to openmp files is necesssary (under review on
https://reviews.llvm.org/D67190), because it determines ELF format
version based on endianness, which is incorrect.

Reviewed by:	alfredo.junior_eldorado.org.br, #manpages
Differential Revision: https://reviews.freebsd.org/D21532
2019-09-26 18:24:04 +00:00
Kyle Evans
e12ff89136 Further normalize copyright notices
- s/C/c/ where I've been inconsistent about it
- +SPDX tags
- Remove "All rights reserved" where possible

Requested by:	rgrimes (all rights reserved)
2019-09-26 16:19:22 +00:00
David Bright
d4f4430503 Correct mistake in MLINKS introduced in r352747
Messed up a merge conflict resolution and didn't catch that before
commit.

Sponsored by:	Dell EMC Isilon
2019-09-26 16:13:17 +00:00
David Bright
9afb12bab4 Add an shm_rename syscall
Add an atomic shm rename operation, similar in spirit to a file
rename. Atomically unlink an shm from a source path and link it to a
destination path. If an existing shm is linked at the destination
path, unlink it as part of the same atomic operation. The caller needs
the same permissions as shm_unlink to the shm being renamed, and the
same permissions for the shm at the destination which is being
unlinked, if it exists. If those fail, EACCES is returned, as with the
other shm_* syscalls.

truss support is included; audit support will come later.

This commit includes only the implementation; the sysent-generated
bits will come in a follow-on commit.

Submitted by:	Matthew Bryan <matthew.bryan@isilon.com>
Reviewed by:	jilles (earlier revision)
Reviewed by:	brueffer (manpages, earlier revision)
Relnotes:	yes
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D21423
2019-09-26 15:32:28 +00:00
Kyle Evans
a631497fca Add SPDX tags to recently added files
Reported by:	Pawel Biernacki
2019-09-25 22:53:30 +00:00
Kyle Evans
c34a5f16fa posix_spawn(3): handle potential signal issues with vfork
Described in [1], signal handlers running in a vfork child have
opportunities to corrupt the parent's state. Address this by adding a new
rfork(2) flag, RFSPAWN, that has vfork(2) semantics but also resets signal
handlers in the child during creation.

x86 uses rfork_thread(3) instead of a direct rfork(2) because rfork with
RFMEM/RFSPAWN cannot work when the return address is stored on the stack --
further information about this problem is described under RFMEM in the
rfork(2) man page.

Addressing this has been identified as a prerequisite to using posix_spawn
in subprocess on FreeBSD [2].

[1] https://ewontfix.com/7/
[2] https://bugs.python.org/issue35823

Reviewed by:	jilles, kib
Differential Revision:	https://reviews.freebsd.org/D19058
2019-09-25 19:22:03 +00:00
Kyle Evans
079c5b9ed8 rfork(2): add RFSPAWN flag
When RFSPAWN is passed, rfork exhibits vfork(2) semantics but also resets
signal handlers in the child during creation to avoid a point of corruption
of parent state from the child.

This flag will be used by posix_spawn(3) to handle potential signal issues.

Reviewed by:	jilles, kib
Differential Revision:	https://reviews.freebsd.org/D19058
2019-09-25 19:20:41 +00:00
Dimitry Andric
5763a8cf06 Do not left-shift a negative number (inducing undefined behavior in
C/C++) in exp(3), expf(3), expm1(3) and expm1f(3) during intermediate
computations that compute the IEEE-754 bit pattern for |2**k| for
integer |k|.

The implementations of exp(3), expf(3), expm1(3) and expm1f(3) need to
compute IEEE-754 bit patterns for 2**k in certain places.  (k is an
integer and 2**k is exactly representable in IEEE-754.)

Currently they do things like 0x3FF0'0000+(k<<20), which is to say they
take the bit pattern representing 1 and then add directly to the
exponent field to get the desired power of two.  This is fine when k is
non-negative.

But when k<0 (and certain classes of input trigger this), this
left-shifts a negative number -- an operation with undefined behavior in
C and C++.

The desired semantics can be achieved by instead adding the
possibly-negative k to the IEEE-754 exponent bias to get the desired
exponent field, _then_ shifting that into its proper overall position.

(Note that in case of s_expm1.c and s_expm1f.c, there are SET_HIGH_WORD
and SET_FLOAT_WORD uses further down in each of these files that perform
shift operations involving k, but by these points k's range has been
restricted to 2 < k <= 56, and the shift operations under those
circumstances can't do anything that would be UB.)

Submitted by:	Jeff Walden, https://github.com/jswalden
Obtained from:	https://github.com/freebsd/freebsd/pull/411
Obtained from:	https://github.com/freebsd/freebsd/pull/412
MFC after:	3 days
2019-09-25 18:50:57 +00:00
Kyle Evans
a9ac5e1424 sysent: regenerate after r352705
This also implements it, fixes kdump, and removes no longer needed bits from
lib/libc/sys/shm_open.c for the interim.
2019-09-25 18:09:19 +00:00
Kyle Evans
3e25d1fb61 Add linux-compatible memfd_create
memfd_create is effectively a SHM_ANON shm_open(2) mapping with optional
CLOEXEC and file sealing support. This is used by some mesa parts, some
linux libs, and qemu can also take advantage of it and uses the sealing to
prevent resizing the region.

This reimplements shm_open in terms of shm_open2(2) at the same time.

shm_open(2) will be moved to COMPAT12 shortly.

Reviewed by:	markj, kib
Differential Revision:	https://reviews.freebsd.org/D21393
2019-09-25 18:03:18 +00:00
Kyle Evans
f17221ee7a Update fcntl(2) after r352695 2019-09-25 17:33:12 +00:00
Ed Maste
ef36db58da remove obsolete i386 MD memchr implementation
bde reports (in a reply to r351700 commit mail):
    This uses scasb, which was last optimal on the 8086, or perhaps the
    original i386.  On freefall, it is several times slower than the
    naive translation of the naive C code.

Reported by:	bde
Reviewed by:	kib, markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21785
2019-09-25 16:49:22 +00:00
Yuri Pankov
3c78771400 lib/libc/regex: fix build with REDEBUG defined
Reviewed by:	kevans
Differential Revision:	https://reviews.freebsd.org/D21760
2019-09-24 12:21:01 +00:00
Konstantin Belousov
f9bf9282c9 Fix destruction of the robust mutexes.
If robust mutex' owner terminated, causing kernel-assisted state
recovery, and then pthread_mutex_destroy() is executed as the next
action, assert is triggered about mutex still being on the list.
Ignore the mutex linkage in pthread_mutex_destroy() for shared robust
mutexes with dead owner, same as for enqueue_mutex().

Reported by:	avg
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-09-23 13:24:31 +00:00
Sean Eric Fagan
ba7a55d934 Add two options to allow mount to avoid covering up existing mount points.
The two options are

* nocover/cover:  Prevent/allow mounting over an existing root mountpoint.
E.g., "mount -t ufs -o nocover /dev/sd1a /usr/local" will fail if /usr/local
is already a mountpoint.
* emptydir/noemptydir:  Prevent/allow mounting on a non-empty directory.
E.g., "mount -t ufs -o emptydir /dev/sd1a /usr" will fail.

Neither of these options is intended to be a default, for historical and
compatibility reasons.

Reviewed by:	allanjude, kib
Differential Revision:	https://reviews.freebsd.org/D21458
2019-09-23 04:28:07 +00:00
Mike Karels
789f4e26e6 Add support for ps -H on corefiles in libkvm
Add support for kernel threads in kvm_getprocs() and the underlying
kvm_proclist() in libkvm when fetching from a kernel core file. This
has been missing/needed for several releases, when kernel threads became
normal threads.  The loop over the processes now contains a sub-loop for
threads, which iterates beyond the first thread only when threads are
requested.  Also set some fields such as tid that were previously
uninitialized.

Reviewed by:	vangyzen jhb(earlier revision)
MFC after:	4 days
Sponsored by:	Forcepoint LLC
Differential Revision:	https://reviews.freebsd.org/D21461
2019-09-22 13:56:27 +00:00
Dimitry Andric
0f80acb965 Merge ^/head r352436 through r352536. 2019-09-19 19:26:12 +00:00
Dimitry Andric
7fc776a827 Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb, and openmp
release 9.0.0 r372316, and update version numbers.
2019-09-19 19:25:01 +00:00
Li-Wen Hsu
05a42b7621 Whitespace cleanup, no functional change
Sponsored by:	The FreeBSD Foundation
2019-09-19 13:25:19 +00:00
Li-Wen Hsu
e751215341 Temporarily add test_write_filter_zstd BROKEN_TESTS as it always fails in CI
There is no trivial way to mark single libarchive test skip currently so just
add it to BROKEN_TESTS for now.

PR:		240683
Sponsored by:	The FreeBSD Foundation
2019-09-19 13:23:25 +00:00
Ruslan Bukin
230754ccca Add support for BERI statcounters.
BERI stands for Bluespec Extensible RISC Implementation, based on MIPS.

BERI has not implemented standard MIPS perfomance monitoring counters,
instead it provides statistical counters.

BERI statcounters have a several limitations:
- They can't be written
- They don't support start/stop operation
- None of hardware interrupt is provided on a counter overflow.

So make it separate to hwpmc_mips module and support process/system
counting mode only.

Sponsored by:	DARPA, AFRL
2019-09-18 16:13:50 +00:00
Konstantin Belousov
382e01c8dc sysctl: use names instead of magic numbers.
Replace magic numbers with symbols for internal sysctl operations.
Convert in-kernel and libc consumers.

Submitted by:	Pawel Biernacki
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21693
2019-09-18 16:13:10 +00:00
Baptiste Daroussin
d567f909ef Add native support for zstd to libarchive
Note that old pkg will failed to build after this. A recent ports tree (one
providing pkg 1.12+) is required to build. Older already built pkg, should
continue working as expected

PR:		238797
Exp run by:	antoine
Reviewed by:	cem
Approved by:	cem
Differential Revision:	https://reviews.freebsd.org/D20752
2019-09-18 07:57:56 +00:00
Konstantin Belousov
55894117b1 Return EISDIR when directory is opened with O_CREAT without O_DIRECTORY.
Reviewed by:	bcr (man page), emaste (previous version)
PR:	240452
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
DIfferential revision:	https://reviews.freebsd.org/D21634
2019-09-17 18:32:18 +00:00
Andriy Gapon
8569a95e76 fixup up fallout from r352447 in libbe
I totally forgot that we now have another in-tree consumer of libzfs.

MFC after:	3 days
X-MFC with:	r352447
2019-09-17 14:15:48 +00:00
Dimitry Andric
419f843fff Merge ^/head r352319 through r352435. 2019-09-17 06:08:15 +00:00
Dimitry Andric
2c3f47a727 Another round of attempting to squelch -Wdeprecated-declarations, which
has become very trigger-happy with libc++ 9.0.0.

It does not help that gcc's implementation of this warning is even more
trigger-happy, in the sense that it already warns on the declaration
itself, not when you are using it.  This is very annoying with our use
of -Wsystem-headers.  That should really be disabled for gcc.
2019-09-17 06:07:08 +00:00
Konstantin Belousov
505a58ec45 Increase the size of the send and receive buffers for YP client rpc
calls to max allowed UDP datagram size.

Since max allowed size both for keys and values where increased, the
old sizes of around 1K cause ypmatch(3) failures, while plain maps
fetches work.

The buffers were reduced in r34146 from default UDP rpcclient values
to 1024/2304 due to the key and value size being 1K.

Reviewed by:	slavash
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D21586
2019-09-16 06:42:01 +00:00
Dimitry Andric
23e2b4d9b1 Instead of disabling gcc's deprecated declaration warnings about e.g.
std::auto_ptr in a whole bunch of individual Makefiles, make the warning
globally non-fatal instead.  This is similar to what was done to many
more non-fatal warnings from newer gcc versions.
2019-09-14 19:16:28 +00:00