Commit Graph

20279 Commits

Author SHA1 Message Date
Konstantin Belousov
03d89f7a04 pkru(3) man page.
Reviewed by:	alc, markj, jilles
With more input from:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D19211
2019-02-20 10:03:03 +00:00
Konstantin Belousov
a2d95495ee Add usermode helpers for for Intel userspace protection keys feature.
Reviewed by:	markj
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D18893
2019-02-20 09:56:23 +00:00
Enji Cooper
4783c2dda5 Fix test compilation for gmock_main tests, similar to r344345 2019-02-20 09:33:55 +00:00
Enji Cooper
4dedfa3214 Fix test compilation for gmock tests, similar to r344345 2019-02-20 08:04:08 +00:00
Enji Cooper
c5da30a59d Add glue Makefile.inc to make the next hop to the googletest/Makefile.inc 2019-02-20 08:01:00 +00:00
Enji Cooper
92ede4717e Oh wait... those helper programs were for the py tests
We don't need to install those. Leave that testing for the package's tests
2019-02-20 07:52:55 +00:00
Enji Cooper
140b363941 Apply lessons learned in r344345 on gtest_main tests
The key difference is that some of the programs were previously being compiled
and installed as tests, which is incorrect. Treat them like helpers instead.
2019-02-20 07:49:09 +00:00
Enji Cooper
dae8f2468d Fix gtest test compilation and build more tests
My previous work to integrate these tests was incomplete/incorrect, because I
misunderstood how the cmake macros worked.

This addresses items with the gtest tests, which in turn fixes test compilation
and adds more tests which I had previously missed.

Due to an unknown issue with gtest_stress_test, I had to add pthread to LIBADD,
even though I shouldn't have added it to that (it was failing to link -lpthread
to libprivategtest.a). Add a XXX comment to note that something's awry there
and deserves additional investigation.
2019-02-20 07:29:50 +00:00
Enji Cooper
f53e266234 Add more port-related flags to Makefile.inc 2019-02-20 06:54:32 +00:00
Enji Cooper
fedec5d9ee Sprinkle ${HAS_TESTS} in Makefiles
While here, reindent `SUBDIR.${MK_TESTS}`
2019-02-20 06:03:21 +00:00
Enji Cooper
04773d6d93 Add glue Makefile for tests under lib/googletest/... 2019-02-20 03:07:11 +00:00
Enji Cooper
988a9f7e39 Use GTESTS instead of PLAIN_TESTS_CXX
The former is from googletest.test.mk, whereas the latter is from plain.test.mk.

As noted in r344328, Kyua will adopt more native GoogleTest support. Thus, it's
more desirable to make the test interface more of an opaque blackbox for the
testcase implementer.
2019-02-20 01:11:30 +00:00
Enji Cooper
df13161740 Fix up dependency finding for libg{mock,test}_main
- Look up the corresponding non-*_main libraries via LDFLAGS using the
  directories provided in src.libnames.mk. This will allow the libraries to
  be built in the "make libraries" phase of buildworld.
- gtest_main relies on gtest, but didn't explicitly call out the dependency
  in `LIBADD`. Fill in this missing blank.
2019-02-20 00:19:11 +00:00
Enji Cooper
1ab321591f Unconditionally build/install libg{mock,test}_main
They are supporting libraries and as such, will need to be built/installed
when MK_TESTS == no during buildworld, i.e., the "make libraries" phase.

Otherwise, dependent components cannot rely on the libraries, like
`cddl/usr.sbin/zfsd/tests`.
2019-02-20 00:15:25 +00:00
Enji Cooper
f5817a3803 Add missing INCSGROUPS definition
This will install the gmock headers, like the gtest headers
2019-02-19 22:46:50 +00:00
Enji Cooper
43e25f47a7 Add MK_GOOGLETEST option for building/installing gmock/gtest
These libraries don't compile on non-C++-11 capable compilers, e.g., g++ 4.2.1
and its corresponding implementation of the c++ library, i.e., libstdc++.

Blacklist compilation on all non-C++-11 capable compilers and give others the
option of opting out of building/installing gmock/gtest via MK_GOOGLETEST.

This option is controlled by MK_CXX and MK_TESTS, as ATF compilation is.
2019-02-19 22:40:32 +00:00
Enji Cooper
c90e23db39 Make server_cat(..) handle short receives
In short, the prior code was far too simplistic when it came to calling recv(2)
and failed intermittently (or in the case of Jenkins, deterministically).

Handle short recv(2)s by checking the return code and incrementing the window
into the buffer by the number of received bytes. If the number of received
bytes <= 0, then bail out of the loop, and test the total number of received
bytes vs the expected number of bytes sent for equality, and base whether or
not the test passes/fails on that fact.

Remove the expected failure, now that the hdtr testcases deterministically pass
on my host after this change [1].

PR:		234809 [1], 235200
Reviewed by:	asomers
Approved by:	emaste (mentor)
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D19188
2019-02-19 22:19:31 +00:00
Sean Eric Fagan
237f4abc57 Remove some redundant code in _posix1e_acl_strip_np
This was discovered through examination -- acl_copy_entry() copies the
tag type and permset fields.

Reviewed by:	trasz, pfg
Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D19240
2019-02-19 19:15:15 +00:00
Enji Cooper
30e009fc3a MFhead@r344270 2019-02-19 03:46:32 +00:00
Gleb Smirnoff
aa43298309 With r343051 UMA switched from atomic counts to counter(9) and now kernel
reports snap counts of how much a zone alloced and how much it freed.  It
may happen that snap values doesn't match, e.g alloced - freed < 0.
Workaround that in memstat library.

Reported by:	pho
2019-02-18 21:27:13 +00:00
Dimitry Andric
40af08b3a3 Add one additional file to libllvmminimal, since in some cases (e.g.
upgrading from stable/10 to stable/11) symbols from it are needed to
link llvm-tblgen and clang-tblgen.

Reported by:	kib
MFC after:	3 days
2019-02-18 19:07:15 +00:00
Gleb Smirnoff
5bfb2e008d Imaginary cat jumped my keyboard! 2019-02-15 23:46:34 +00:00
Gleb Smirnoff
66fb0b1ad7 For 32-bit machines rollback the default number of vnode pager pbufs
back to the lever before r343030.  For 64-bit machines reduce it slightly,
too.  Together with r343030 I bumped the limit up to the value we use at
Netflix to serve 100 Gbit/s of sendfile traffic, and it probably isn't a
good default.

Provide a loader tunable to change vnode pager pbufs count. Document it.
2019-02-15 23:36:22 +00:00
Ed Maste
bcf99d2d99 Add WITH_PIE knob to build Position Independent Executables
Building binaries as PIE allows the executable itself to be loaded at a
random address when ASLR is enabled (not just its shared libraries).

With this change PIE objects have a .pieo extension and INTERNALLIB
libraries libXXX_pie.a.

MK_PIE is disabled for some kerberos5 tools, Clang, and Subversion, as
they explicitly reference .a libraries in their Makefiles.  These can
be addressed on an individual basis later.  MK_PIE is also disabled for
rtld-elf because it is already position-independent using bespoke
Makefile rules.

Currently only dynamically linked binaries will be built as PIE.

Discussed with:	dim
Reviewed by:	kib
MFC after:	1 month
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18423
2019-02-15 22:22:38 +00:00
Dimitry Andric
c981cbbd13 Merge ^/head r343956 through r344177. 2019-02-15 21:50:45 +00:00
Dimitry Andric
640dd76f2c Merge llvm, clang, compiler-rt, libc++, lld, and lldb release_80 branch
r354130, resolve conflicts, and bump version numbers.
2019-02-15 21:44:42 +00:00
Dimitry Andric
8e85c80dd7 Vendor import of compiler-rt release_80 branch r354130:
https://llvm.org/svn/llvm-project/compiler-rt/branches/release_80@354130
2019-02-15 20:49:11 +00:00
Dimitry Andric
5df36aae64 Vendor import of clang release_80 branch r354130:
https://llvm.org/svn/llvm-project/cfe/branches/release_80@354130
2019-02-15 20:49:05 +00:00
Dimitry Andric
519e1985ae Vendor import of llvm release_80 branch r354130:
https://llvm.org/svn/llvm-project/llvm/branches/release_80@354130
2019-02-15 20:48:51 +00:00
Konstantin Belousov
071bca67ee Unify i386 and amd64 getcontextx.c, and use ifuncs while there.
In particular, use ifuncs for __getcontextx_size(), also calculate the
size of the extended save area in resolver.  Same for __fillcontextx2().

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-14 14:02:33 +00:00
Konstantin Belousov
b5d72efb7a x86 __vdso_gettc(): use machine/cpufunc.h function for CPUID.
Based on the discussion with:	jkim
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-14 13:59:00 +00:00
Leandro Lupori
9031358d67 silence cast-align warnings from clang on powerpc64
silence the following warning when compiling libthr with clang 8
for powerpc64 architecture:

usr/src/lib/libthr/arch/powerpc/include/pthread_md.h:82:10: error:
cast from 'uint8_t *' (aka 'unsigned char *') to 'struct tcb *'
increases required alignment from 1 to 8 [-Werror,-Wcast-align]
82:  return ((struct tcb *)(_tp - TP_OFFSET));

Submitted by:	alfredo.junior_eldorado.org.br
Reviewed by:	git_bdragon.rtk0.net, emaste, kib, jhibbits, luporl
Differential Revision:	https://reviews.freebsd.org/D18807
2019-02-13 18:28:53 +00:00
Enji Cooper
fd89d5ad05 Compile the tests without -Werror for now
I'm working on resolving the issue upstream on github
2019-02-13 05:41:04 +00:00
Enji Cooper
f3c5273d31 Merge build glue for libraries and tests done on github
I need to doublecheck my work vs the port, but I believe that this covers the
initial integration of all upstream tests.

Ref: https://github.com/ngie-eign/freebsd/tree/googletest-integration
2019-02-13 04:58:15 +00:00
David E. O'Brien
60cc4a3e2d Note that readpassphrase() came into FreeBSD's libc at 4.6. 2019-02-13 04:52:01 +00:00
Kyle Evans
be7dd42376 libbe(3): Fix be_destroy behavior w.r.t. deep BE snapshots and -o
be_destroy is documented to recursively destroy a boot environment.  In the
case of snapshots, one would take this to mean that these are also
recursively destroyed.  However, this was previously not the case.
be_destroy would descend into the be_destroy callback and attempt to
zfs_iter_children on the top-level snapshot, which is bogus.

Our alternative approach is to take note of the snapshot name and iterate
through all of fs children of the BE to try destruction in the children.

The -o option is also fixed to work properly with deep BEs.  If the BE was
created with `bectl create -e otherDeepBE newDeepBE`, for instance, then a
recursive snapshot of otherDeepBE would have been taken for construction of
newDeepBE but a subsequent destroy with BE_DESTROY_ORIGIN set would only
clean up the snapshot at the root of otherDeepBE: ${BEROOT}/otherDeepBE@...

The most recent iteration instead pretends not to know how these things
work, verifies that the origin is another BE and then passes that back
through be_destroy to DTRT when snapshots and deep BEs may be in play.

MFC after:	1 week
2019-02-13 04:19:08 +00:00
Martin Matuska
a39fc08da2 MFV r344063:
Sync libarchive with vendor.

Relevant vendor changes:
  PR #1085: Fix a null pointer dereference bug in zip writer
  PR #1110: ZIP reader added support for XZ, LZMA, PPMD8 and BZIP2
            decopmpression
  PR #1116: Add support for 64-bit ar format
  PR #1120: Fix a 7zip crash [1] and a ISO9660 infinite loop [2]
  PR #1125: RAR5 reader - fix an invalid read and a memory leak
  PR #1131: POSIX reader - do not fail when tree_current_lstat() fails
            due to ENOENT [3]
  PR #1134: Delete unnecessary null pointer checks before calls of free()
  OSS-Fuzz 10843: Force intermediate to uint64_t to make UBSAN happy.
  OSS-Fuzz 11011: Avoid buffer overflow in rar5 reader

PR:		233006 [3]
Security:	CVE-2019-1000019 [1], CVE-2019-1000020 [2]
MFC after:	2 weeks
2019-02-12 23:24:45 +00:00
Kyle Evans
446ae812b0 libbe(3): Belatedly note the BE_DESTROY_ORIGIN option added in r343977
X-MFC-With: r343977
2019-02-12 02:16:21 +00:00
Mariusz Zaborski
b5d787d93b libnv: fix memory leaks
nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().

nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain
of nvpairs (as resetting it prevents nvlist_destroy() from freeing it).
Note: freeing the chain in nvlist_destroy() is not sufficient, because
it would still leak through nvlist_take_nvlist_array().  This affects
all nvlist_*_nvlist_array() use

Submitted by:	Mindaugas Rasiukevicius <rmind@netbsd.org>
Reported by:	clang/gcc ASAN
MFC after:	2 weeks
2019-02-10 23:28:55 +00:00
Kyle Evans
13c62c50e3 libbe(3): Add a destroy option for removing the origin
Currently origin snapshots are left behind when a BE is destroyed, whether
it was an auto-created snapshot or explicitly specified via, for example,
`bectl create -e be@mysnap ...`.

Removing it automatically could be argued as a POLA violation in some
circumstances, so provide a flag to be_destroy for it. An accompanying
option will be added to bectl(8) to utilize this.

Some minor style/consistency nits in the affected areas also addressed.

Reported by:	Shawn Webb
MFC after:	1 week
2019-02-10 21:19:09 +00:00
Sergey Kandaurov
9b9a527843 Sync "struct addrinfo" declaration with netdb.h.
Notably, unlike in OpenBSD, which the man page was copied from,
ai_canonname and ai_addr come in different order.

PR:		225880
MFC after:	1 week
2019-02-10 19:07:47 +00:00
Dimitry Andric
9e43c218d5 Merge ^/head r343807 through r343955. 2019-02-10 12:49:34 +00:00
Peter Jeremy
885bfcda11 Replace calls to sin(x) and cos(x) with a single call to sincos().
Replace calls to sinf(x) and cosf(x) with a single call to sincosf().

Submitted by:	Steve Kargl <sgk@troutmask.apl.washington.edu>
Reviewed by:	bde
Approved by:	grog
MFC after:	3 days
2019-02-10 08:46:07 +00:00
Sergey Kandaurov
78c8b9477c Document the ENOBUFS errno in setsockopt(2).
In particular, it is the case if SO_SNDBUF/SO_RCVBUF would exceed sb_max_adj.

PR:		200649
MFC after:	1 week
2019-02-09 21:33:32 +00:00
Dimitry Andric
bc8fc00705 Amend r343442, by only expecting the lib.msun.cbrt_test.cbrtl_powl and
trig_test.reduction test cases to fail, if the fixes from r343916 have
not yet been applied to the base compiler.

Reported by:    lwhsu
PR:		234040
Upstream PR:	https://bugs.llvm.org/show_bug.cgi?id=40206
MFC after:	1 week
2019-02-08 18:31:54 +00:00
Dimitry Andric
94ba333f9c Pull in r352607 from upstream llvm trunk (by Craig Topper):
[X86] Add FPSW as a Def on some FP instructions that were missing it.

Pull in r353141 from upstream llvm trunk (by Craig Topper):

  [X86] Connect the default fpsr and dirflag clobbers in inline
  assembly to the registers we have defined for them.

  Summary:
  We don't currently map these constraints to physical register numbers
  so they don't make it to the MachineIR representation of inline
  assembly.

  This could have problems for proper dependency tracking in the
  machine schedulers though I don't have a test case that shows that.

  Reviewers: rnk

  Reviewed By: rnk

  Subscribers: eraman, llvm-commits

  Tags: #llvm

  Differential Revision: https://reviews.llvm.org/D57641

Pull in r353489 from upstream llvm trunk (by Craig Topper):

  [X86] Add FPCW as a register and start using it as an implicit use on
  floating point instructions.

  Summary:
  FPCW contains the rounding mode control which we manipulate to
  implement fp to integer conversion by changing the roudning mode,
  storing the value to the stack, and then changing the rounding mode
  back. Because we didn't model FPCW and its dependency chain, other
  instructions could be scheduled into the middle of the sequence.

  This patch introduces the register and adds it as an implciit def of
  FLDCW and implicit use of the FP binary arithmetic instructions and
  store instructions. There are more instructions that need to be
  updated, but this is a good start. I believe this fixes at least the
  reduced test case from PR40529.

  Reviewers: RKSimon, spatel, rnk, efriedma, andrew.w.kaylor

  Subscribers: dim, llvm-commits

  Tags: #llvm

  Differential Revision: https://reviews.llvm.org/D57735

These should fix a problem in clang 7.0 where it would sometimes emit
long double floating point instructions in a slightly wrong order,
leading to failures in our libm tests.  In particular, the cbrt_test
test case 'cbrtl_powl' and the trig_test test case 'reduction'.

Also bump __FreeBSD_cc_version, to be able to detect this in our test
suite.

Reported by:    lwhsu
PR:		234040
Upstream PR:	https://bugs.llvm.org/show_bug.cgi?id=40206
MFC after:	1 week
2019-02-08 18:24:53 +00:00
Brooks Davis
60835849e5 GC a historical artifact.
Back in 1993, the fgetln (then fgetline) interface was changed to not
return a C string.  The change was accomplished by ifdefing out the code
that did the termination.  Changing the interface would violate our API
stability rules so remove the old implementation.

Sponsored by:	DARPA, AFRL
2019-02-07 23:17:44 +00:00
Sean Eric Fagan
18bb746b40 r343881 had an uninitialized error. This fixes that.
PR:             233849
Reported by:    Andre Albsmeier
MFC after:      1 month
Sponsored by:   iXsystems Inc
Differential Revision:  https://reviews.freebsd.org/D18785
2019-02-07 22:10:20 +00:00
Sean Eric Fagan
148d31b83d r339008 broke repquota for UFS. This rectifies that.
Refactor the function calls and tests so that, on UFS, the proper fields
are filled out.

PR:		233849
Reported by:	Andre Albsmeier
Reviewed by:	mav, delphij
MFC after:	1 month
Sponsored by:	iXsystems Inc
Differential Revision:	https://reviews.freebsd.org/D18785
2019-02-07 21:51:39 +00:00
Konstantin Belousov
fd90aeaabd Add comment noting that the strange spelling of GenuineIntel is for reason.
Requested by:	rpokala
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-07 03:58:29 +00:00
Konstantin Belousov
5925a98339 Use ifunc to select the barrier instruction for RDTSC.
This optimizes out runtime switch and removes yet another cpuid from
libc.

Note that this is the first use of ifunc in i386 libc, so
ifunc-capable toolchain is required for building runnable userspace on
i386, same as on amd64.

Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-07 02:56:10 +00:00
Jason A. Harmening
f048354550 r341692 changed cap_syslog(3) to preserve the stdio descriptors inherited
from its parent so that LOG_PERROR would work.  However, this caused
dhclient(8)'s stdio streams to remain open across daemonization, breaking
the ability to capture its foreground output as done in netconfig_ipv4.

Fix this by reverting r341692 and instead passing the parent's stderr
descriptor as an argument to cap_openlog() only when LOG_PERROR is specified
in logopt.

PR:	234514
Suggested by:	markj
Reported by:	Shawn Webb
Reviewed by:	markj, oshogbo
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D18989
2019-02-06 04:36:28 +00:00
Dimitry Andric
8e69ae1c97 Merge ^/head r343712 through r343806. 2019-02-05 19:50:46 +00:00
Dimitry Andric
c8630eab15 Merge llvm, clang, compiler-rt, libc++, lld, and lldb release_80 branch
r353167, resolve conflicts, and bump version numbers.
2019-02-05 19:48:24 +00:00
Dimitry Andric
648606c4b5 Vendor import of compiler-rt release_80 branch r353167:
https://llvm.org/svn/llvm-project/compiler-rt/branches/release_80@353167
2019-02-05 18:39:32 +00:00
Dimitry Andric
d87c9e7da2 Vendor import of clang release_80 branch r353167:
https://llvm.org/svn/llvm-project/cfe/branches/release_80@353167
2019-02-05 18:39:15 +00:00
Dimitry Andric
e79719ce60 Vendor import of llvm release_80 branch r353167:
https://llvm.org/svn/llvm-project/llvm/branches/release_80@353167
2019-02-05 18:38:58 +00:00
Matt Macy
334fd3dabc Fix deterministic builds by sorting input to fts in jevents
Reported by: emaste@
2019-02-05 00:31:25 +00:00
Warner Losh
52467047aa Regularize the Netflix copyright
Use recent best practices for Copyright form at the top of
the license:
1. Remove all the All Rights Reserved clauses on our stuff. Where we
   piggybacked others, use a separate line to make things clear.
2. Use "Netflix, Inc." everywhere.
3. Use a single line for the copyright for grep friendliness.
4. Use date ranges in all places for our stuff.

Approved by: Netflix Legal (who gave me the form), adrian@ (pmc files)
2019-02-04 21:28:25 +00:00
Konstantin Belousov
e4314da2e4 Fixes for very early use of the pthread_mutex_* and libthr malloc.
When libthr is statically linked into the binary, order of the
constructors execution is not deterministic.  It is possible for the
application constructor to use pthread_mutex_* functions before the
libthr initialization was done.

Handle it by:
- making thr_malloc.c locking functions operational when curthread is not
  yet set;
- making __thr_malloc_init() idempotent, allowing more than one call to it;
- unconditionally calling __thr_malloc_init() before initializing
  a process-private mutex.

Reported and tested by:	mmel
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-02-04 21:16:15 +00:00
Enji Cooper
8e0c33c6d3 Avoid the DNS lookup for "localhost"
ci.FreeBSD.org does not have access to a DNS resolver/network (unlike my test
VM), so in order for the test to pass on the host, it needs to avoid the DNS
lookup by using the numeric host address representation.

PR:		235200
Reviewed by:	asomers, lwhsu
Approved by:	emaste (mentor)
MFC after:	2 weeks
MFC with:	r343362, r343365, r343367-r343368, r343461
Differential Revision: https://reviews.freebsd.org/D19026
2019-02-04 19:12:45 +00:00
Michael Tuexen
bac4817b3a Add missing SCTP_EOR entry.
MFC after:		3 days
2019-02-04 13:30:47 +00:00
Dimitry Andric
c2c227a536 Merge ^/head r343571 through r343711. 2019-02-03 11:41:43 +00:00
Eric van Gyzen
50b06886a7 libm: squelch -Woverflow from gcc6
Sponsored by:	Dell EMC Isilon
2019-02-01 23:15:54 +00:00
Kyle Evans
9303f81955 libc/tests: Add test case for jemalloc/libthr bug fixed in r343566
Submitted by:	Andrew Gierth (original reproducer; kevans massaged for atf)
Reviewed by:	kib
MFC after:	2 weeks
X-MFC-with:	r343566 (or after)
Differential Revision:	https://reviews.freebsd.org/D19027
2019-01-31 02:49:24 +00:00
Konstantin Belousov
d49ca25de6 Rename rtld-elf/malloc.c to rtld-elf/rtld_malloc.c.
Then malloc.c file name is too generic to use it for libthr.a.

Sponsored by:	The FreeBSD Foundation
MFC after:	13 days
2019-01-30 16:28:27 +00:00
Hans Petter Selasky
21be80ae80 Add support for Audio Sink and Audio Source profiles to sdpd(8).
This allows user-space programs like virtual_oss(8) to act
as a Bluetooth speaker device.

MFC after:		1 week
Sponsored by:		Mellanox Technologies
2019-01-30 09:44:54 +00:00
Dimitry Andric
7e565c552a Merge ^/head r343320 through r343570. 2019-01-30 07:10:33 +00:00
Konstantin Belousov
9a52756044 i386: Merge PAE and non-PAE pmaps into same kernel.
Effectively all i386 kernels now have two pmaps compiled in: one
managing PAE pagetables, and another non-PAE. The implementation is
selected at cold time depending on the CPU features. The vm_paddr_t is
always 64bit now. As result, nx bit can be used on all capable CPUs.

Option PAE only affects the bus_addr_t: it is still 32bit for non-PAE
configs, for drivers compatibility. Kernel layout, esp. max kernel
address, low memory PDEs and max user address (same as trampoline
start) are now same for PAE and for non-PAE regardless of the type of
page tables used.

Non-PAE kernel (when using PAE pagetables) can handle physical memory
up to 24G now, larger memory requires re-tuning the KVA consumers and
instead the code caps the maximum at 24G. Unfortunately, a lot of
drivers do not use busdma(9) properly so by default even 4G barrier is
not easy. There are two tunables added: hw.above4g_allow and
hw.above24g_allow, the first one is kept enabled for now to evaluate
the status on HEAD, second is only for dev use.

i386 now creates three freelists if there is any memory above 4G, to
allow proper bounce pages allocation. Also, VM_KMEM_SIZE_SCALE changed
from 3 to 1.

The PAE_TABLES kernel config option is retired.

In collaboarion with: pho
Discussed with:	emaste
Reviewed by:	markj
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D18894
2019-01-30 02:07:13 +00:00
Konstantin Belousov
381c2d2e9a Untangle jemalloc and mutexes initialization.
The need to use libc malloc(3) from some places in libthr always
caused issues.  For instance, per-thread key allocation was switched to
use plain mmap(2) to get storage, because some third party mallocs
used keys for implementation of calloc(3).

Even more important, libthr calls calloc(3) during initialization of
pthread mutexes, and jemalloc uses pthread mutexes.  Jemalloc provides
some way to both postpone the initialization, and to make
initialization to use specialized allocator, but this is very fragile
and often breaks.  See the referenced PR for another example.

Add the small malloc implementation used by rtld, to libthr. Use it in
thr_spec.c and for mutexes initialization. This avoids the issues with
mutual dependencies between malloc and libthr in principle.  The
drawback is that some more allocations are not interceptable for
alternate malloc implementations.  There should be not too much memory
use from this allocator, and the alternative, direct use of mmap(2) is
obviously worse.

PR:	235211
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D18988
2019-01-29 22:46:44 +00:00
Konstantin Belousov
e5ac304989 Bump SPECNAMELEN to MAXNAMLEN.
This includes the bump for cdevsw d_version.  Otherwise, the impact on
the ABI (not KBI) is surprisingly low.  The most important affected
interface is devname(3) and ttyname(3) which already correctly handle
long names (and ttyname(3) should not be affected at all).

Still, due to the d_version bump, I argue that the change is not MFC-able.

Requested by:	mmacy
Reviewed by:	jhb
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D18932
2019-01-27 00:46:06 +00:00
Stefan Eßer
59ba78ccae Slightly improve previous commit that silenced a Clang Scan warning.
The strdup() call does not take advantage of the known length of the
source string. Replace by malloc() and memcpy() utilizimng the pre-
calculated string length.

Submitted by:	cperciva
Reported by:	rgrimes
MFC after:	2 weeks
2019-01-26 22:24:15 +00:00
Stefan Eßer
4268f3b3e0 Silence Clang Scan warning about potentially unsafe use of strcpy.
While this is a false positive, the use of strdup() simplifies the code.

MFC after:	2 weeks
2019-01-26 21:30:26 +00:00
Mariusz Zaborski
28f4385e45 libcasper: do not run registered exit functions
Casper library should not use exit(3) function because before setting it up
applications may register it. Casper doesn't depend on any registered exit
function, so it safe to change this.

Reported by:	jceel
MFC after:	2 weeks
2019-01-26 14:10:49 +00:00
Enji Cooper
4ce7bd2527 Fix reporting errors with gai_strerror(..)
The return value (`err`) should be checked; not the `errno` value.

PR:		235200
Approved by:	emaste (mentor)
Reviewed by:	asomers, lwhsu
MFC after:	28 days
MFC with:	r343362, r343365, r343367-r343368
Differential Revision: https://reviews.freebsd.org/D18969
2019-01-26 03:43:12 +00:00
Enji Cooper
2b9ecf4896 Document that sendfile will return an invalid value for sbytes if provided an invalid address
This is meant to clarify the fact that the system call will not fail
with -1/EFAULT, as one might expect, when reading the sendfile(2)
manpage today.

While here, pet the mandoc linter, when dealing with the section that
describes valid values for `flags`.

PR:	232210
MFC after:	2 weeks
Approved by:	emaste (mentor)
Reviewed by:	glebius, 0mp
Differential Revision: https://reviews.freebsd.org/D18949
2019-01-25 19:56:02 +00:00
Li-Wen Hsu
24df7b15fa Temporarily mark lib.msun.{cbrt_test.cbrtl_powl,trig_test.reduction}
expected failure after clang700-import merge

PR:		234040
Reviewed by:	ngie, markj
Approved by:	markj (mentor)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18938
2019-01-25 18:48:20 +00:00
Enji Cooper
e190da54d3 Fix up r343367
I should have only changed the format qualifier with the `size_t` value,
`length`, not the other [`off_t`] value, `dest_file_size`.

MFC after:	1 month
MFC with:	r343362, r343365, r343367
Approved by:	emaste (mentor; implicit)
Reported by:	gcc 8.x
2019-01-23 23:48:57 +00:00
Enji Cooper
8dd6af34bc Unbreak the build on architectures where size_t isn't synonymous with uintmax_t
I should have used `%zu` instead of `%ju` with `size_t` types.

MFC after:	1 month
MFC with:	r343362, r343365
Approved by:	emaste (mentor; implicit)
Reviewed by:	asomers
Pointyhat to:	ngie
Submitted by:	asomers
Differential Revision: https://reviews.freebsd.org/D18935
2019-01-23 23:30:55 +00:00
Enji Cooper
de00e09d82 Unbreak the gcc build with sendfile_test after r343362
gcc 8.x is more pedantic than clang 7.x with format strings and the tests
passed `void*` variables while supplying `%s` (which is technically
incorrect).

Make the affected `void*` variables use `char*` storage instead to address
this issue, as the compiler will upcast the values to `char*`.

MFC after:	1 month
MFC with:	r343362
Approved by:	emaste (mentor; implicit)
Reviewed by:	asomers
Differential Revision: https://reviews.freebsd.org/D18934
2019-01-23 23:06:39 +00:00
Enji Cooper
b29e142648 Add [initial] functional tests for sendfile(2) as lib/libc/sys/sendfile
These testcases exercise a number of functional requirements for sendfile(2).

The testcases use IPv4 and IPv6 domain sockets with TCP, and were confirmed
functional on UFS and ZFS. UDP address family sockets cannot be used per the
sendfile(2) contract, thus using UDP sockets is outside the scope of
testing the syscall in positive cases. As seen in
`:s_negative_udp_socket_test`, UDP is used to test the sendfile(2) contract
to ensure that EINVAL is returned by sendfile(2).

The testcases added explicitly avoid testing out `SF_SYNC` due to the
complexity of verifying that support. However, this is a good next logical
item to verify.

The `hdtr_positive*` testcases work to a certain degree (the header
testcases pass), but the trailer testcases do not work (it is an expected
failure). In particular, the value received by the mock server doesn't match
the expected value, and instead looks something like the following (using
python array notation):

`trailer[:]message[1:]`

instead of:

`message[:]trailer[:]`

This makes me think there's a buffer overrun issue or problem with the
offset somewhere in the sendfile(2) system call, but I need to do some
other testing first to verify that the code is indeed sane, and my
assumptions/code isn't buggy.

The `sbytes_negative` testcases that check `sbytes` being set to an
invalid value resulting in `EFAULT` fails today as the other change
(which checks `copyout(9)`) has not been committed [1]. Thus, it
should remain an expected failure (see bug 232210 for more details
on this item).

Next steps for testing sendfile(2):
1. Fix the header/trailer testcases so that they pass.
2. Setup if_tap interface and test with it, instead of using "localhost", per
   @asomers's suggestion.
3. Handle short recv(2)'s in `server_cat(..)`.
4. Add `SF_SYNC` support.
5. Add some more negative tests outside the scope of the functional contract.

MFC after:	1 month
Reviewed by:	asomers
Approved by:	emaste (mentor)
PR: 		232210
Sponsored by:   Netflix, Inc
Differential Revision: https://reviews.freebsd.org/D18625
2019-01-23 22:00:17 +00:00
Kyle Evans
16ac070581 libbe(3): simplify import, allow replication streams
Previously, we directly used libzfs_core's lzc_receive to import to a
temporary snapshot, then cloned the snapshot and setup the properties. This
failed when attempting to import replication streams with questionable
error.

libzfs's zfs_receive is a much better fit here, so we now use it instead
with the destination dataset and let libzfs take care of the dirty details.
be_import is greatly simplified as a result.

Reported by:	Marie Helene Kvello-Aune <freebsd@mhka.no>
MFC after:	1 week
2019-01-23 02:09:15 +00:00
Dimitry Andric
88148a071b Merge ^/head r343202 through r343319. 2019-01-22 20:24:58 +00:00
Dimitry Andric
af44a0113e Now for the release_80 branch, update version numbers for llvm, clang
and lld, and regenerate config headers.
2019-01-22 20:24:10 +00:00
Dimitry Andric
8dabf201f0 Update compiler-rt build glue. 2019-01-22 20:08:25 +00:00
Dimitry Andric
58a7b5d64a Update libc++ build glue, and ObsoleteFiles. 2019-01-22 19:03:51 +00:00
Dimitry Andric
6e3f21a20c Update lldb build glue to make MK_LLDB=yes work. 2019-01-22 18:03:45 +00:00
Mark Johnston
504b5801bd List caph_limit_stream() in the synopsis.
Fix typos while here.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-01-21 20:56:29 +00:00
Dimitry Andric
efa7559714 Update llvm and clang build glue to make MK_CLANG_EXTRAS=yes and
MK_CLANG_FULL=yes work.
2019-01-21 19:39:35 +00:00
Dimitry Andric
5e86819c72 Add and remove sources from libllvm and libclang, to make the minimal
clang executable (with all options except targets off) build.
2019-01-21 06:52:55 +00:00
Dimitry Andric
676320a018 Update libllvmminimal and llvm-tblgen Makefiles, so llvm-tblgen and
clang-tblgen can build.
2019-01-20 18:53:32 +00:00
Dimitry Andric
0bf31f1fdc Update version numbers, and regenerate config headers for llvm, clang,
lld and lldb.  Update ObsoleteFiles.inc and OptionalObsoleteFiles.inc.
2019-01-20 18:34:30 +00:00
Dimitry Andric
292698a853 Vendor import of clang release_80 branch r351543:
https://llvm.org/svn/llvm-project/cfe/branches/release_80@351543
2019-01-19 18:44:43 +00:00
Dimitry Andric
3edec5c15a Vendor import of llvm release_80 branch r351543:
https://llvm.org/svn/llvm-project/llvm/branches/release_80@351543
2019-01-19 18:44:22 +00:00
Dimitry Andric
e2fd426bda Vendor import of lld trunk r351319 (just before the release_80 branch
point):
https://llvm.org/svn/llvm-project/lld/trunk@351319
2019-01-19 10:05:49 +00:00
Dimitry Andric
6012fe9abb Vendor import of libc++ trunk r351319 (just before the release_80
branch point):
https://llvm.org/svn/llvm-project/libcxx/trunk@351319
2019-01-19 10:05:35 +00:00
Dimitry Andric
0646903fc1 Vendor import of compiler-rt trunk r351319 (just before the release_80
branch point):
https://llvm.org/svn/llvm-project/compiler-rt/trunk@351319
2019-01-19 10:05:08 +00:00
Dimitry Andric
676fbe8105 Vendor import of clang trunk r351319 (just before the release_80 branch
point):
https://llvm.org/svn/llvm-project/cfe/trunk@351319
2019-01-19 10:04:05 +00:00
Dimitry Andric
d8e91e4626 Vendor import of llvm trunk r351319 (just before the release_80 branch
point):
https://llvm.org/svn/llvm-project/llvm/trunk@351319
2019-01-19 10:01:25 +00:00
Dag-Erling Smørgrav
c9cf7cb85b Revert r343093 until I can address the issues raised by kib@. 2019-01-17 16:50:50 +00:00
Kirk McKusick
88640c0e8b Create new EINTEGRITY error with message "Integrity check failed".
An integrity check such as a check-hash or a cross-correlation failed.
The integrity error falls between EINVAL that identifies errors in
parameters to a system call and EIO that identifies errors with the
underlying storage media. EINTEGRITY is typically raised by intermediate
kernel layers such as a filesystem or an in-kernel GEOM subsystem when
they detect inconsistencies. Uses include allowing the mount(8) command
to return a different exit value to automate the running of fsck(8)
during a system boot.

These changes make no use of the new error, they just add it. Later
commits will be made for the use of the new error number and it will
be added to additional manual pages as appropriate.

Reviewed by:    gnn, dim, brueffer, imp
Discussed with: kib, cem, emaste, ed, jilles
Differential Revision: https://reviews.freebsd.org/D18765
2019-01-17 06:35:45 +00:00
Jilles Tjoelker
fbdcf603c8 libedit: Avoid out of bounds read in 'bind' command
This is CVS revision 1.31 from NetBSD lib/libedit/chartype.c:
Make sure that argv is NULL terminated since functions like tty_stty rely
on it to be so (Gerry Swinslow)

This broke when the wide-character support was enabled in libedit. The
conversion from multibyte to wide-character did not supply the apparently
expected terminating NULL in the new argv array.

PR:		233343
Submitted by:	Yuichiro NAITO
Obtained from:	NetBSD
MFC after:	1 week
2019-01-16 21:59:18 +00:00
Dag-Erling Smørgrav
9b35e90238 Implement dlopenat(3).
MFC after:	3 weeks
2019-01-16 12:12:40 +00:00
Konstantin Belousov
ea7e7006db Implement shmat(2) flag SHM_REMAP.
Based on the description in Linux man page.

Reviewed by:	markj, ngie (previous version)
Sponsored by:	Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D18837
2019-01-16 05:15:57 +00:00
Gleb Smirnoff
cf64f5197b This was missed in r343051: make uz_allocs, uz_frees and uz_fails counter(9). 2019-01-15 18:47:19 +00:00
Gleb Smirnoff
bb15d1c778 o Move zone limit from keg level up to zone level. This means that now
two zones sharing a keg may have different limits. Now this is going
  to work:

  zone = uma_zcreate();
  uma_zone_set_max(zone, limit);
  zone2 = uma_zsecond_create(zone);
  uma_zone_set_max(zone2, limit2);

  Kegs no longer have uk_maxpages field, but zones have uz_items. When
  set, it may be rounded up to minimum possible CPU bucket cache size.
  For small limits bucket cache can also be reconfigured to be smaller.
  Counter uz_items is updated whenever items transition from keg to a
  bucket cache or directly to a consumer. If zone has uz_maxitems set and
  it is reached, then we are going to sleep.

o Since new limits don't play well with multi-keg zones, remove them. The
  idea of multi-keg zones was introduced exactly 10 years ago, and never
  have had a practical usage. In discussion with Jeff we came to a wild
  agreement that if we ever want to reintroduce the idea of a smart allocator
  that would be able to choose between two (or more) totally different
  backing stores, that choice should be made one level higher than UMA,
  e.g. in malloc(9) or in mget(), or whatever and choice should be controlled
  by the caller.

o Sleeping code is improved to account number of sleepers and wake them one
  by one, to avoid thundering herd problem.

o Flag UMA_ZONE_NOBUCKETCACHE removed, instead uma_zone_set_maxcache()
  KPI added. Having no bucket cache basically means setting maxcache to 0.

o Now with many fields added and many removed (no multi-keg zones!) make
  sure that struct uma_zone is perfectly aligned.

Reviewed by:	markj, jeff
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D17773
2019-01-15 00:02:06 +00:00
Pedro F. Giffuni
76c528d350 msun: reduce diff between src/e_j0.c and src/e_j0f.c
PR:	229501
MFC after:	1 week
2019-01-14 15:48:35 +00:00
Justin Hibbits
5d031e332f Create crtsavres.o for powerpc builds
Summary:
GCC expects to link in a crtsavres.o on powerpc platforms.  On
powerpc64 this is an empty file, but on powerpc and powerpcspe this does contain
some save/restore functions, which may not actually be necessary for newer
modern GCC and clang.  This appeases the in-tree gcc, though, and is needed in
order to switch to the BSD CRTRBEGIN.

PR:	233751
Reviewed By: andrew
Differential Revision: https://reviews.freebsd.org/D18826
2019-01-12 21:29:54 +00:00
Cy Schubert
f2aac055e4 Disable FTS3, FTS4, and RTREE in bundled and private sqlite3.
Suggested by:	delphij@
Reviewed by:	delphij@ (for secteam@)
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D18786
2019-01-12 17:56:23 +00:00
Ed Maste
ae500c1ff8 Update to ELF Tool Chain r3668
Highlights:
- Make sure that only TLS sections are sorted into TLS segment.
- Fixed multiple errors in "Section to Segment mapping".
- Man page updates
- ar improvements
- elfcopy: avoid filter_reloc uninitialized variable for rela
- elfcopy: avoid stripping relocations from static binaries
- readelf: avoid printing directory in front of absolute path
- readelf: add NT_FREEBSD_FEATURE_CTL FreeBSD note type
- test improvements

NOTES:

Some of these changes originated in FreeBSD and simply reduce diffs
between contrib and vendor.

ELF Tool Chain ar is not (currently) used in FreeBSD, and there are
improvements in both FreeBSD and ELF Tool Chain ar that are not in
the other.

Sponsored by:	The FreeBSD Foundation
2019-01-10 14:35:23 +00:00
Kyle Evans
0a603a6ece libbe(3): Change be_mount to mount/unmount child datasets
This set of changes is geared towards making bectl respect deep boot
environments when they exist and are mounted. The deep BE composition
functionality (`bectl add`) remains disabled for the time being. This set of
changes has no effect for the average user. but allows deep BE users to
upgrade properly with their current setup.

libbe(3): Open the target boot environment and get a zfs handle, then pass
that with the target mountpoint to be_mount_iter; If the BE_MNT_DEEP flag is
set call zfs_iter_filesystems and mount the child datasets.

Similar logic is employed when unmounting the datasets, save for children
are unmounted first.

bectl(8): Change bectl_cmd_jail to pass the BE_MNT_DEEP flag when
calling be_mount as well as call be_unmount when cleaning up after the
jail has exited instead of umount(2) directly.

PR:		234795
Submitted by:	Wes Maag <jwmaag_gmail.com> (test additions by kevans)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D18796
2019-01-10 03:27:20 +00:00
Kyle Evans
fc13fc1c3a libbe(3): move altroot augmentation bits around a little bit
We could perhaps have a method that does this given a dataset, but it's yet
clear that we'll always want to bypass the altroot when we grab the
mountpoint. For now, we'll refactor things a bit so we grab the altroot
length when libbe is initialized and have a common method that does the
necessary augmentation (replace with / if it's the root, return a pointer to
later in the string if not).

This will be used in some upcoming work to make be_mount work properly for
deep BEs.

MFC after:	1 week
2019-01-09 22:31:10 +00:00
Xin LI
bf6e258bc4 Enable use of Capsicum sandbox when there is only one
input file (or the input is stdin) and we are not creating
output file.

Reported by:	Lasse Collin
MFC after:	1 month
2019-01-09 05:30:46 +00:00
Konstantin Belousov
3fbc2e00d1 Add a tunable which changes mincore(2) algorithm to only report data
from the local mapping.

Enable the setting by default.
The article behind the change: https://arxiv.org/abs/1901.01161

Reviewed by:	markj
Discussed with:	emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D18764
2019-01-07 22:10:48 +00:00
Pedro F. Giffuni
bdd8abc6d6 pow(3): Workaround possible signed shift Undefined Behavior.
j is int32_t and thus j<<31 is undefined if j==1.

Hinted by:	muusl-lib (git 688d3da0f1730daddbc954bbc2d27cc96ceee04c)
Discussed with:	freebsd-numerics (kargl)
2019-01-07 17:35:09 +00:00
Kyle Evans
f08dac4e90 libbe(3): Don't allow bootfs to be destroyed
Previously, the following sequence of events was feasible under some
circumstance:

bectl create test
bectl activate test
# the test BE dataset gets promoted and set as bootfs
bectl destroy test

I was unable to reproduce the destroy succeeding, but we should be rejecting
this before it even gets to libzfs because it would leave the system in an
inconsistent state. Forcing the user to be explicit as to which environment
should be activated instead is much better.

Reported by:	Graham Perrin <grahamperrin@gmail.com>
MFC after:	3 days
2019-01-07 16:16:47 +00:00
Jilles Tjoelker
8cc4b29d5a thr_wake(2): Minor mdoc fixes
MFC after:	1 week
2019-01-06 21:34:05 +00:00
Ed Maste
6871d48825 Add explicit csu test dependency
lib/csu/tests/dynamiclib requires libh_csu.so be built first.  I'm not
sure this is the most correct/best way to address this but it solves
the issue in my testing.

PR:		233734
Sponsored by:	The FreeBSD Foundation
2019-01-04 18:35:25 +00:00
Mark Johnston
2f2ddd68a5 Support MSG_DONTWAIT in send*(2).
As it does for recv*(2), MSG_DONTWAIT indicates that the call should
not block, returning EAGAIN instead.  Linux and OpenBSD both implement
this, so the change makes porting easier, especially since we do not
return EINVAL or so when unrecognized flags are specified.

Submitted by:	Greg V <greg@unrelenting.technology>
Reviewed by:	tuexen
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D18728
2019-01-04 17:31:50 +00:00
Kyle Evans
253b638eab getopt_long(3): fix case of malformed long opt
When presented with an arg string like '-l-', getopt_long will successfully
parse out the 'l' short option, then proceed to match '--' against the first
longopts entry as it later does a strncmp with len=0. This latter bit is
arguably another bug in itself, but presumably not a practical issue as all
callers of parse_long_options are already doing the right thing (except this
one pointed out).

An opt string like '-l-' should be considered malformed and throw a bad
argument rather than behaving as if '--' were passed. It cannot possibly do
what the invoker expects, and it's probably the result of a typo (ls -l- a)
rather than any intent.

Reported by:	Tony Overfield <toverfield@yahoo.com>
Reviewed by:	imp
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D18616
2019-01-04 03:13:24 +00:00
Romain Tartière
2f1a2bdf81 Add man page for dlvsym(3)
Add a short description of the function to the appropriate man page and add
reference to it where it makes sense.

Reviewed by:	bcr, markj, 0mp
Approved by:	markj
Differential Revision:	https://reviews.freebsd.org/D18725
2019-01-03 21:31:17 +00:00
Pedro F. Giffuni
50757b1452 msun: Fix some old typos.
Seen in a posting from July 27 by "CM Graff" in musl-libc.
2018-12-31 15:43:06 +00:00
Xin LI
1e4da04f06 Ensure buffer is nul-terminated.
MFC after:	2 weeks
2018-12-31 03:08:01 +00:00
Conrad Meyer
a0483764f3 Update to Zstandard 1.3.8
This merge brings in a couple new files, which needed to be attached to the
build; a new dependency on <limits.h>, which must be stubbed; and a name
change in the Context parameter constants, from ZSTD_p_foo to ZSTD_c_foo.

Significantly, it fixes a kernel build error with GCC where floating-point
functions were included in the kernel build, by hiding them under the same
compile-time #ifdef that already covered their invocation.  That issue was
introduced to FreeBSD in the 1.3.7 update and tracked upstream here:

  https://github.com/facebook/zstd/issues/1386

The full 1.3.8 release notes can be found on Github:

  https://github.com/facebook/zstd/releases/tag/v1.3.8

Relnotes:	yes
2018-12-29 21:18:01 +00:00
Conrad Meyer
af73257b09 import zstd 1.3.8 2018-12-29 06:51:10 +00:00
Konstantin Belousov
311a17259e Fix WITHOUT_NLS build after r342551.
Reported by:	gj
MFC after:	13 days
Sponsored by:	The FreeBSD Foundation
2018-12-28 16:08:49 +00:00
Justin Hibbits
003fdafbea libm: Include float.h to get LDBL_MANT_DIG
The long double aliases of double functions are only exposed as aliases if
LDBL_MANT_DIG is 53 (same as DBL_MANT_DIG).  Without float.h included these
files were not exposing weak aliases as expected, leading to link failures
if programs use the *l functions.  This should fix editors/calligra on
targets with 64-bit long double, which uses erfl and erfcl.  Found on
powerpc64.

Reviewed by:	kargl@
2018-12-28 01:34:08 +00:00
Konstantin Belousov
cd49e866fc Bump sys_errlist size to keep ABI backward-compatible for some time.
Addition of the new errno values requires adding new elements to
sys_errlist array, which is actually ABI-incompatible, since ELF
records the object size.  Expand array in advance to 150 elements so
that we have our users to go over the issue only once, at least until
more than 53 new errors are added.

I did not bumped the symbol version, same as it was not done for
previous increases of the array size.  Runtime linker only copies as
much data into binary object on copy relocation as the binary'object
specifies.  This is not fixable for binaries which access sys_errlist
directly.

While there, correct comment and calculation of the temporary buffer
size for the message printed for unknown error.  The on-stack buffer
is used only for the number and delimiter since r108603.

Requested by:	mckusick
Reviewed by:	mckusick, yuripv
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential revision:	https://reviews.freebsd.org/D18656
2018-12-27 13:02:15 +00:00
Pedro F. Giffuni
09ed804717 gai_strerror() - Update string error messages according to RFC 3493.
Error messages in gai_strerror(3) vary largely among OSs.

For new software we largely replaced the obsoleted EAI_NONAME and
with EAI_NODATA but we never updated the corresponding message to better
match the intended use. We also have references to ai_flags and ai_family
which are not very descriptive for non-developer end users.

Bring new new error messages based on informational RFC 3493, which has
obsoleted RFC 2553, and make them consistent among the header adn
manpage.

MFC after:	1 month
Differentical Revision:	D18630
2018-12-23 18:15:48 +00:00
Yuri Pankov
e2a87ae3af regcomp: revert part of r341838 which turned out to be unrelated
and caused issues with search in less.

PR:		234066
Reviewed by:	pfg
Differential revision:	https://reviews.freebsd.org/D18611
2018-12-19 23:28:56 +00:00
Conrad Meyer
8c1c50ff87 Allow multi-byte thousands separators in strfmon(3)
PR:	234010
Reported by:	Jon Tejnung <jon AT herrskogen.se>
Reviewed by:	yuripv
Differential Revision:	https://reviews.freebsd.org/D18605
2018-12-19 22:57:47 +00:00
Dimitry Andric
176fdeee33 Update clang, llvm, lld, lldb, compiler-rt and libc++ version number to
7.0.1 release r349250.  There were no functional changes since the 7.0.1
rc3 import.

PR:		230240, 230355
Relnotes:	yes
MFC after:	2 months
X-MFC-With:	r341825
2018-12-15 14:08:41 +00:00
Dimitry Andric
5e0b1b5fb3 Vendor import of clang 7.0.1 release r349250:
https://llvm.org/svn/llvm-project/cfe/tags/RELEASE_701/final@349250
2018-12-15 13:52:12 +00:00
Alex Richardson
032f0fbbb6 Allow bootstrapping libnv on macOS and Linux
MacOS/Linux do not define struct cmsgcred but we need to bootstrap libnv
when building on non-FreeBSD systems. Since they are not used during
bootstrap we can just omit these two functions there.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D18472
2018-12-14 21:16:04 +00:00
Mariusz Zaborski
e57d2a07c0 libcapsicum: add missing links
Reported by:	manu
2018-12-12 20:32:17 +00:00
Yuri Pankov
547bc083d6 regcomp: reduce size of bitmap for multibyte locales
This fixes the obscure endless loop seen with case-insensitive
patterns containing characters in 128-255 range;  originally
found running GNU grep test suite.

Our regex implementation being kludgy translates the characters
in case-insensitive pattern to bracket expression containing both
cases for the character and doesn't correctly handle the case when
original character is in bitmap and the other case is not, falling
into the endless loop going through in p_bracket(), ordinary(),
and bothcases().

Reducing the bitmap to 0-127 range for multibyte locales solves this
as none of these characters have other case mapping outside of bitmap.
We are also safe in the case when the original character outside of
bitmap has other case mapping in the bitmap (there are several of those
in our current ctype maps having unidirectional mapping into bitmap).

Reviewed by:	bapt, kevans, pfg
Differential revision:	https://reviews.freebsd.org/D18302
2018-12-12 04:23:00 +00:00
Kirk McKusick
8f829a5cf0 Continuing efforts to provide hardening of FFS. This change adds a
check hash to the filesystem inodes. Access attempts to files
associated with an inode with an invalid check hash will fail with
EINVAL (Invalid argument). Access is reestablished after an fsck
is run to find and validate the inodes with invalid check-hashes.
This check avoids a class of filesystem panics related to corrupted
inodes. The hash is done using crc32c.

Note this check-hash is for the inode itself and not any of its
indirect blocks. Check-hash validation may be extended to also
cover indirect block pointers, but that will be a separate (and
more costly) feature.

Check hashes are added only to UFS2 and not to UFS1 as UFS1 is
primarily used in embedded systems with small memories and low-powered
processors which need as light-weight a filesystem as possible.

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-12-11 22:14:37 +00:00
Dimitry Andric
959530cc41 Upgrade our copies of clang, llvm, lld, lldb, compiler-rt and libc++ to
the upstream release_70 branch r348686 (effectively, 7.0.1 rc3).  The
release will follow very soon, but no more functional changes are
expected.

Release notes for llvm, clang and lld 7.0.0 are available here:
<http://releases.llvm.org/7.0.0/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/clang/docs/ReleaseNotes.html>
<http://releases.llvm.org/7.0.0/tools/lld/docs/ReleaseNotes.html>

PR:		230240, 230355
Relnotes:	yes
MFC after:	2 months
2018-12-11 19:05:28 +00:00
Dimitry Andric
893405b45f Merge ^/head r341764 through r341812. 2018-12-11 06:47:04 +00:00
Konstantin Belousov
eba8ab0e3e Remove special case handling for getfhat(fd, NULL, handle).
There is no reason for it to behave differently from openat(fd, NULL).
Also the handling did not worked because the substituted path was from
the system address space, causing EFAULT.

Submitted by:	Jack Halford <jack@gandi.net>
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D18501
2018-12-11 02:48:49 +00:00
Dimitry Andric
67350cb56a Merge ^/head r340918 through r341763. 2018-12-09 11:39:45 +00:00
Dimitry Andric
0b9890fcbf Merge llvm, clang, lld, lldb, compiler-rt and libc++ release_70 branch
r348686 (effectively 7.0.1 rc3), resolve conflicts, and bump version
numbers.

PR:		230240, 230355
2018-12-09 11:36:04 +00:00
Dimitry Andric
52c5eb8567 Vendor import of clang release_70 branch r348686:
https://llvm.org/svn/llvm-project/cfe/branches/release_70@348686
2018-12-08 14:31:58 +00:00
Dimitry Andric
cfd4990248 Vendor import of llvm release_70 branch r348686:
https://llvm.org/svn/llvm-project/llvm/branches/release_70@348686
2018-12-08 14:31:49 +00:00
Mark Johnston
08c4a937a6 Let the cap_syslog capability inherit stdio descriptors.
Otherwise cap_openlog(LOG_PERROR) doesn't work.

Reviewed by:	oshogbo
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18457
2018-12-07 15:52:50 +00:00
Konstantin Belousov
d1fd400a80 Add new file handle system calls.
Namely, getfhat(2), fhlink(2), fhlinkat(2), fhreadlink(2).  The
syscalls are provided for a NFS userspace server (nfs-ganesha).

Submitted by:	Jack Halford <jack@gandi.net>
Sponsored by:	Gandi.net
Tested by:	pho
Feedback from:	brooks, markj
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D18359
2018-12-07 15:17:29 +00:00
Mark Johnston
8e2a46c8bd Add zdopen(3) to complement zopen(3).
zdopen() can be used in capability mode.  Update zopen.3 accordingly
and fix some grammar nits while I'm here.

Reviewed by:	delphij
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18456
2018-12-06 20:03:06 +00:00
Yuri Pankov
16e2120af9 syslog(3): stdarg.h is needed only for vsyslog()
Reported by:	Brian Wood <woodbrian77@gmail.com>
Reviewed by:	asomers, ngie
Differential revision:	https://reviews.freebsd.org/D18334
2018-12-06 18:03:18 +00:00
Kirk McKusick
fb14e73cb4 Normally when an attempt is made to mount a UFS/FFS filesystem whose
superblock has a check-hash error, an error message noting the
superblock check-hash failure is printed and the mount fails. The
administrator then runs fsck to repair the filesystem and when
successful, the filesystem can once again be mounted.

This approach fails if the filesystem in question is a root filesystem
from which you are trying to boot. Here, the loader fails when trying
to access the filesystem to get the kernel to boot. So it is necessary
to allow the loader to ignore the superblock check-hash error and make
a best effort to read the kernel. The filesystem may be suffiently
corrupted that the read attempt fails, but there is no harm in trying
since the loader makes no attempt to write to the filesystem.

Once the kernel is loaded and starts to run, it attempts to mount its
root filesystem. Once again, failure means that it breaks to its prompt
to ask where to get its root filesystem. Unless you have an alternate
root filesystem, you are stuck.

Since the root filesystem is initially mounted read-only, it is
safe to make an attempt to mount the root filesystem with the failed
superblock check-hash. Thus, when asked to mount a root filesystem
with a failed superblock check-hash, the kernel prints a warning
message that the root filesystem superblock check-hash needs repair,
but notes that it is ignoring the error and proceeding. It does
mark the filesystem as needing an fsck which prevents it from being
enabled for writing until fsck has been run on it. The net effect
is that the reboot fails to single user, but at least at that point
the administrator has the tools at hand to fix the problem.

Reported by:    Rick Macklem (rmacklem@)
Discussed with: Warner Losh (imp@)
Sponsored by:   Netflix
2018-12-06 00:09:39 +00:00
Alan Somers
006678fd05 stat(2): clarify which syscalls modify file timestamps
The list of syscalls that modify st_atim, st_mtim, and st_ctim was quite out
of date and probably not accurate to begin with.  Update it, and make it
clear that the list is open-ended.

Reviewed by:	kib
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D18410
2018-12-05 17:28:40 +00:00
Alex Richardson
abc73e04c2 Allow bootstrapping libopenbsd on Linux
The getdtablecount.c file won't compile on Linux but it seems like none of
the bootstrap tools actually need it.

Reviewed By:	emaste, brooks
Differential Revision: https://reviews.freebsd.org/D14244
2018-12-05 10:58:02 +00:00
Brooks Davis
db19a093bb Remove MD __sys_* private symbols.
No references to any of these exist in the tree. The list was also
erratic with different architectures exporting different things
(arm64 and riscv exported none).

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18425
2018-12-05 00:46:09 +00:00
Sean Bruno
8dfd9dcd55 Revert r340997 at the request of multiple users.
- breaks ports-mgmt/pkg build for mips64, powerpc64 and i386 for some users.

--- pkg-static ---
/usr/lib/liblzma.a(stream_encoder_mt.o): In function `mythread_cond_init':
/usr/local/poudriere/jails/ppc64/usr/src/contrib/xz/src/common/mythread.h:230:
undefined reference to `pthread_condattr_init'

Reported by:	jhibbits zeising
2018-12-04 03:23:14 +00:00
Konstantin Belousov
a66732de4f Print type designator 'D' for the KF_TYPE_DEV files.
No type-specific data is provided by the kernel.

Sponsored by:	Mellanox Technologies
MFC after:	1 week
2018-12-03 23:42:04 +00:00
Brooks Davis
52af996782 Remove declarations of syscalls not used in libthr.
Reviewed by:	kib
Sponsored by:	DARPA, AFRL
2018-12-03 22:09:23 +00:00
Ed Maste
0e450664ad disable BIND_NOW in libc, libthr, and rtld
An issue remains with BIND_NOW and processes using threads.  For now,
restore libc's BIND_NOW disable, and also disable BIND_NOW in rtld and
libthr.

A patch is in review (D18400) that likely fixes this issue, but just
disable BIND_NOW pending further testing after it is committed.

PR:		233333
Sponsored by:	The FreeBSD Foundation
2018-12-03 15:59:46 +00:00
Justin Hibbits
0f3a4adcb2 Fix PowerPC64 ELFv1-specific problem in __elf_phdr_match_addr() leading to crash
in threaded programs that unload libraries.

Summary:
The GNOME update to 3.28 exposed a bug in __elf_phdr_match_addr(), which leads
to a crash when building devel/libsoup on powerpc64.

Due to __elf_phdr_match_addr() limiting its search to PF_X sections, on the
PPC64 ELFv1 ABI, it was never matching function pointers properly.

This meant that libthr was never cleaning up its atfork list in
__pthread_cxa_finalize(), so if a library with an atfork handler was unloaded,
libthr would crash on the next fork.

Normally, the null pointer check it does before calling the handler would avoid
this crash, but, due to PPC64 ELFv1 using function descriptors instead of raw
function pointers, a null check against the pointer itself is insufficient, as
the pointer itself was not null, it was just pointing at a function descriptor
that had been zeroed. (Which is an ABI violation.)

Calling a zeroed function descriptor on PPC64 ELFv1 causes a jump to address 0
with a zeroed r2 and r11.

Submitted by:	git_bdragon.rtk0.net
Reviewed By:	kib
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D18364
2018-12-01 20:39:20 +00:00
Dimitry Andric
6ae2bfad8a Vendor import of clang release_70 branch r348011:
https://llvm.org/svn/llvm-project/cfe/branches/release_70@348011
2018-12-01 15:41:40 +00:00
Dimitry Andric
d17fea9f41 Vendor import of llvm release_70 branch r348011:
https://llvm.org/svn/llvm-project/llvm/branches/release_70@348011
2018-12-01 15:41:24 +00:00
Mateusz Guzik
ddf6571230 amd64: align target memmove buffer to 16 bytes before using rep movs
See the review for sample test results.

Reviewed by:	kib (kernel part)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18401
2018-12-01 14:20:32 +00:00
Mateusz Guzik
94243af2da amd64: handle small memmove buffers with overlapping stores
Handling sizes of > 32 backwards will be updated later.

Reviewed by:	kib (kernel part)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D18387
2018-11-30 20:58:08 +00:00
Mariusz Zaborski
77ebcc05ea libcasper: do not fail if Capsicum is not avilable
Casper service can be also used on the kernels without Capsicum support.

Reported by:	sbruno
Tested by:	sbruno
2018-11-30 19:27:14 +00:00
Eric van Gyzen
dbb1c64e82 Use _thr_isthreaded() and _thr_setthreaded() wrappers
...instead of directly using the global variable.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-11-30 03:02:49 +00:00
Eric van Gyzen
ad8c236b28 _thr_setthreaded() cannot fail; change return type to void
Also remove logic to avoid unnecessary stores to the global variable.
Thread creation and destruction are heavy enough that any supposed savings
is in the noise.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-11-30 03:01:32 +00:00
Mateusz Guzik
2847cfce54 amd64: remove stale attribution for memmove work
While the routine started as expanded bcopy, it is now entirely rewritten.

Sponsored by:	The FreeBSD Foundation
2018-11-30 00:47:36 +00:00
Mateusz Guzik
dd219e5ea5 amd64: tidy up copying backwards in memmove
For non-ERMS case the code used handle possible trailing bytes with
movsb first and then followed it up with movsq. This also happened
to alter how calculations were done for other cases.

Handle the tail with regular movs, just like when copying forward.
Use leaq to calculate the right offset from the get go, instead of
doing separate add and sub.

This adjusts the offset for non-rep cases so that they can be used
to handle the tail.

The routine is still a work in progress.

Sponsored by:	The FreeBSD Foundation
2018-11-30 00:45:10 +00:00
Dag-Erling Smørgrav
a768df3e91 When deciding whether to send the complete URL or just the document part,
we were looking at the original URL rather than the one we were currently
processing.  This meant that if we were trying to retrieve an HTTP URL but
were redirected to an HTTPS URL, and HTTPS proxying was enabled, we would
send an invalid request and most likely get garbage back.

MFC after:	3 days
2018-11-27 16:23:17 +00:00
Dag-Erling Smørgrav
cdd2df880d Add a “skip_dsn” option to g_part's bootcode verb to prevent g_part_mbr
from setting the volume serial number.  This unbreaks older boot blocks
that don't support serial numbers, and allows boot0cfg to set the serial
number itself if requested by the user.

Submitted by:	lev@, yuripv@
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D17386
2018-11-27 14:58:19 +00:00
Dag-Erling Smørgrav
ceedec4bce A few more cases where strcasecmp() is no longer required.
MFC after:	1 week
2018-11-27 11:22:19 +00:00
Dag-Erling Smørgrav
8d9de5b10a Improve URL parsing. In particular, convert scheme and host to lowercase.
MFC after:	1 week
2018-11-27 10:45:14 +00:00
Dag-Erling Smørgrav
b68815f088 Support proxying FTP over HTTPS, not just HTTP.
There is probably a PR for this, but I can't find this, or remember who
submitted it.  The patch got lost in the noise of another that wasn't
ready to commit.

MFC after:	3 days
2018-11-27 10:06:41 +00:00
Dag-Erling Smørgrav
e86caa0bd0 Style cleanup. 2018-11-27 09:41:47 +00:00
Kirk McKusick
4391a9c0e9 Bring up to date with recently added functions berase(3), getinode(3),
putinode(3), sbget(3), and sbput(3).

Sponsored by: Netflix
2018-11-26 23:09:45 +00:00
Martin Matuska
d28d5e1457 libarchive configuration changes
- move HAVE_BZLIB_H, HAVE_LIBLZMA and HAVE_LZMA_H to config_freebsd.h
- activate support for multi-threaded lzma encoding [1]

PR:		233543 [1]
Reported by:	cem
MFC after:	1 week
2018-11-26 21:45:27 +00:00
Ed Maste
cac83db9b7 revert r340640 "libc: forcibly disable BIND_NOW"
When immediate bind mode is requested, as of r340675 rtld processes
irelocs in PLT immediately after other PLT relocs.  That addresses the
libc + BIND_NOW startup crash the workaround is no longer needed.

PR:		233333
2018-11-26 13:56:19 +00:00
Baptiste Daroussin
6bbd1d19fd libedit: improve multibyte support
Until this commit libedit only supported UTF-8 for multibyte charset
Improve it to support other multibyte charsets

Tested with eucJP and SJIS charsets.
Note that this change as been review and committed in upstream libedit
as well via christos@NetBSD

Submitted by:	naito.yuichiro _at_ gmail.com
Reviewed by:	bapt, pfg, yuripv, 0mp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D17903
2018-11-26 08:16:33 +00:00
Dimitry Andric
c1532e70f8 Merge ^/head r340869 through r340917. 2018-11-25 11:15:01 +00:00
Andrew Turner
266900be14 Re-enable the dynamiclib tests. These should be fixed by r340910. 2018-11-24 18:25:00 +00:00
Andrew Turner
e5c0fd4be6 Add the missing 0 at the end of the .jcr section.
Without this the dynamic library test was failing as it was calling
_Jv_RegisterClasses multiple times.
2018-11-24 18:23:53 +00:00
Andrew Turner
ab1e0d2410 Disable the dynamiclib test until a failure can be debugged 2018-11-24 16:05:40 +00:00
Dimitry Andric
3d5db45595 Merge ^/head r340427 through r340868. 2018-11-24 14:46:06 +00:00
Andrew Turner
e1fa5626cf Mark the function called by the MIPS .init/.fini sequence with .local.
As with r328939 we need to mark local symbols as such. Without this the
assembly parser treats the symbols as global and created relocations
against these private symbols.

MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-11-23 16:33:03 +00:00
Yuri Pankov
63cbe8d1d9 regexec: fix processing multibyte strings.
Matcher function incorrectly assumed that moffset that we get from
findmust is in bytes. Fix this by introducing a stepback function,
taking short path if MB_CUR_MAX is 1, and going back byte-by-byte,
checking if we have a legal character sequence otherwise.

PR:		153502
Reviewed by:	pfg, kevans
Approved by:	kib (mentor, implicit)
Differential revision:	https://reviews.freebsd.org/D18297
2018-11-23 15:49:18 +00:00
Mateusz Piotrowski
f5c8cb4c35 Cross-reference libbe(3) and bectl(8).
Those two manual pages are already referencing each other in the HISTORY
sections, which people might skip. Mention those manual pages explicitly in
the SEE ALSO sections.  Also, remove a reference to be(1) from libbe(3).

Reviewed by:	bcr
Approved by:	krion (mentor, implicit), mat (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D18136
2018-11-21 12:46:28 +00:00
Mateusz Piotrowski
cc58f749fb libbe(3): Put each error value in separate line.
As requested by a TODO in the source code.

Reviewed by:	bcr
Approved by:	krion (mentor, implicit), mat (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D18063
2018-11-21 11:22:44 +00:00
Ed Maste
b18e62b6da csu: use BSD-1-clause license on csu files
Copyright on these files is held by kib@ and/or the Foundation, and both
agree to this change.

Approved by:	kib
2018-11-20 21:04:20 +00:00
Ed Maste
bbce18b8d1 Add NT_FREEBSD_FEATURE_CTL ELF note to csu
This note will be used to allow binaries to opt out of, or in to,
upcoming vulnerability mitigation and other features.  It is not yet
connected but being added now to facilitate testing and ensure
compatibility with existing kernels and tools.

Reviewed by:	brooks, jhb, kib, markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17438
2018-11-20 20:59:49 +00:00
Edward Tomasz Napierala
a4f02d5d6d Make libthr(3) use sysconf(_SC_NPROCESSORS_CONF); this shaves off
two calls to sysctl(2) from the binary startup.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D18046
2018-11-19 18:24:08 +00:00
Edward Tomasz Napierala
3a9cdc5132 Make sysconf(_SC_PAGESIZE) return the value from getpagesize(3).
That avoids a syscall - getpagesize(3) gets the value from the ELF
aux strings.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17989
2018-11-19 18:23:17 +00:00
Ed Maste
00bf4d2515 libc: forcibly disable BIND_NOW
Building libc WITH_BIND_NOW results in segfault at process start.  For
now force BIND_NOW off until the root cause can be identified and fixed.

PR:		233333
Sponsored by:	The FreeBSD Foundation
2018-11-19 18:12:39 +00:00
Mariusz Zaborski
2863fd2f27 libcasper: provide compatibility with the old version of service
Some external tools like tcpdump(1) have upstream the changes with old limits
name. Because of that provide compatibility with the old names.

Reported by:	emaste
2018-11-19 17:22:52 +00:00
Kyle Evans
4ab5187d38 libbe(3): Handle non-ZFS rootfs better
If rootfs isn't ZFS, current version will emit an error claiming so and fail
to initialize libbe. As a consumer, bectl -r (undocumented) can be specified
to operate on a BE independently of whether on a UFS or ZFS root.

Unbreak this for the UFS case by only erroring out the init if we can't
determine a ZFS dataset for rootfs and no BE root was specified. Consumers
of libbe should take care to ensure that rootfs is non-empty if they're
trying to use it, because this could certainly be the case.

Some check is needed before zfs_path_to_zhandle because it will
unconditionally emit to stderr if the path isn't a ZFS filesystem, which is
unhelpful for our purposes.

This should also unbreak the bectl(8) tests on a UFS root, as is the case in
Jenkins' -test runs.

MFC after:	3 days
2018-11-19 16:47:21 +00:00
Kyle Evans
af43c24d3d libbe(3): Properly account for altroot when creating new BEs
Previously we would blindly copy the 'mountpoint' property, which includes
the altroot. The altroot needs to be snipped off prior to setting it on the
new BE, though, or you'll end up with a new BE and a mountpoint of /mnt with
altroot=/mnt

MFC after:	3 days
2018-11-19 02:16:20 +00:00
Kyle Evans
cc624025b4 bectl(3)/libbe(3): Allow BE root to be specified
Add an undocumented -r option preceding the bectl subcommand to specify a BE
root to operate out of. This will remain undocumented for now, as some
caveats apply:

- BEs cannot be activated in the pool that doesn't contain the rootfs
- bectl create cannot work out of the box without the -e option right now,
  since it defaults to the rootfs and cross-pool cloning doesn't work like
  that (IIRC)

Plumb the BE root through to libbe(3) so that some things -can- be done to
it, e.g.

bectl -r tank/ROOT create -e default upgrade
bectl -r tank/ROOT mount upgrade /mnt

this aides in some upgrade setups where rootfs is not necessarily ZFS, and
also makes it easier/possible to regression-test bectl when combined with a
file-backed zpool.

MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D18029
2018-11-19 02:12:08 +00:00
Mariusz Zaborski
fd253945ac Update the names in the LIMITS and EXAMPLES sections after r340363.
Reported by:	markj
2018-11-18 11:11:27 +00:00
Kyle Evans
efd6500d03 libbe(3): Rewrite be_unmount to stop mucking with getmntinfo(2)
Go through the ZFS layer instead; given a BE, we can derive the dataset,
zfs_open it, then zfs_unmount. ZFS takes care of the dirty details and
likely gets it more correct than we did for more interesting setups.

MFC after:	3 days
2018-11-17 19:19:37 +00:00
Kyle Evans
51aecc893a libbe(3): rewrite init to support chroot usage
libbe(3) currently uses zfs_be_root and locates which of its children is
currently mounted at "/". This is reasonable, but not correct in the case of
a chroot, for two reasons:

- chroot root may be of a different zpool than zfs_be_root
- chroot root will not show up as mounted at "/"

Fix both of these by rewriting libbe_init to work from the rootfs down.
zfs_path_to_zhandle on / will resolve to the dataset mounted at the new
root, rather than the real root. From there, we can derive the BE root/pool
and grab the bootfs off of the new pool. This does no harm in the average
case, and opens up bectl to operating on different pools for scenarios where
one may be, for instance, updating a pool that generally gets re-rooted into
from a separate UFS root or zfs bootpool.

While here, I've also:
- Eliminated the check for /boot and / to be on the same partition. This
  leaves one open to a setup where /boot (and consequently, kernel/modules)
  are not included in the boot environment. This may very well be an
  intentional setup done by someone that knows what they're doing, we should
  not kill BE usage because of it.

- Eliminated the validation bits of BEs and snapshots that enforced
  'mountpoint' to be "/" -- this broke when trying to operate on an imported
  pool with an altroot, but we need not be this picky.

Reported by:	philip
Reviewed by:	philip, allanjude (previous version)
Tested by:	philip
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D18012
2018-11-17 19:15:29 +00:00
Mateusz Guzik
088ac3ef4b amd64: handle small memset buffers with overlapping stores
Instead of jumping to locations which store the exact number of bytes,
use displacement to move the destination.

In particular the following clears an area between 8-16 (inclusive)
branch-free:

movq    %r10,(%rdi)
movq    %r10,-8(%rdi,%rcx)

For instance for rcx of 10 the second line is rdi + 10 - 8 = rdi + 2.
Writing 8 bytes starting at that offset overlaps with 6 bytes written
previously and writes 2 new, giving 10 in total.

Provides a nice win for smaller stores. Other ones are erratic depending
on the microarchitecture.

General idea taken from NetBSD (restricted use of the trick) and bionic
string functions (use for various ranges like in this patch).

Reviewed by:	kib (previous version)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17660
2018-11-16 00:44:22 +00:00
Mateusz Guzik
ad2ff705a4 amd64: sync up libc memset with the kernel version
- tidy up memset to have rax set earlier for small sizes
- finish the tail in memset with an overlapping store
- align memset buffers to 16 bytes before using rep stos

Sponsored by:	The FreeBSD Foundation
2018-11-15 20:28:35 +00:00
Mateusz Guzik
6fff634455 amd64: convert libc bzero to a C func to avoid future bloat
Reviewed by:	kib (previous version)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17549
2018-11-15 20:20:39 +00:00
Alan Somers
a14a34ef62 fcntl.2: document an additional error condition
MFC after:	2 weeks
2018-11-15 16:13:25 +00:00
Konstantin Belousov
1c4ca77890 Add d_off support for multiple filesystems.
The d_off field has been added to the dirent structure recently.
Currently filesystems don't support this feature.  Support has been
added and tested for zfs, ufs, ext2fs, fdescfs, msdosfs and unionfs.
A stub implementation is available for cd9660, nandfs, udf and
pseudofs but hasn't been tested.

Motivation for this feature: our usecase is for a userspace nfs server
(nfs-ganesha) with zfs.  At the moment we cache direntry offsets by
calling lseek once per entry, with this patch we can get the offset
directly from getdirentries(2) calls which provides a significant
speedup.

Submitted by:	Jack Halford <jack@gandi.net>
Reviewed by:	mckusick, pfg, rmacklem (previous versions)
Sponsored by:	Gandi.net
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D17917
2018-11-14 14:18:35 +00:00
Yuri Pankov
98f3acfaaf Fix WITHOUT_ICONV build after r340276.
Reported by:	olivier
Approved by:	kib (mentor, implicit)
2018-11-14 09:06:15 +00:00
Dimitry Andric
6149ed01a1 Merge ^/head r340368 through r340426. 2018-11-14 06:46:44 +00:00
Kirk McKusick
9fc5d538fc In preparation for adding inode check-hashes, clean up and
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getinode / putinode interface.

Convert the utilities that had been using the undocumented
interface to use the new documented interface.

No functional change (as for now the libufs library does not
do inode check-hashes).

Reviewed by:  kib
Tested by:    Peter Holm
Sponsored by: Netflix
2018-11-13 21:40:56 +00:00
Mark Johnston
991666adc7 Ensure that libnv can be used when kern.trap_enotcap=1.
libnv used fcntl(fd, F_GETFL) to test whether fd is a valid file
descriptor.  Aside from being racy, this check requires CAP_FCNTL
rights on fd.  Instead, use fcntl(fd, F_GETFD), which does not require
any capability rights.

Also remove some redundant fd_is_valid() checks to avoid extra system
calls; in many cases we were performing this check immediately before
dup()ing the descriptor.

Reviewed by:	cem, oshogbo (previous version)
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D17963
2018-11-13 20:07:55 +00:00
Andrew Turner
b7305f91e4 Run __cxa_finalize in shared objects in the destructor path.
When we have .dtors call them before .dtor handling, otherwise call from
a destructor.

PR:		233056
MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-11-13 15:28:27 +00:00
Ed Maste
7c807d7708 retire LINKER_FEATURES filter flag
And build libdl unconditionally.  All supported FreeBSD linkers accept
-F / --filter so there is no need to test for support.

Discussed with:	kib
Sponsored by:	The FreeBSD Foundation
2018-11-12 20:44:22 +00:00
Konstantin Belousov
44748aa487 Create namespace for the symbols added during 13-CURRENT cycle.
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-11-12 19:12:14 +00:00
Mariusz Zaborski
2607c01b66 Fix typo in the comparison.
This fix build with gcc.

Reported by:	jenkins
2018-11-12 18:37:31 +00:00
Mariusz Zaborski
cdd6ea94b0 libcasper: introduce cap_fileargs service
cap_fileargs is a Casper service which helps to sandbox applications that need
access to the filesystem namespace. The main purpose of the service is to make
easy to capsicumize applications that works on multiple files passed in argv.

We have a couple example of using it but we still treat this service as an
experimental one.

Reviewed by:	emsate (previous version), jonathan (partially)
Discussed with:	many
Differential Revision:	https://reviews.freebsd.org/D14407
2018-11-12 17:40:47 +00:00
Dimitry Andric
9b5cb2f639 Merge ^/head r340235 through r340367. 2018-11-12 16:41:41 +00:00
Mariusz Zaborski
752d135e0d libcasper: ange the name of limits in cap_dns so the intentions are obvious.
Reported by:	pjd
MFC after:	3 weeks
2018-11-12 15:52:45 +00:00
Konstantin Belousov
5b1fb8ec66 First draft of documentation for AT/O_BENEATH handling of the absolute
paths.

It was decided that committing the code and drafting of the man page
update is better than allowing the code to rot until wordsmithing
happens.

Reviewed by:	jilles (previous version)
Discussed with:	brooks, jilles, emaste
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D17714
2018-11-11 01:46:48 +00:00
Edward Tomasz Napierala
8d113f4aec Don't call stat(2) on nsswitch.conf(5) every time nsdispatch(3)
and dependent functions (eg getpwname(3)) get called. This can
improve performance of binaries that perform a lot of name
lookups, such as gssd(8). It also matches documented behaviour
of Linux and Solaris.

The old code is left in place, should anyone need it, guarded
by #ifdef NS_REREAD_CONF.

Reviewed by:	imp, bcr
MFC after:	2 weeks
Relnotes:	yes
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17934
2018-11-10 23:07:46 +00:00
Kyle Evans
8d4ce3586f libbe(3): Set canmount properly when activating a new BE
The previously activated BE should have canmount=noauto set on it upon
activation of the new BE, but we previously did not touch canmount on either
old or new BE.

PR:		233113
MFC after:	3 days
2018-11-10 20:42:29 +00:00
Alan Somers
98f8234b13 libjail: fix handling of allow.mount.fusefs in jailparam_init
fusefs is inconsistently named. The kernel module is named "fuse", but the
mount helper is named "mount_fusefs" and the jail(8) parameter is named
"allow.mount.fusefs". Special case it in libjail.

Reviewed by:	jamie
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D17929
2018-11-10 03:10:22 +00:00
Ed Maste
961eb44346 libllvm: Move SampleProfWriter to SRCS_MIN
It is required by llvm-profdata, now built by default under the
LLVM_COV knob.  The additional complexity that would come from avoiding
building it if CLANG_EXTRAS and LLVM_COV are both disabled is not worth
the small savings in build time.

Sponsored by:	The FreeBSD Foundation
2018-11-09 19:51:26 +00:00
Yuri Pankov
9eb7d595e1 Reset persistent mbstates when rune locale encoding changes.
This was shown to be a problem by side effect of now-enabled test case,
which was going through C, en_US.UTF-8, ja_JP.SJIS, and ja_JP.eucJP,
and failing eventually as data in mbrtowc's mbstate, that was
perfectly correct for en_US.UTF-8 was treated as incorrect for
ja_JP.SJIS, failing the entire test case.

This makes the persistent mbstates to be per ctype-component,
and not per-locale so we could easily reset the mbstates when
only LC_CTYPE is changed.

Reviewed by:	bapt, pfg
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17796
2018-11-09 03:32:53 +00:00
Dimitry Andric
8c1538224a Merge ^/head r340213 through r340234. 2018-11-07 18:54:24 +00:00
Dimitry Andric
c06e7b66a1 Merge ^/head r340126 through r340212. 2018-11-07 18:52:28 +00:00
Andrew Turner
a0e4ca397e Add the (untested) mips and sparc64 .init call sequences.
The BSD crtbegin/crtend code now builds on all architectures, however
further work is needed to check if it works correctly.

MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-11-07 09:49:25 +00:00
Mark Johnston
35b4d0f125 Document the fact that cap_limit_set() always frees the input nvlist.
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-11-05 21:01:45 +00:00
Kyle Evans
bc82609ae2 Move pmc* bits behind MK_PMC to fix WITHOUT_PMC build
No objection from:	mmacy
MFC after:	3 days
2018-11-05 00:20:58 +00:00
Yuri Pankov
dd7c41a378 Add hybrid C.UTF-8 locale being identical to default C locale except
that it uses the same ctype maps and functions as other UTF-8 locales.

Reviewed by:	bapt, cem, eadler
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17833
2018-11-04 22:13:22 +00:00
Mariusz Zaborski
d9c2248dd5 libcasper: using explicit_bzero in cap_grp service
Please notice that we still don't clean information in nvlist structures.

Submitted by:	David CARLIER <devnexen@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D16777
2018-11-04 19:43:59 +00:00
Mariusz Zaborski
577dff6a8d libcasper: fix limitations in dns service
The getaddrinfo(3) and gethostbyname(3) are used to return the address for a
given hostname. The getnameinfo(3) and gethostbyaddr(3) are used to return
hostname for a given address. Right now in casper, we have two limitations:
- NAME which allows resolving DNS names.
- ADDR which allows to do revert DNS lookups.

Before this change the rights was mixed up:
NAME - getnameinfo(3) and gethostbyname(3)
ADDR - gethostbyaddr(3) and getaddrinfo(3)

Which no matters on limitation allowed us to resolve DNS names and do DNS
lookups basically by using a different set of functions.

Now the NAME type allows getaddrinfo(3) and gethostbyname (3)functions,
and the ADDR names allow to use gethostbyaddr(3) and getnameinfo(3) functions.

Reviewed by:	pjd, bcr
MFC after:	3 weeks
Discussed with:	hrs
Differential Revision:	https://reviews.freebsd.org/D16930
2018-11-04 19:38:54 +00:00
Mariusz Zaborski
db9a6e4178 libcasper: Update example in man page to use cap_getnameinfo function.
Reviewed by:	hrs
Differential Revision:	https://reviews.freebsd.org/D16931
2018-11-04 19:31:44 +00:00
Mariusz Zaborski
fe24949034 libcasper: Document the cap_getaddrinfo and cap_getnameinfo functions
Reviewed by:	hrs
Differential Revision:	https://reviews.freebsd.org/D16929
2018-11-04 19:29:19 +00:00
Mariusz Zaborski
9e5493752a Fix a recusive call introduce in the r340130. 2018-11-04 18:52:59 +00:00
Dimitry Andric
f46f6c14b4 Update LLVM_VERSION_PATCH define in llvm-config.h. 2018-11-04 17:56:09 +00:00
Mariusz Zaborski
5a453d5f5b libcapsicum: Introduce caph_{rights,ioctls,fcntls}_limit
The idea behind those functions is not to force consumers to remember that there
is a need to check errno on failure. We already have a caph_enter(3) function
which does the same for cap_enter(2).

MFC after:	2 weeks
2018-11-04 17:22:58 +00:00
Eric van Gyzen
0a48773f8c Update expat to 2.2.6
Update contrib/expat by merging from the vendor branch.

Update expat_config.h manually, using
    make -C /usr/ports/textproc/expat2 configure
as a baseline.

MFC after:	1 month
Relnotes:	yes
Security:	yes; see contrib/expat/Changes since 2.2.0
Sponsored by:	Dell EMC Isilon
2018-11-04 16:08:59 +00:00
Dimitry Andric
2a22df74e9 Merge ^/head r339813 through r340125. 2018-11-04 15:49:06 +00:00
Dimitry Andric
689486003b Merge llvm, clang, lld, lldb, compiler-rt and libc++ release_70 branch
r346007 (effectively 7.0.1 rc2), resolve conflicts, and bump version
numbers.

PR:		230240, 230355
2018-11-04 15:46:30 +00:00
Dimitry Andric
ce9602e289 Vendor import of clang release_70 branch r346007:
https://llvm.org/svn/llvm-project/cfe/branches/release_70@346007
2018-11-04 14:42:02 +00:00
Dimitry Andric
86392292ee Vendor import of llvm release_70 branch r346007:
https://llvm.org/svn/llvm-project/llvm/branches/release_70@346007
2018-11-04 14:41:47 +00:00
Mateusz Piotrowski
6d080f8660 csqrt(3): Fix some typos in the manual page
Submitted by:	Steve Kargl <sgk@troutmask.apl.washington.edu>
Reviewed by:	eadler
Approved by:    krion (mentor, implicit), mat (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17831
2018-11-04 14:11:38 +00:00
Matt Macy
dacc43df34 Add aditional counter descriptions to AMD 0x17
Submitted by:	Somalapuram Amaranath
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D17401
2018-11-04 06:24:27 +00:00
Yuri Pankov
32a8ec8bdd strptime: make %k and %l specifiers match their description in
strftime(3), and allow them to process space-padded input.

PR:		230720
Submitted by:	rlittle@inetco.com (original version)
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17761
2018-11-03 23:37:13 +00:00
Eric van Gyzen
77c85705a6 Vendor import of expat 2.2.6
Sponsored by:	Dell EMC Isilon
2018-11-02 22:10:07 +00:00
Conrad Meyer
78c2a9806e kern_poll: Restore explanatory comment removed in r177374
The comment isn't stale.  The check is bogus in the sense that poll(2)
does not require pollfd entries to be unique in fd space, so there is no
reason there cannot be more pollfd entries than open or even allowed
fds.  The check is mostly a seatbelt against accidental misuse or
abuse.  FD_SETSIZE, while usually unrelated to poll, is used as an
arbitrary floor for systems with very low kern.maxfilesperproc.

Additionally, document this possible EINVAL condition in the poll.2
manual.

No functional change.

Reviewed by:	markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D17671
2018-11-01 23:46:23 +00:00
Kyle Evans
4635676d25 libbe(3): Don't promote non-cloned BEs
Most easily reproducible by attempting to activate the currently activated
BE, one would get a "not a cloned filesystem" error instead of success or a
sane message.

PR:		232488
MFC after:	3 days
2018-11-01 14:00:56 +00:00
Ed Maste
57706376c6 Add __used to __CTOR_LIST__ and __DTOR_LIST__
Enabling BSD_CRTBEGIN on amd64 resulted in
error: unused variable '__CTOR_LIST__'.

__CTOR_LIST__ is indeed unused in crtbegin.c; it marks the beginning of
the .ctors array and is used in crtend.c.  Annotate __DTOR_LIST__ as
well for consistency.

Discussed with:	andrew
MFC with:	r339738
Sponsored by:	The FreeBSD Foundation
2018-10-31 14:19:58 +00:00
Andrew Turner
8522d140a5 Build the csu tests on all architectures.
The tests haven't been run them, but this is enough to build them so I can
get feedback on if the various crt.h headers are correct.

MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-10-30 14:44:12 +00:00
Andrew Turner
ca34591f29 Disable the .preinit_array test in DSOs, ld.bfd fails to link objects with
the section.

MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-10-30 11:19:47 +00:00
Andrew Turner
bb966f87bc Fix the location of the static keyword.
MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-10-30 10:16:21 +00:00
Andrew Turner
6ec0ee844c Run the csu tests on a DSO. This builds the tests into a shared library,
then runs these from the base test programs. With this we can check
crtbeginS.o and crtendS.o are working as expected.

MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-10-30 09:43:26 +00:00
Andrew Turner
9e26e65c16 The jcr argument to _Jv_RegisterClasses is used, stop marking it otherwise.
MFC with:	r339738
Sponsored by:	DARPA, AFRL
2018-10-30 09:36:31 +00:00
Yuri Pankov
281c29899f Connect libc/tests/time to the build, adding test cases for strptime()
issues fixed recently, and disabling the failing ones (mostly due to TZ
parsing differences with NetBSD).

Reviewed by:	ngie
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D17546
2018-10-30 02:37:23 +00:00