Commit Graph

252462 Commits

Author SHA1 Message Date
Matt Macy
1ea0b9a5a9 Fix userboot after r364355
r364355 replaced init_zfs_bootenv with init_zfs_boot_options and
neglected to update userboot in the process.
2020-08-25 17:23:33 +00:00
Matt Macy
5db7b015e9 Give stronger guidance with regards to upgrading root pools 2020-08-25 17:19:15 +00:00
Bjoern A. Zeeb
60e7f66931 rtsol(d): add script for "M bit"
While we do support the "O bit" running a script (usually to start a
dhcpv6 client) we have no options for setups which set the "M bit" for,
e.g., static address assignment as in EC2.

Duplicate most of the "O bit" logic to also start a script for the
"M bit" with the one difference: if the "M bit" is set we will not
start the script for the "O bit" as well (per RFC 4861, Section 4.2).

Reviewed by:	hrs, markj
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D26099
2020-08-25 16:09:23 +00:00
Vincenzo Maffione
ae750d5cdf iflib: netmap: publish all the receive buffer
At initialization time, the netmap RX refill function used to
prepare the NIC RX ring with N-1 buffers rather than N (with
N equal to the number of descriptors in the NIC RX ring).
This is not how netmap is supposed to work, as it would keep
kring->nr_hwcur not in sync with the NIC "next index to refill"
(i.e., fl->ifl_pidx). Instead we prepare N buffers, although we
still publish (with isc_rxd_flush()) only the first N-1 buffers,
to avoid the NIC producer pointer to overrun the NIC consumer
pointer (for NICs where this is a real issue, e.g. Intel ones).

MFC after:	2 weeks
2020-08-25 15:19:45 +00:00
Mateusz Guzik
a459a6cfe7 vfs: respect PRIV_VFS_LOOKUP in vaccess_smr
Reported by:	novel
2020-08-25 14:18:50 +00:00
Mark Johnston
411096d034 Permit vm_page_wire() to be called on pages not belonging to an object.
For such pages ref_count is effectively a consumer-managed field, but
there is no harm in calling vm_page_wire() on them.
vm_page_unwire_noq() handles them as well.  Relax the vm_page_wire()
assertions to permit this case which is triggered by some out-of-tree
code. [1]

Also guard a conditional assertion with INVARIANTS.  Otherwise the
conditions are evaluated even though the result is unused. [2]

Reported by:	bz, cem [1], kib [2]
Reviewed by:	dougm, kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D26173
2020-08-25 13:45:06 +00:00
Alex Richardson
0c54932d50 Fix -Wundef warnings when building liblua
We need to define the LUA_FLOAT_INT64 macro even if we don't use it (copied
from stand/luaconf.h). While touching luaconf.h.dist also sync it with the
the 5.3.5 release version (matches the one in lib/liblua).

Reviewed By:	kevans
Differential Revision: https://reviews.freebsd.org/D25977
2020-08-25 13:30:34 +00:00
Alex Richardson
6b3a148c52 style.Makefile: list CSTD between WARNS and CFLAGS
This was suggested by emaste in https://reviews.freebsd.org/D25928 and
matches most uses in the tree.
2020-08-25 13:30:29 +00:00
Alex Richardson
6424881cc8 Fix makefs bootstrap on macOS after D25563
The macOS assert.h header does not define static_assert when compiling in
C99 mode. To fix this compile with -std=c11.

Reviewed By:	emaste
Differential Revision: https://reviews.freebsd.org/D25928
2020-08-25 13:30:24 +00:00
Alex Richardson
dce3fcd4ff Fix typo in r364325 that broke tinderbox with -DBUILD_WITH_STRICT_TMPPATH
${TARGET_ARCH} is empty here which results in empy MAKE_PARAMS being
passed to the buildkernel phase. This breaks the build when using the
strict TMPPATH since cc will not be included in $PATH.

Reviewed By:	jhb
2020-08-25 13:30:19 +00:00
Alex Richardson
3ce13dbc23 Use bootstrapped install(1) install of tools/install.sh in world stage
This should be noticeably faster due to fewer processes being forked and
also handles other flags such as -S or writing to METALOG.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D26039
2020-08-25 13:30:14 +00:00
Alex Richardson
eb51ce8efb Fix running the builddtb target on a noexec file system
Obtained from:	CheriBSD
2020-08-25 13:30:09 +00:00
Alex Richardson
2b6ee34cf6 Pass -fuse-ld=/path/to/ld if ${LD} != "ld"
This is needed so that setting LD/XLD is not ignored when linking with $CC
instead of directly using $LD. Currently only clang accepts an absolute
path for -fuse-ld= (Clang 12+ will add a new --ld-path flag), so we now
warn when building with GCC and $LD != "ld" since that might result in the
wrong linker being used.

We have been setting XLD=/path/to/cheri/ld.lld in CheriBSD for a long time and
used a similar version of this patch to avoid linking with /usr/bin/ld.
This change is also required when building FreeBSD on an Ubuntu with Clang:
In that case we set XCC=/usr/lib/llvm-10/bin/clang and since
/usr/lib/llvm-10/bin/ does not contain a "ld" binary the build fails with
`clang: error: unable to execute command: Executable "ld" doesn't exist!`
unless we pass -fuse-ld=/usr/lib/llvm-10/bin/ld.lld.

This change passes -fuse-ld instead of copying ${XLD} to WOLRDTMP/bin/ld
since then we would have to ensure that this file does not exist while
building the bootstrap tools. The cross-linker might not be compatible with
the host linker (e.g. when building on macos: host-linker= Mach-O /usr/bin/ld,
cross-linker=LLVM ld.lld).

Reviewed By:	brooks, emaste
Differential Revision: https://reviews.freebsd.org/D26055
2020-08-25 13:30:03 +00:00
Alex Richardson
5bb9250e0a Add necessary Makefile.inc1 infrastructure for building on non-FreeBSD
The most awkward bit in this patch is the bootstrapping of m4:
We can't simply use the host version of m4 since that is not compatible
with the flags passed by lex (at least on macOS, possibly also on Linux).
Therefore we need to bootstrap m4, but lex needs m4 to build and m4 also
depends on lex (which needs m4 to generate any files). To work around this
cyclic dependency we can build a bootstrap version of m4 (with pre-generated
files) then use that to build the real m4.

This patch also changes the xz/unxz/dd tools to always use the host version
since the version in the source tree cannot easily be bootstrapped on macOS
or Linux.

Reviewed By:	brooks, imp (earlier version)
Differential Revision: https://reviews.freebsd.org/D25992
2020-08-25 13:29:57 +00:00
Alex Richardson
7ec1ec4fdb Add missing FreeBSD functions to -legacy when building on macOS/Linux
In most cases this simply builds the file from lib/libc for missing
functions (e.g. strlcpy on Linux etc.). In cases where this is not possible
I've added an implementation to tools/build/cross-build.

The fgetln.c/fgetwln.c/closefrom.c compatibility code was obtained from
https://gitlab.freedesktop.org/libbsd/libbsd, but I'm not sure it makes
sense to import it into to contrib just for these three bootstrap files.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D25978
2020-08-25 13:23:31 +00:00
Hans Petter Selasky
d141d1cc5f Add mlx5en(4) to the list of supported netdump network drivers.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-08-25 13:21:49 +00:00
Alex Richardson
d1d32c3dc4 Add Linux/macOS compatibility system headers to tools/build/cross-build
These headers are required in order to build the bootstrap tools on macOS
and Linux. A follow-up commit will add implementations of functions that
don't exist on those operating systems to -legacy when bootstrapping.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D14316
2020-08-25 13:18:53 +00:00
Michael Tuexen
1951fa791e RFC 3465 defines a limit L used in TCP slow start for limiting the number
of acked bytes as described in Section 2.2 of that document.
This patch ensures that this limit is not also applied in congestion
avoidance. Applying this limit also in congestion avoidance can result in
using less bandwidth than allowed.

Reported by:		l.tian.email@gmail.com
Reviewed by:		rrs, rscheff
MFC after:		3 days
Sponsored by:		Netflix, Inc.
Differential Revision:	https://reviews.freebsd.org/D26120
2020-08-25 09:42:03 +00:00
Dimitry Andric
46c8c5540f Add atomic and bswap functions to libcompiler_rt
There have been several mentions on our mailing lists about missing
atomic functions in our system libraries (e.g. __atomic_load_8 and
friends), and recently I saw __bswapdi2 and __bswapsi2 mentioned too.

To address this, add implementations for the functions from compiler-rt
to the system compiler support libraries, e.g. libcompiler_rt.a and and
libgcc_s.so.

This also needs a small fixup in compiler-rt's atomic.c, to ensure that
32-bit mips can build correctly.

Bump __FreeBSD_version to make it easier for port maintainers to detect
when these functions were added.

MFC after:	2 weeks
Differential Revision: https://reviews.freebsd.org/D26159
2020-08-25 06:49:10 +00:00
Matt Macy
b3fc6d6612 Fix libzfs build failures for some
In some environments adding zstd/include to the include path
would cause zstd's stdlib.h to hide the system one.
2020-08-25 03:43:52 +00:00
Matt Macy
13e4dbd245 Mention OpenZFS merge in UPDATING 2020-08-25 02:42:48 +00:00
Matt Macy
251779a055 Bump __FreeBSD_version for OpenZFS switchover 2020-08-25 02:22:49 +00:00
Matt Macy
9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Rick Macklem
22df1ffd81 Fix hangs with processes stuck sleeping on btalloc on i386.
r358097 introduced a problem for i386, where kernel builds will intermittently
get hung, typically with many processes sleeping on "btalloc".
I know nothing about VM, but received assistance from rlibby@ and markj@.

rlibby@ stated the following:
   It looks like the problem is that
   for systems that do not have UMA_MD_SMALL_ALLOC, we do
           uma_zone_set_allocf(vmem_bt_zone, vmem_bt_alloc);
   but we haven't set an appropriate free function.  This is probably why
   UMA_ZONE_NOFREE was originally there.  When NOFREE was removed, it was
   appropriate for systems with uma_small_alloc.

   So by default we get page_free as our free function.  That calls
   kmem_free, which calls vmem_free ... but we do our allocs with
   vmem_xalloc.  I'm not positive, but I think the problem is that in
   effect we vmem_xalloc -> vmem_free, not vmem_xfree.

   Three possible fixes:
    1: The one you tested, but this is not best for systems with
       uma_small_alloc.
    2: Pass UMA_ZONE_NOFREE conditional on UMA_MD_SMALL_ALLOC.
    3: Actually provide an appropriate vmem_bt_free function.

   I think we should just do option 2 with a comment, it's simple and it's
   what we used to do.  I'm not sure how much benefit we would see from
   option 3, but it's more work.

This patch implements #2. I haven't done a comment, since I don't know
what the problem is.

markj@ noted the following:
   I think the suggested patch is ok, but not for the reason stated.
   On platforms without a direct map the problem is:
   to allocate btags we need a slab,
   and to allocate a slab we need to map a page, and to map a page we need
   to allocate btags.

   We handle this recursion using a custom slab allocator which specifies
   M_USE_RESERVE, allowing it to dip into a reserve of free btags.
   Because the returned slab can be used to keep the reserve populated,
   this ensures that there are always enough free btags available to
   handle the recursion.

   UMA_ZONE_NOFREE ensures that we never reclaim free slabs from the zone.
   However, when it was removed, an apparent bug in UMA was exposed:
   keg_drain() ignores the reservation set by uma_zone_reserve()
   in vmem_startup().
   So under memory pressure we reclaim the free btags that are needed to
   break the recursion.
   That's why adding _NOFREE back fixes the problem: it disables the
   reclamation.

   We could perhaps fix it more cleverly, by modifying keg_drain() to always
   leave uk_reserve slabs available.

markj@'s initial patch failed testing, so committing this patch was agreed
upon as the interim solution.
Either rlibby@ or markj@ might choose to add a comment to it.

PR:		248008
Reviewed by:	rlibby, markj
2020-08-25 00:58:14 +00:00
Matt Macy
eda14cbc26 Initial import from vendor-sys branch of openzfs 2020-08-24 23:31:26 +00:00
Niclas Zeising
8d9b400f9d drm2: Update deprecation message
Update the deprecation message in the drm2 (aka legacy drm) drivers to point
towards the graphics/drm-kmod ports for all architectures, not just amd64.
drm-kmod has support for more architectures these days, and the
graphics/drm-legacy-kmod port is being deprecated.

Approved by:	imp
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D26174
2020-08-24 22:53:23 +00:00
Matt Macy
3b0ce0e28d Vendor import of openzfs master @ 184df27eef0abdc7ab2105b21257f753834b936b
Sponsored by:	iX Systems, Inc.
2020-08-24 22:48:19 +00:00
Konstantin Belousov
f1e1c4ad73 Restore workaround for sysret fault on non-canonical address after LA57.
Sponsored by:	The FreeBSD Foundation
2020-08-24 22:12:45 +00:00
Dimitry Andric
2a5220e12e After r364732, we can now enable MK_OPENMP for aarch64 by default.
PR:		248864
MFC after:	2 weeks
2020-08-24 20:40:26 +00:00
Dimitry Andric
16794618f7 Merge commit cde8f4c16 from llvm git (by me):
Move special va_list handling to kmp_os.h

  Instead of copying and pasting the same #ifdef expressions in
  multiple places, define a type and a pair of macros in kmp_os.h, to
  handle whether va_list is pointer-like or not:

  * kmp_va_list is the type to use for __kmp_fork_call()
  * kmp_va_deref() dereferences a va_list, if necessary
  * kmp_va_addr_of() takes the address of a va_list, if necessary

  Also add FreeBSD to the list of OSes that has a non pointer-like
  va_list. This can now be easily extended to other OSes too.

  Reviewed By: AndreyChurbanov

  Differential Revision: https://reviews.llvm.org/D86397

This should enable building of LLVM's OpenMP on AArch64. Addition to
share/mk will follow in a subsequent commit.

PR:		248864
MFC after:	2 weeks
2020-08-24 20:37:18 +00:00
Vincenzo Maffione
b7d6913862 netmap: use FreeBSD guards for epoch calls
EPOCH calls are FreeBSD specific. Use guards to protect these, so
that the code can compile under Linux.

MFC after:	1 week
2020-08-24 20:28:21 +00:00
Alexander V. Chernikov
592d300e34 Remove RT_LOCK mutex from rte.
rtentry lock traditionally served 2 purposed: first was protecting refcounts,
 the second was assuring consistent field access/changes.
Since route nexthop introduction, the need for the former disappeared and
 the need for the latter reduced.
To be more precise, the following rte field are mutable:

rt_nhop (nexthop pointer, updated with RIB_WLOCK, passed in rib_cmd_info)
rte_flags (only RTF_HOST and RTF_UP, where RTF_UP gets changed at rte removal)
rt_weight (relative weight, updated with RIB_WLOCK, passed in rib_cmd_info)
rt_expire (time when rte deletion is scheduled, updated with RIB_WLOCK)
rt_chain (deletion chain pointer, updated with RIB_WLOCK)
All of them are updated under RIB_WLOCK, so the only remaining concern is the reading.

rt_nhop and rt_weight (addressed in this review) are read under rib lock and
 stored in the rib_cmd_info, so the caller has no problem with consitency.
rte_flags is currently read unlocked in rtsock reporting (however the scope
 is only RTF_UP flag, which is pretty static).
rt_expire is currently read unlocked in rtsock reporting.
rt_chain accesses are safe, as this is only used at route deletion.

rt_expire and rte_flags reads will be dealt in a separate reviews soon.

Differential Revision:	https://reviews.freebsd.org/D26162
2020-08-24 20:23:34 +00:00
Warner Losh
17c511a422 Fix silly typo... 2020-08-24 20:02:13 +00:00
Warner Losh
636f6c7536 Document devd event change from r364725 2020-08-24 19:49:22 +00:00
Warner Losh
3c41ca8807 Document the kern -> kernel name change for resume events.
MFC After: 3 days
2020-08-24 19:35:27 +00:00
Warner Losh
f87655ec76 Change the resume notification event from 'kern' to 'kernel'
We have both a system of 'kern' and of 'kernel'. Prefer the latter and
convert this notification to use 'kernel' instead of 'kern'. As a
transition period, continue to also generate the 'kern' notification
until sometime after FreeBSD 13 is branched.

MFC After: 3 days
2020-08-24 19:35:15 +00:00
Mateusz Guzik
f9cdb0775e cache: remove leftover assert in vn_fullpath_any_smr
It is only valid when !slash_prefixed. For slash_prefixed the length
is properly accounted for later.

Reported by:	markj (syzkaller)
2020-08-24 18:23:58 +00:00
Cy Schubert
d709c07b34 Update unbound version number.
MFC after:	1 month
X-MFC with:	r364721
2020-08-24 18:17:13 +00:00
Cy Schubert
25039b37d3 MFV 364468:
Update unbound 1.10.1 --> 1.11.0.

MFH:		1 month
2020-08-24 18:14:04 +00:00
Cy Schubert
0a5eb308d3 MFV 364467:
Update sqlite to 3.33.0 (3330000).

Release announcement at https://www.sqlite.org/releaselog/3_33_0.html.

MFC after:	1 month
2020-08-24 18:13:44 +00:00
Fernando Apesteguía
3f8c71d1c9 w(1): Add EXAMPLES to man page
Add small example section showing general use and -d and -h flags

Approved by:	manpages (bcr@)
Differential Revision:	https://reviews.freebsd.org/D26172
2020-08-24 17:57:08 +00:00
Dimitry Andric
75b4d546cd Merge llvm, clang, compiler-rt, libc++, libunwind, lld, lldb and openmp
release/11.x llvmorg-11.0.0-rc2-0-g414f32a9e86.

MFC after:	6 weeks
X-MFC-With:	r364284
2020-08-24 17:43:23 +00:00
Dimitry Andric
bdc6feb28f Vendor import of llvm-project branch release/11.x
llvmorg-11.0.0-rc2-0-g414f32a9e86.
2020-08-24 17:20:50 +00:00
Maxim Sobolev
d065b3eb9e In the endless batch mode (-B), terminate if and when stdout is closed.
That mode is useful to call gstat from other app, however kinda useless
since gstat won't exit and stay running forever when its parent process
has long gone.

MFC after:	2 weeks
2020-08-24 16:45:23 +00:00
Warner Losh
0e533c72bc When copying over the binaries, use '-p' to preserve date/time
Although I can't reproduce it, others are seeing different lex/yacc
programs always regenerated after my change to copy rather than
symlink the files. The reported fix is to add '-p' to the copies.
Since it doesn't hurt, go head and add it, though the reasons for
this mattering remain at best obscure and poorly articulated.
2020-08-24 16:06:11 +00:00
Marc Fonvieille
e0e236ba88 s/redundacy/redundancy
MFC after:	1 week
2020-08-24 14:04:59 +00:00
Marc Fonvieille
633963a352 Add missing Korean doc package entry, remove non-existent Serbian doc
package entry.

Approved by:	re (gjb)
MFC after:	1 week
2020-08-24 14:00:12 +00:00
Leandro Lupori
fb27a570ad [PowerPC] Make new auxv format default
Assume ELF images without OSREL use the new auxv format.

This is specially important for rtld, that is not tagged. Using
direct exec mode with new (ELFv2) binaries that expect the new auxv
format would result in crashes otherwise.

Unfortunately, this may break direct exec'ing old binaries,
but it seems better to correctly support new binaries by default,
considering the transition to ELFv2 happened quite some time
ago. If needed, a sysctl may be added to allow old auxv format to
be used when OSREL is not found.

Reviewed by:	bdragon
Sponsored by:	Eldorado Research Institute (eldorado.org.br)
Differential Revision:	https://reviews.freebsd.org/D25651
2020-08-24 13:40:35 +00:00
Bjoern A. Zeeb
2b9f12f6b2 net80211: enhance getflags*() and ieee80211_add_channel*()
For ieee80211_add_channel+*() we are passing in an int flag for
ht40 and in some cases another int flag for vht80 where we'd only
need two bits really.
Convert these variables to a bitflag and fold them together into one.
This also allows for VHT160 and VHT80P80 and whatever may come to
be considered. Define the various options currently needed.

Change the drivers (rtwn and rsu) which actually set this bit to non-0.
For convenience the "1" currently used for HT40 is preserved.

Enahnce getflags_5ghz() to handle the full set of VHT flags based
on the input flags from the the driver.

Update the regdomain implementation as well to make use of the new
flags and deal with higher [V]HT bandwidths.

ieee80211_add_channel() specifically did not take flags so it will
not support naything beyond 20Mhz channels.

Note: I am not entirely happy with the "cbw_flag[s]" name, but we
do use chan_flags elsewhere already.

MFC after:	2 weeks
Reviewed by:	adrian, gnn
Sponsored by:	Rubicon Communications, LLC (d/b/a "Netgate")
Differential revision:	https://reviews.freebsd.org/D26091
2020-08-24 13:15:08 +00:00
Peter Grehan
a333a508a2 cpu_auxmsr: assert caller is preventing CPU migration.
Submitted by:	Adam Fenn (adam at fenn dot io)
Requested by:	kib
Reviewed by:	kib, grehan
Approved by:	kib
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D26166
2020-08-24 11:49:49 +00:00