Commit Graph

246102 Commits

Author SHA1 Message Date
Andriy Gapon
8491540808 MFV r354383: 10592 misc. metaslab and vdev related ZoL bug fixes
illumos/illumos-gate@555d674d5d
555d674d5d

https://www.illumos.org/issues/10592
  This is a collection of recent fixes from ZoL:
  8eef997679 Error path in metaslab_load_impl() forgets to drop ms_sync_lock
  928e8ad47d Introduce auxiliary metaslab histograms
  425d3237ee Get rid of space_map_update() for ms_synced_length
  6c926f426a Simplify log vdev removal code
  21e7cf5da8 zdb -L should skip leak detection altogether
  df72b8bebe Rename range_tree_verify to range_tree_verify_not_present
  75058f3303 Remove unused vdev_t fields

Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Author: Serapheim Dimitropoulos <serapheim@delphix.com>
MFC after:	4 weeks
2019-11-21 13:35:43 +00:00
Andrew Turner
68cad68149 Add kcsan_md_unsupported from NetBSD.
It's used to ignore virtual addresses that may have a different physical
address depending on the CPU.

Sponsored by:	DARPA, AFRL
2019-11-21 13:22:23 +00:00
Andrew Turner
bba0065f0d Fix the bus_space functions with KCSAN on arm64.
Arm64 doesn't define the bus_space_set_multi_stream and
bus_space_set_region_stream functions. Don't try to define them there.

Sponsored by:	DARPA, AFRL
2019-11-21 13:12:58 +00:00
Andrew Turner
1b8c58f283 Fix for style(9): use parentheses around return statements.
Reported by:	kib
Sponsored by:	DARPA, AFRL
2019-11-21 12:29:20 +00:00
Andrew Turner
849aef496d Port the NetBSD KCSAN runtime to FreeBSD.
Update the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime to work in
the FreeBSD kernel. It is a useful tool for finding data races between
threads executing on different CPUs.

This can be enabled by enabling KCSAN in the kernel config, or by using the
GENERIC-KCSAN amd64 kernel. It works on amd64 and arm64, however the later
needs a compiler change to allow -fsanitize=thread that KCSAN uses.

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22315
2019-11-21 11:22:08 +00:00
Andriy Gapon
489912da7b MFV r354382,r354385: 10601 10757 Pool allocation classes
illumos/illumos-gate@663207adb1
663207adb1

10601 Pool allocation classes
https://www.illumos.org/issues/10601
  illumos port of ZoL Pool allocation classes. Includes at least these two
  commits:
  441709695 Pool allocation classes misplacing small file blocks
  cc99f275a Pool allocation classes

10757 Add -gLp to zpool subcommands for alt vdev names
https://www.illumos.org/issues/10757
  Port from ZoL of
  d2f3e292d Add -gLp to zpool subcommands for alt vdev names
  Note that a subsequent ZoL commit changed -p to -P
  a77f29f93 Change full path subcommand flag from -p to -P

Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Håkan Johansson <f96hajo@chalmers.se>
Portions contributed by: Richard Yao <ryao@gentoo.org>
Portions contributed by: Chunwei Chen <david.chen@nutanix.com>
Portions contributed by: loli10K <ezomori.nozomu@gmail.com>
Author: Don Brady <don.brady@delphix.com>

11541 allocation_classes feature must be enabled to add log device

illumos/illumos-gate@c1064fd7ce
c1064fd7ce

https://www.illumos.org/issues/11541
  After the allocation_classes feature was integrated, one can no longer add a
  log device to a pool unless that feature is enabled. There is an explicit check
  for this, but it is unnecessary in the case of log devices, so we should handle
  this better instead of forcing the feature to be enabled.

Author: Jerry Jelinek <jerry.jelinek@joyent.com>

FreeBSD notes.
I faithfully added the new -g, -L, -P flags, but only -g does something:
vdev GUIDs are displayed instead of device names.  -L, resolve symlinks,
and -P, display full disk paths, do nothing at the moment.
The use of special vdevs is backward compatible for read-only access, so
root pools should be bootable, but exercise caution.

MFC after:	4 weeks
2019-11-21 08:20:05 +00:00
Warner Losh
c81e4a699b Include MACHINE in the which variable selection to use section. 2019-11-21 06:48:30 +00:00
Warner Losh
7b692b8e0f Add table for MACHINE_CPUARCH
Add table and also some additional verbage of which one to use.
2019-11-21 06:44:19 +00:00
Glen Barber
fb1c294d47 Revert r354896, r354899, r354900:
Fix build.

Sponsored by:	Rubicon Communications, LLC (netgate.com)
2019-11-21 04:12:08 +00:00
Ed Maste
21ad8c5eb0 mark arm.arm (v4/v5) kernels as NO_UNIVERSE for now
r354290 removed arm.arm from universe, but arm.arm kernels were still
found and built during the kernel stage.  I'm not aware of a better way
to address this at the moment, but since there aren't many arm.arm
kernels anyhow just add an explicit NO_UNIVERSE to them.

Reported by:	rpokala
2019-11-21 03:10:02 +00:00
Kyle Evans
a8761a2a55 bcm2835_sdhci: clean up DMA segments in error handling path
Later parts assume that this would've been done if interrupts are enabled,
but this is the only case in which that wouldn't have been true. This commit
also reorders operations such that we're done touching slot/slot->intmask
before we call back into the SDHCI framework and exit.
2019-11-21 02:49:41 +00:00
Kyle Evans
b61ac06706 bcm2835_sdhci: roll back r354823
r354823 kicked DATA_END handling out of the DMA interrupt path "to make
things easy", but this was likely a mistake -- if we know we're done after
we've finished pending DMA operations, we should go ahead and acknowledge
it rather than waiting for the controller to finalize it. If it's not ready,
we'll simply re-enable interrupts and wait for it anyways, to be re-entered
in sdhci_data_intr.
2019-11-21 02:47:55 +00:00
Kyle Evans
e68976d877 Revert r354930: wrong diff, right message. 2019-11-21 02:44:05 +00:00
Kyle Evans
7af945c666 bcm2835_sdhci: clean up DMA segments in error handling path
Later parts assume that this would've been done if interrupts are enabled,
but this is the only case in which that wouldn't have been true. This commit
also reorders operations such that we're done touching slot/slot->intmask
before we call back into the SDHCI framework and exit.
2019-11-21 02:41:22 +00:00
Gleb Smirnoff
71f0077631 Remove sio(4).
It had been disconnected from build in r181233 in 2008.

Reviewed by:	imp
2019-11-21 01:24:49 +00:00
Mark Johnston
e7485a436b Avoid relying on pollution from libcasper.h.
Reported by:	sjg
Sponsored by:	The FreeBSD Foundation
2019-11-21 00:17:14 +00:00
Warner Losh
1cdb8eb8fe Add --esp/-E argument to print the currently booted ESP
Add code to decode the BootCurrent and BootXXXX variable it points at
to deduce the ESP used to boot the system. By default, it prints the
path to that device. With --unix-path (-p) it will instead print the
current mount point for the ESP, if any (or an error). With
--device-path (-d) it wil print the UEFI device path for the ESP.

Note: This is the best guess based on the UEFI variables. If the ESP
is part of a gmirror, etc, that won't be reported. If by some weird
chance there was a complicated series of chain boots, this may not be
what you want. For setups that don't add layers on top of the raw
devices, it is accurate.

Differential Revision: https://reviews.freebsd.org/D22432
2019-11-20 23:58:36 +00:00
Pedro F. Giffuni
9af6c78cd4 MFV 354917, 354918, 354919
openresolv: update to version 3.9.2

MFC after:	3 weeks
2019-11-20 23:56:20 +00:00
Warner Losh
6467506baf Create /etc/os-release file.
Each boot, regenerate /var/run/os-release based on the currently running
system. Create a /etc/os-release symlink pointing to this file (so that this
doesn't create a new reason /etc can not be mounted read-only).

This is compatible with what other systems do and is what the sysutil/os-release
port attempted to do, but in an incomplete way. Linux, Solaris and DragonFly all
implement this natively as well. The complete standard can be found at
https://www.freedesktop.org/software/systemd/man/os-release.html

Moving this to the base solves both the non-standard location problem with the
port, as well as the lack of update of this file on system update.

Bump __FreeBSD_version to 1300060

PR: 238953
Differential Revision:  https://reviews.freebsd.org/D22271
2019-11-20 23:45:31 +00:00
Pedro F. Giffuni
2d823aa1c7 Import openresolv 3.9.0
More information at: https://roy.marples.name/projects/openresolv
2019-11-20 22:13:14 +00:00
Warner Losh
599d032a6d Standardize EFI's ESP mount point.
Mount the UEFI ESP on /boot/efi. No current system uses this by default, but
there are many ad-hoc schemes that do this in /efi or /esp or /uefi and adding a
new directory at the top-level would have a much higher likelihood of
collision. Document this in /etc/mtree/BSD.root.mtree and create EFIDIR and
related variables in bsd.own.mk.

Differential Revision: https://reviews.freebsd.org/D21344
2019-11-20 21:06:29 +00:00
Alexander Motin
cc453b2272 Set handling for some "Logical unit not ready" errors.
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-11-20 20:00:03 +00:00
Conrad Meyer
c41faf5591 random/ivy: Trivial refactoring
It is clearer to me to return success/error (true/false) instead of some
retry count linked to the inline assembly implementation.

No functional change.

Approved by:	core(csprng) => csprng(markm)
Differential Revision:	https://reviews.freebsd.org/D22454
2019-11-20 19:55:43 +00:00
Conrad Meyer
61287be181 Re-apply fixed r354847
unifdef(1): Improve worst-case bound on symbol resolution

Use RB_TREE to make some algorithms O(lg N) and O(N lg N) instead of O(N)
and O(N^2).

While here, remove arbitrarily limit on number of macros understood.

Reverts r354877 and r354878, which disabled the (correct) test.

PR:		242095
Reported by:	lwhsu
2019-11-20 19:43:34 +00:00
Conrad Meyer
0877992e6b Revert r354847 for now
It was broken.

PR:		242095
Reported by:	lwhsu
2019-11-20 19:07:22 +00:00
Dimitry Andric
1d91c857d7 Fix typo: deprected -> deprecated. 2019-11-20 19:07:08 +00:00
Brooks Davis
303d2dd8f1 Make the warning for deprecated NO_ variables an error.
Support for NO_CTF, NO_DEBUG_FILES, NO_INSTALLLIB, NO_MAN, NO_PROFILE,
and NO_WARNS as deprecated in 2014 with a warning added for each one
found. Turn these into error in preperation for removal of compatability
support before FreeBSD 13.

Reviewed by:	imp
Relnotes:	yes
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D22448
2019-11-20 18:36:58 +00:00
Dimitry Andric
a1aaa66be8 Add explanatory comments for the different SRCS_xxx variables used in
the Makefiles for libllvm and libclang.  While here, cleanup a commented
out SRCS entry in libllvmminimal's Makefile.

MFC after:	3 days
2019-11-20 18:12:01 +00:00
Andrew Turner
09a65f9ff5 As with r354905 use uint16_t to store aflags on the stack and as function
arguments as the aflags size in vm_page_t has increased.

Sponsored by:	DARPA, AFRL
2019-11-20 18:00:43 +00:00
Ed Maste
f3cdc62295 src.conf.5: regen for several recent changes
r354289 armv6: Switch to LLD by default
r354290 Take arm.arm (armv5) out of universe
r354348 armv6, armv7: Switch to llvm-libunwind by default
r354660 Enable the RISC-V LLVM backend by default.

as well as lib32 changes

Sponsored by:	The FreeBSD Foundation
2019-11-20 17:57:46 +00:00
Andrew Turner
ad216bc10d Use atomic_load_16 to load aflags as it's a uint16_t after r354820.
Sponsored by:	DARPA, AFRL
2019-11-20 17:49:58 +00:00
Ed Maste
63b149ff63 src.conf.5: regen after r354902, WITHOUT_AMD by default 2019-11-20 17:49:32 +00:00
Ed Maste
9bf64d023d Add description for WITH_AMD
WITHOUT_AMD is now the default as of r354902.

Sponsored by:	The FreeBSD Foundation
2019-11-20 17:45:31 +00:00
Ed Maste
5882cf7159 disable amd(8) by default
As of FreeBSD 10.1 the autofs(5) is available for automounting, and the
amd man page has indicated that the in-tree copy of amd is obsolete.
Disable it by default for now, with the expectation that it will be
removed before FreeBSD 13.0.

Reviewed by:	kevans
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22460
2019-11-20 17:37:45 +00:00
Alex Richardson
52918a7b1b Allow boostrapping xinstall on Linux
Linux does not have st_flags so we have to avoid using it there.

Reviewed By:	emaste, imp
Differential Revision: https://reviews.freebsd.org/D22446
2019-11-20 17:24:49 +00:00
Li-Wen Hsu
7cef5cac04 Use the correct variable, also limit the scope to bfd
PR:		242109
Reported by:	jhb
Sponsored by:	The FreeBSD Foundation
2019-11-20 16:54:21 +00:00
Li-Wen Hsu
f10ddee841 Limit the workaround to riscv only
PR:		242109
Sponsored by:	The FreeBSD Foundation
2019-11-20 16:35:58 +00:00
Alan Cox
48c22f0eb2 Until every possible root cause for an "invalid ASID" assertion failure is
resolved, assign every pmap a valid ASID when it is first initialized.
2019-11-20 16:32:13 +00:00
Ed Maste
e8c56fba29 sshd: make getpwclass wrapper MON_ISAUTH not MON_AUTH
In r339216 a privsep wrapper was added for login_getpwclass to address
PR 231172.  Unfortunately the change used the MON_AUTH flag in the
wrapper, and MON_AUTH includes MON_AUTHDECIDE which triggers an
auth_log() on each invocation.  getpwclass() does not participate in the
authentication decision, so should be MON_ISAUTH instead.

PR:		234793
Submitted by:	Henry Hu
Reviewed by:	Yuichiro NAITO
MFC after:	1 week
2019-11-20 16:30:37 +00:00
Li-Wen Hsu
53c772bbd7 Workaround riscv64 build when using binutils 2.33.1
PR:		242109
Reviewed by:	bapt
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22457
2019-11-20 16:20:49 +00:00
Doug Moore
83704cc236 Instead of looking up a predecessor or successor to the current map
entry, when that entry has been seen already, keep the
already-looked-up value in a variable and use that instead of looking
it up again.

Approved by: alc, markj (earlier version), kib (earlier version)
Differential Revision: https://reviews.freebsd.org/D22348
2019-11-20 16:06:48 +00:00
Andrew Turner
0cb5357037 Import the NetBSD Kernel Concurrency Sanitizer (KCSAN) runtime.
KCSAN is a tool to find concurrent memory access that may race each other.
After a determined number of memory accesses a cell is created, this
describes the current access. It will then delay for a short period
to allow other CPUs a chance to race. If another CPU performs a memory
access to an overlapping region during this delay the race is reported.

This is a straight import of the NetBSD code, it will be adapted to
FreeBSD in a future commit.

Sponsored by:	DARPA, AFRL
2019-11-20 14:37:48 +00:00
Mateusz Guzik
d578a4256e cache: minor stat cleanup
Remove duplicated stats and move numcachehv from debug to vfs.cache.
2019-11-20 12:08:32 +00:00
Mateusz Guzik
d957f3a4f0 vfs: perform a more racy check in vfs_notify_upper
Locking mp does not buy anything interms of correctness and only contributes to
contention.
2019-11-20 12:07:54 +00:00
Mateusz Guzik
4de1818baf linux: avoid overhead of P_CONTROLT checks if possible
Sponsored by:	The FreeBSD Foundation
2019-11-20 12:06:29 +00:00
Mateusz Guzik
1fccb43c39 vfs: change si_usecount management to count used vnodes
Currently si_usecount is effectively a sum of usecounts from all associated
vnodes. This is maintained by special-casing for VCHR every time usecount is
modified. Apart from complicating the code a little bit, it has a scalability
impact since it forces a read from a cacheline shared with said count.

There are no consumers of the feature in the ports tree. In head there are only
2: revoke and devfs_close. Both can get away with a weaker requirement than the
exact usecount, namely just the count of active vnodes. Changing the meaning to
the latter means we only need to modify it on 0<->1 transitions, avoiding the
check plenty of times (and entirely in something like vrefact).

Reviewed by:	kib, jeff
Tested by:	pho
Differential Revision:	https://reviews.freebsd.org/D22202
2019-11-20 12:05:59 +00:00
Konstantin Belousov
da248a69aa amd64: in double fault handler, do not rely on sane gsbase value.
Typical reasons for doublefault faults are either kernel stack
overflow or bugs in the code that manipulates protection CPU state.
The later code is the code which often has to set up gsbase for
kernel.  Switching to explicit load of GSBASE MSR in the fault handler
makes it more probable to output a useful information.

Now all IST handlers have nmi_pcpu structure on top of their stacks.

It would be even more useful to save gsbase value at the moment of the
fault.  I did not this because I do not want to modify PCB layout now.

Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-11-20 11:12:19 +00:00
Andriy Gapon
a4619d8461 zpool.8: remove a paragraph about quorum disks
FreeBSD has no such thing.
illumos and ZoL manuals do not talk about quorum disks either.
Only Oracle ZFS mentions them.

MFC after:	1 week
2019-11-20 08:56:01 +00:00
Andriy Gapon
54d1762459 fix up r354804, resolve merge conflicts in zpool.8
Somehow I managed to commit the manual page with unresolved conflicts in
it.

While here, I also replaced .sp with .Pp.

MFC after:	3 weeks
X-MFC with:	r354804
2019-11-20 08:49:13 +00:00
Li-Wen Hsu
4ddce76706 Only skip failing test case in CI.
PR:		242095
Sponsored by:	The FreeBSD Foundation
2019-11-20 05:34:02 +00:00