Commit Graph

244802 Commits

Author SHA1 Message Date
Alan Somers
bbacbaccf9 ZFS: mark hotspare_scrub_002_pos as an expected failure
"zpool scrub" doesn't detect all errors on active spares in raidz arrays

PR:		241069
MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 20:19:05 +00:00
Alan Somers
87015aac27 ZFS: fix the delegate tests
These tests have never worked correctly

* Replace runwattr with sudo
* Fix a scoping bug with the "dtst" variable
* Cleanup user properties created during tests
* Eliminate the checks for refreservation and send support. They will always
  be supported.
* Fix verify_fs_snapshot. It seemed to assume that permissions would not yet
  be delegated, but that's not how it's actually used.
* Combine verify_fs_promote with verify_vol_promote
* Remove some useless sleeps
* Fix backwards condition in verify_vol_volsize
* Remove some redundant cleanup steps in the tests. cleanup.ksh will handle
  everything.
* Disable some parts of the tests that FreeBSD doesn't support:
    * Creating snapshots with mkdir
    * devices
    * shareisci
    * sharenfs
    * xattr
    * zoned

The sharenfs parts could probably be reenabled with more work to remove the
Solarisms.

MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D21898
2019-10-07 20:13:49 +00:00
Alan Somers
063f9d1295 zfs: skip the zfsd tests if zfsd is not running
MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D21878
2019-10-07 19:50:22 +00:00
Alan Somers
95becc8898 zfs: fix the zdb_001_neg test
The test needed to be updated for r331701 (MFV illumos 8671400), which added
a "-k" option.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 19:48:18 +00:00
Alan Somers
f8069eab4c ZFS: fix the zpool_get_002_pos test
ZFS has grown some additional properties that hadn't been added to the
config file yet.  While I'm here, improve the error message, and remove a
superfluous command.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 19:24:50 +00:00
Edward Tomasz Napierala
1a13f2e6b4 Introduce stats(3), a flexible statistics gathering API.
This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof).  Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs.  The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by:	sef (earlier version)
Obtained from:	Netflix
Relnotes:	yes
Sponsored by:	Klara Inc, Netflix
Differential Revision:	https://reviews.freebsd.org/D20477
2019-10-07 19:05:05 +00:00
Alan Somers
d253c454bb zfs: fix the slog_012_neg test
This test attempts to corrupt a file-backed vdev by deleting it and then
recreating it with truncate.  But that doesn't work, because the pool
already has the vdev open, and it happily hangs on to the open-but-deleted
file.  Fix by truncating the file without deleting it.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 18:55:40 +00:00
Alan Somers
ba23dca684 ZFS: fix several zvol_misc tests
* Adapt zvol_misc_001_neg to use dumpon instead of Solaris's dumpadm
* Disable zvol_misc_003_neg, zvol_misc_005_neg, and zvol_misc_006_pos,
  because they involve using a zvol as a dump device, which FreeBSD does not
  yet support.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 18:21:33 +00:00
Hans Petter Selasky
4715738b12 Compile time assert a valid subsystem for all VNET init and uninit functions.
Using VNET init and uninit functions outside the given range has undefined
behaviour.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-07 14:24:59 +00:00
Hans Petter Selasky
204e2f30d9 Factor out VNET shutdown check into an own vnet structure field.
Remove the now obsolete vnet_state field. This greatly simplifies the
detection of VNET shutdown and avoids code duplication.

Discussed with:	bz@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-07 14:15:41 +00:00
Hans Petter Selasky
ac66be4122 Make control endpoint quirk for xhci(4) configurable.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-07 13:40:29 +00:00
Andriy Gapon
3251c5ae51 fix up r353168, add atomic_swap_64 to i386 version of opensolaris_atomic.S
The compatibility code for the atomic operations in ZFS code is a bit
messy.  In some cases the native definitions are directly made
available, in some cases there are emulated operations in
opensolaris_atomic.c and in yet other cases there are atomic operations
implemented in assembly that were obtained from OpenSolaris / illumos.

This commit adds atomic_swap_64 for use with i386 userland.
The code is copied from illumos.

I am not sure why FreeBSD does not provide that operation natively.
Maybe because we try (or pretend) to support processors that did not
have the necessary instructions.

While here I also added atomic_load_64 for the same reasons.
This is original code based on iilumos atomic_swap_64 and FreeBSD
atomic_load_acq_64_i586.

Pointyhat to:	avg
MFC after:	1 week
2019-10-07 12:53:27 +00:00
Andriy Gapon
862c20fd89 MFV r350898, r351075: 8423 8199 7432 Implement large_dnode pool feature
8423 8199 7432 Implement large_dnode pool feature

7432 Large dnode pool feature
8199 multi-threaded dmu_object_alloc()
8423 Implement large_dnode pool feature
10406 large_dnode changes broke zfs recv of legacy stream

llumos/illumos-gate@54811da5ac
54811da5ac
https://www.illumos.org/issues/8423
https://www.illumos.org/issues/8199
https://www.illumos.org/issues/7432

illumos/illumos-gate@811964cd9f
811964cd9f
https://www.illumos.org/issues/10406

  ZoL issues:
  Improved dnode allocation #6564
  Clean up large dnode code #6262
  Fix dnode_hold() freeing dnode behavior #8172
  Fix dnode allocation race #6414, #6439
  Partial: Raw sends must be able to decrease nlevels #6821, #6864
  Remove unnecessary txg syncs from receive_object() Closes #7197

This updates FreeBSD large_dnode code (that was imported from ZoL) to a
version that was committed to illumos.  It has some cleanups,
improvements and fixes comparing to what we have in FreeBSD now.
I think that the most significant update is 8199 multi-threaded
dmu_object_alloc().

This commit reverts r351077 that was a revert of r351074 and r351076 and
restores those changes.  Required atomic operations should be available
now on all platforms where we build ZFS.

Obtained from:	illumos
MFC after:	3 weeks
2019-10-07 08:14:45 +00:00
Emmanuel Vadot
0a457350fd arm: dts: ti: Fix mmc3 instance by setting it to disabled
DTS Import of Linux 5.3 added a patch that rework the L3 mmc instance
in the AM335x SoC but removed the status = 'disabled' on the node.
This cause the kernel to probe the device even if the board doesn't
have this mmc used and since we don't correctly activate the clock
for this module we panic with an external data abort.
Beaglebone(s) don't have this device anyway so simply disabling it.
Patch for the DTS was sent upstream.
https://patchwork.kernel.org/patch/11176921/

PR:		241089
Reported by:	phk
2019-10-07 08:11:49 +00:00
Andriy Gapon
f1fd3654e7 ZFS: unconditionally use atomic_swap_64
Previously, the code used a plain store on platforms that lacked
atomic_swap_64 and possibly some other platforms as the condition worked
only if atomic_swap_64 was a macro.

MFC after:	1 week
X-MFC after:	r353166, r353167
2019-10-07 08:00:54 +00:00
Andriy Gapon
cf78c4beae ZFS: add emulation of atomic_swap_64 and atomic_load_64
Some 32-bit platforms do not provide 64-bit atomic operations that ZFS
requires, either in userland or at all.  We emulate those operations for
those platforms using a mutex.  That is not entirely correct and it's
very efficient.  Besides, the loads are plain loads, so torn values are
possible.

Nevertheless, the emulation seems to work for some definition of work.

This change adds atomic_swap_64, which is already used in ZFS code, and
atomic_load_64 that can be used to prevent torn reads.

MFC after:	1 week
2019-10-07 07:54:34 +00:00
Andriy Gapon
eab7984cfe add atomic_load_64 for mipsn32
It's just an alias for atomic_load_acq_64 (same as on i386).

MFC after:	1 week
2019-10-07 07:42:26 +00:00
Andriy Gapon
80475f9523 align use of cp15_pmccntr_get with its availability
According to ian, the only armv6 cpu we support is the 1176, so this
change is effectively a no-op.
The change is just to make the code more self-consistent.
The issue was noticed by a standalone module build for armv6.

Reviewed by:	ian
MFC after:	3 weeks
2019-10-07 07:37:42 +00:00
Alan Cox
bc285d6a8f Eliminate an unused declaration. The variable in question is only defined
and used on sparc64.

MFC after:	1 week
2019-10-07 04:22:03 +00:00
Alan Cox
e227f09282 Eliminate a redundant bzero(). The l0 page table page was already zeroed
by efi_1t1_page().

MFC after:	1 week
2019-10-07 03:37:28 +00:00
Kyle Evans
605261992c Revert r352557: powerpc/loader: Install ubldr without stripping
This was committed due to what was later diagnosed as an msdosfs bug
preventing in-place strip. This bug was fixed in r352564, and we agreed to
keep the workaround in for a bit to allow the driver fix a suitable amount
of propagation time for folks building/installing powerpc/ubldr, seeing as
how we were not in any hurry to revert.
2019-10-07 03:28:11 +00:00
Justin Hibbits
cd51f7df9c loader/powerpc64: Fix HV check for CAS usage
Logic was backwards.  The function returns true if it *is* running as a
hypervisor, whereas we want to only call the CAS utility if we're running as a
guest.

Reported by:	Shawn Anastasio <shawn@anastas.io>
2019-10-07 03:05:32 +00:00
Justin Hibbits
02e7952133 powerpc64/pmap: Fix release order to match lock order in moea64_enter()
Page PV lock is always taken first, so should be released last.  This also
(trivially) shortens the hold time of the pmap lock.

Submitted by:	mjg
2019-10-07 02:36:42 +00:00
Randall Stewart
5b63b22075 Brad Davis identified a problem with the new LRO code, VLAN's
no longer worked. The problem was that the defines used the
same space as the VLAN id. This commit does three things.
1) Move the LRO used fields to the PH_per fields. This is
   safe since the entire PH_per is used for IP reassembly
   which LRO code will not hit.
2) Remove old unused pace fields that are not used in mbuf.h
3) The VLAN processing is not in the mbuf queueing code. Consequently
   if a VLAN submits to Rack or BBR we need to bypass the mbuf queueing
   for now until rack_bbr_common is updated to handle the VLAN properly.

Reported by:	Brad Davis
2019-10-06 22:29:02 +00:00
Mateusz Guzik
e35cd9e38f ufs: add root vnode caching
See r353150.

Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D21646
2019-10-06 22:18:03 +00:00
Mateusz Guzik
d511f93e45 nfsclient: add root vnode caching
See r353150.

Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D21646
2019-10-06 22:17:29 +00:00
Mateusz Guzik
7682d0be2b tmpfs: add root vnode caching
See r353150.

Sponsored by:   The FreeBSD Foundation
Differential Revision:  https://reviews.freebsd.org/D21646
2019-10-06 22:17:11 +00:00
Mateusz Guzik
559ac49d41 devfs: add root vnode caching
See r353150.

Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21646
2019-10-06 22:16:55 +00:00
Mateusz Guzik
818631b634 zfs: add root vnode caching
This replaces the approach added in r338927.

See r353150.

Sponsored by:	The FreeBSD Foundation
2019-10-06 22:16:00 +00:00
Mateusz Guzik
dc20b834ca vfs: add optional root vnode caching
Root vnodes looekd up all the time, e.g. when crossing a mount point.
Currently used routines always perform a costly lookup which can be
trivially avoided.

Reviewed by:	jeff (previous version), kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21646
2019-10-06 22:14:32 +00:00
Mateusz Guzik
6114fc8b85 amd64 pmap: implement per-superpage locks
The current 256-lock sized array is a problem in the following ways:
- it's way too small
- there are 2 locks per cacheline
- it is not NUMA-aware

Solve these issues by introducing per-superpage locks backed by pages
allocated from respective domains.

This significantly reduces contention e.g. during poudriere -j 104.
See the review for results.

Reviewed by:	kib
Discussed with:	jeff
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21833
2019-10-06 22:13:35 +00:00
Justin Hibbits
d137ff5521 powerpc/pmap64: Properly parenthesize PV_LOCK_COUNT macros
As pointed out by mjg, without the parentheses the calculations done against
these macros are incorrect, resulting in only 1/3 of locks being used.

Reported by:	mjg
2019-10-06 19:11:01 +00:00
Toomas Soome
03c9cdf71d loader.efi: for text mode, use STM to scroll the whole screen
Since local UEFI console is implemented on top of framebuffer,
we need to avoid redrawing the whole screen ourselves, but let
Simple Text Mode to do the scroll for us.
2019-10-06 18:38:58 +00:00
Michael Tuexen
63fb39ba7b Plumb an mbuf leak in a code path that should not be taken. Also avoid
that this path is taken by setting the tail pointer correctly.
There is still bug related to handling unordered unfragmented messages
which were delayed in deferred handling.
This issue was found by OSS-Fuzz testing the usrsctp stack and reported in
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=17794

MFC after:		3 days
2019-10-06 08:47:10 +00:00
Kyle Evans
036d2e814b Re-add ALLOW_MIPS_SHARED_TEXTREL, sprinkle it around
Diff partially stolen from CheriBSD; these bits need -Wl,-z,notext in order
to build in an LLVM world. They are needed for all flavors/sizes of MIPS.
This will eventually get fixed in LLVM, but it's unclear when.

Reported by:	arichardson, emaste
Differential Revision:	https://reviews.freebsd.org/D21696
2019-10-06 04:19:49 +00:00
Kyle Evans
29a5f63951 riscv: use the common sub-word {,f}cmpset implementation
Reviewed by:	mhorne
Differential Revision:	https://reviews.freebsd.org/D21888
2019-10-06 01:35:31 +00:00
Yuri Pankov
f01edb6f5d Mark "private use area" characters as printable.
At least some of the characters in E000-F8FF range are used by Powerline
fonts, and having no attributes for these ranges in UnicodeData.txt
other than "Other, Private Use" it should be safe to mark all of them as
printable.  Some actually were before r340491, so this fixes the
regression introduced there as well.

PR:		240911
Reviewed by:	bapt
Tested by:	Daniel Ponte <amigan@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D21850
2019-10-05 22:17:54 +00:00
Kyle Evans
e3f35d562f Remove the remnants of SI_CHEAPCLONE
SI_CHEAPCLONE was introduced in r66067 for use with cloned bpfs. It was
later also used in tty, tun, tap at points. The rough timeline for being
removed in each of these is as follows:

- r181690: bpf switched to use cdevpriv API by ed@
- r181905: ed@ rewrote the TTY later to be mpsafe
- r204464: kib@ removes it from tun/tap, declaring it unused

I've not yet been able to dig up any other consumers in the intervening 9
years. It is no longer set on any devices in the tree and leaves an
interesting situation in make_dev_sv where we're ok with the device already
being set SI_NAMED.
2019-10-05 21:52:06 +00:00
Kyle Evans
d42fecb5c1 kern_conf: fully initialize cloned devices with make_dev_args, too
Attempting to initialize si_drv{1,2} with mda_si_drv{1,2} does not work if
you are operating on cloned devices.

clone_create must be called prior to the make_dev* family to create/return
the device on the clonelist as needed. This device is later returned early
in newdev(), prior to si_drv{0,1,2} initialization.

This patch simply breaks out of the loop if we've found a device and
finishes init.

Reviewed by:	kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21904
2019-10-05 21:44:18 +00:00
Yuri Pankov
b07934b7c0 Pre-generate Big5 charmap from CLDR data.
The one used previously was missing the characters in 0-127 range,
making various tools try to escape them in output.

PR:		235100
Reviewed by:	bapt
Tested by:	Ting-Wei Lan <lantw44@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D21794
2019-10-05 21:28:46 +00:00
Mateusz Guzik
dfa8dae493 devfs: plug redundant bwillwrite avoidance
vn_write already checks for vnode type to see if bwillwrite should be called.

This effectively reverts r244643.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21905
2019-10-05 17:44:33 +00:00
Emmanuel Vadot
c36e2d14ac arm64: rockchip: usb2phy: Add set/get mode
We only support host mode so those functions are just added so
we won't panic when generic-{e,o}hci will set the phy to host mode.

MFC after:	1 month
X-MFC-With:	r353062
2019-10-05 17:36:33 +00:00
Michael Tuexen
2560cb1eb8 Fix a use after free bug when removing remote addresses.
This bug was found by OSS-Fuzz and reported in
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18004

MFC after:		3 days
2019-10-05 13:28:01 +00:00
Michael Tuexen
0941b9dc37 Plumb an mbuf leak found by Mark Wodrich from Google by fuzz testing the
userland stack and reporting it in:
https://github.com/sctplab/usrsctp/issues/396

MFC after:		3 days
2019-10-05 12:34:50 +00:00
Jens Schweikhardt
6777f10c48 Correct grammos and typos. 2019-10-05 09:59:00 +00:00
Michael Tuexen
44f788d793 Fix the adding of padding to COOKIE-ECHO chunks.
Thanks to Mark Wodrich who found this issue while fuzz testing the
usrsctp stack and reported the issue in
https://github.com/sctplab/usrsctp/issues/382

MFC after:		3 days
2019-10-05 09:46:11 +00:00
Alan Somers
4330d7272d ZFS: fix several of the "zpool create" tests
* Remove zpool_create_013_neg.  FreeBSD doesn't have an equivalent of
  Solaris's metadevices.  GEOM would be the equivalent, but since all geoms
  are the same from ZFS's perspective, this test would be redundant with
  zpool_create_012_neg

* Remove zpool_create_014_neg.  FreeBSD does not support swapping to regular
  files.

* Remove zpool_create_016_pos.  This test is redundant with literally every
  other test that creates a disk-backed pool.

* s:/etc/vfstab:/etc/fstab in zpool_create_011_neg

* Delete the VTOC-related portion of zpool_create_008_pos.  FreeBSD doesn't
  use VTOC.

* Replace dumpadm with dumpon and swap with swapon in multiple tests.

* In zpool_create_015_neg, don't require "zpool create -n" to fail.  It's
  reasonable for that variant to succeed, because it doesn't actually open
  the zvol.

* Greatly simplify zpool_create_012_neg.  Make it safer, too, but not
  interfering with the system's regular swap devices.

* Expect zpool_create_011_neg to fail (PR 241070)

* Delete some redundant cleanup steps in various tests

* Remove some unneeeded ATF timeout specifications.  The default is fine.

PR:		241070
MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-05 03:19:53 +00:00
Alan Somers
7d4670b647 ZFS: the hotspare_add_004_neg test needs at least two disks
MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-05 01:34:35 +00:00
Cy Schubert
289850db1c Add missing definition in DEBUG code.
MFC after:	3 days
2019-10-04 22:10:38 +00:00
Conrad Meyer
c76e96edf6 nvdimm(4): Fix Clang build after r353110
Clang spuriously warns about some well-defined C99 static initializers.
Mute it.

X-MFC-With:	r353110
2019-10-04 21:47:09 +00:00