Commit Graph

244831 Commits

Author SHA1 Message Date
Brooks Davis
cc698b4900 Update tcsh to 6.21.00.
This is a bugfix release with no new features.  A number of these fixes
were previously merged into our tree.

Sponsored by:	DARPA, AFRL
2019-10-08 20:59:31 +00:00
Brooks Davis
0ed89481e1 Allow -DNO_CLEAN build across r352689.
Split the LIBCOMPAT case because the usual egrep only matches in
LIBCOMPAT on amd64.
2019-10-08 20:26:51 +00:00
John Baldwin
c59050aab5 Set the FID field in lookaside crypto requests to the rx queue ID.
The PCI block in the adapter requires this field to be set to a valid
queue ID.  It is not clear why it did not fail on all machines, but
the effect was that crypto operations reading input data via DMA
failed with an internal PCI read error on machines with 128G or more
of RAM.

Reported by:	gallatin
Reviewed by:	np
MFC after:	3 days
Sponsored by:	Chelsio Communications
2019-10-08 20:22:05 +00:00
Hans Petter Selasky
08650d170c Fix regression issue after r352989:
As noted by the commit message, callouts are now persistant
and should not be in the auto-zero section of the RQ's and SQ's.
This fixes an assert when using the TX completion event
factor feature with mlx5en(4).

Found by:	gallatin@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-08 19:49:25 +00:00
Glen Barber
48c8f7d454 Rework the logic for installing the pkg(8) configuration.
'quarterly' package sets do not exist for head, so explicitly
install the 'latest' configuration file there.  Otherwise,
fall back to the original conditional evaluation to determine
if the 'latest' or 'quarterly' configuration file should be
installed.

Reported by:	manu
Reviewed by:	manu
MFC after:	3 days
Sponsored by:	Rubicon Communications, LLC (Netgate)
2019-10-08 18:58:23 +00:00
Brooks Davis
320f294678 Import 6.21.00. 2019-10-08 18:20:02 +00:00
Gleb Smirnoff
1e80e4f26c Remove epoch assertion from if_setlladdr(). Originally this function was
protected by IF_ADDR_LOCK(), which was a mutex, so that two simultaneous
if_setlladdr() can't execute. Later it was switched to IF_ADDR_RLOCK(),
likely by a mistake. Later it was switched to NET_EPOCH_ENTER(). Then I
incorrectly added NET_EPOCH_ASSERT() here.

In reality ifp->if_addr never goes away and never changes its length. So,
doing bcopy() in it is always "safe", meaning it won't dereference a wrong
pointer or write into someone's else memory. Of course doing two bcopy() in
parallel would result in a mess of two addresses, but net epoch doesn't
protect against that, neither IF_ADDR_RLOCK() did.

So for now, just remove the assertion and leave for later a proper fix.

Reported by:	markj
2019-10-08 17:55:45 +00:00
Gleb Smirnoff
e4c40a8a71 Quickly plug another regression from r353292. Again, multicast locking needs
lots of work...

Reported by:	pho
2019-10-08 16:59:17 +00:00
Gleb Smirnoff
e9dc46cc30 In DIAGNOSTIC block of if_delmulti_ifma_flags() enter the network epoch.
This quickly plugs the regression from r353292. The locking of multicast
definitely needs a broader review today...

Reported by:	pho, dhw
2019-10-08 16:45:56 +00:00
Mark Johnston
8f52459857 Simplify pmap_page_array_startup() a bit.
No functional change intended.

Sponsored by:	The FreeBSD Foundation
2019-10-08 16:42:50 +00:00
Alan Somers
310a44b232 zfs: fix the zfsd_hotspare_007_pos test
It was trying to destroy the pool while zfsd was detaching the spare, and
"zpool destroy" failed.  Fix by waiting until the spare has fully detached.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-08 15:48:28 +00:00
Alan Somers
dca76f293a zfs: fix the zfsd_autoreplace_003_pos test
The test declared that it only needed 5 disks, but actually tried to use 6.
Fix it to use just 5, which is all it really needs.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-08 15:33:11 +00:00
Mark Johnston
b0fd461524 Avoid erroneously clearing PGA_WRITEABLE in riscv's pmap_enter().
During a CoW fault, we must check for both 4KB and 2MB mappings before
clearing PGA_WRITEABLE on the old mapping's page.  Previously we were
only checking for 4KB mappings.  This was missed in r344106.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-10-08 15:03:48 +00:00
Mateusz Guzik
06d25b07c6 amd64 pmap: allocate pv table entries for gaps in PA
This matches the state prior to r353149 and fixes crashes with DRM
modules.

Reported and tested by:	cy, garga, Krasznai Andras
Fixes: r353149 ("amd64 pmap: implement per-superpage locks")
Sponsored by:	The FreeBSD Foundation
2019-10-08 14:59:50 +00:00
Mark Johnston
7992921bfe Clear PGA_WRITEABLE in riscv's pmap_remove_l3().
pmap_remove_l3() may remove the last mapping of a page, in which case
it must clear PGA_WRITEABLE.

Reported by:	Jenkins, via lwhsu
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-10-08 14:54:35 +00:00
Eric van Gyzen
4661f8af40 Fix problems in the kern_maxfiles__increase test
ATF functions such as ATF_REQUIRE do not work correctly in child processes.
Use plain C functions to report errors instead.

In the parent, check for the untimely demise of children.  Without this,
the test hung until the framework's timeout.

Raise the resource limit on the number of open files.  If this was too low,
the test hit the two problems above.

Restore the kern.maxfiles sysctl OID in the cleanup function.
The body prematurely removed the symlink in which the old value was saved.

Make the test more robust by opening more files.  In fact, due to the
integer division by 4, this was necessary to make the test valid with
some initial values of maxfiles.  Thanks, asomers@.

wait() for children instead of sleeping.

Clean up a temporary file created by the test ("afile").

Reviewed by:	asomers
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D21900
2019-10-08 13:43:05 +00:00
Andriy Gapon
ac99b25298 zfs: use atomic_load_64 to read atomic variable in dmu_object_alloc_impl
As long as we support ZFS on 32-bit platforms we should do this for all
64-bit variables that are modified in a lockless fashion using atomic
operations.  Otherwise, there is a risk of a reading a torn value.

Here is a rationale for why I am doing this in dmu_object_alloc_impl:
- it's very recent code
- the code deals with object IDs and a number of objects in a file
  system can overflow 32 bits
- incorrect allocation of an object ID may result in hard to debug
  problems
- fixing all plain reads of 64-bit atomic variables is not a trivial
  undertaking to do in one shot, so I chose to do it incrementally

MFC after:	3 weeks
X-MFC after:	r353301, r353176
2019-10-08 11:27:48 +00:00
Michael Tuexen
953b78bed9 Validate length before use it, not vice versa.
r353060 should have contained this...
This fixes
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=18070
MFC after:		3 days
2019-10-08 11:07:16 +00:00
Hans Petter Selasky
a362cf527e Fix regression issue after r353274:
Make sure the vnet_shutdown field is not set until after all
VNET_SYSUNINIT()'s in the SI_SUB_VNET_DONE subsystem have been
executed. Especially the vnet_if_return() functions requires that
if_move() is still operational.

Reported by:	lwhsu@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-08 11:06:24 +00:00
Andriy Gapon
db8bee42ce i386: hide more of atomic 64-bit definitions under _KERNEL
At the moment i386 does not provide 64-bit atomic operations in
userland.  Exposing some atomic_*_64 defines can cause unnecessary
confusion.

Discussed with:	kib
MFC after:	2 weeks
2019-10-08 10:50:16 +00:00
Doug Moore
2288078c5e Define macro VM_MAP_ENTRY_FOREACH for enumerating the entries in a vm_map.
In case the implementation ever changes from using a chain of next pointers,
then changing the macro definition will be necessary, but changing all the
files that iterate over vm_map entries will not.

Drop a counter in vm_object.c that would have an effect only if the
vm_map entry count was wrong.

Discussed with: alc
Reviewed by: markj
Tested by: pho (earlier version)
Differential Revision:	https://reviews.freebsd.org/D21882
2019-10-08 07:14:21 +00:00
Justin Hibbits
dbef5f7155 loader/powerpc64: Include generic PVR values in CAS architecture list
Add generic PVR values for PowerISA 2.07 and 3.00.  This allows booting pseries
in QEMU with compatibilty mode enabled.

Submitted by:	Shawn Anastasio <shawn@anastas.io>
2019-10-08 02:36:53 +00:00
Justin Hibbits
84046d16eb powerpc: Implement atomic_(f)cmpset_ for short and char
|
This adds two implementations for each atomic_fcmpset_ and atomic_cmpset_
short and char functions, selectable at compile time for the target
architecture.  By default, it uses a generic shift-and-mask to perform atomic
updates to sub-components of 32-bit words from <sys/_atomic_subword.h>.
However, if ISA_206_ATOMICS is defined it uses the ll/sc instructions for
halfword and bytes, introduced in PowerISA 2.06.  These instructions are
supported by all IBM processors from POWER7 on, as well as the Freescale/NXP
e6500 core.  Although the e5500 and e500mc both implement PowerISA 2.06 they
do not implement these instructions.

As part of this, clean up the atomic_(f)cmpset_acq and _rel wrappers, by
using macros to reduce code duplication.

ISA_206_ATOMICS requires clang or newer binutils (2.20 or later).

Differential Revision:	https://reviews.freebsd.org/D21682
2019-10-08 01:36:34 +00:00
Mark Johnston
cb49ec5431 Improve locking in the IPV6_V6ONLY socket option handler.
Acquire the inp lock before checking whether the socket is already bound,
and around updates to the inp_vflag field.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21867
2019-10-07 23:35:23 +00:00
Mark Johnston
4090e2170d Assert that the PGA_{WRITEABLE,EXECUTABLE} flags do not leak.
Reviewed by:	alc, kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21783
2019-10-07 23:31:17 +00:00
Mateusz Guzik
7b1fbc424a vm: stop trylocking page queues in vm_page_pqbatch_submit
About 11 minutes of poudriere -s -j 104 and probing on return value of
trylocks reveals that over 10% of attempts fail, which in turn means
there are more atomics performed than necessary.

Trylocking was there to try preventing migration, but it's not very likely
to happen if the lock is uncontested.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21925
2019-10-07 23:19:09 +00:00
Gleb Smirnoff
b8a6e03fac Widen NET_EPOCH coverage.
When epoch(9) was introduced to network stack, it was basically
dropped in place of existing locking, which was mutexes and
rwlocks. For the sake of performance mutex covered areas were
as small as possible, so became epoch covered areas.

However, epoch doesn't introduce any contention, it just delays
memory reclaim. So, there is no point to minimise epoch covered
areas in sense of performance. Meanwhile entering/exiting epoch
also has non-zero CPU usage, so doing this less often is a win.

Not the least is also code maintainability. In the new paradigm
we can assume that at any stage of processing a packet, we are
inside network epoch. This makes coding both input and output
path way easier.

On output path we already enter epoch quite early - in the
ip_output(), in the ip6_output().

This patch does the same for the input path. All ISR processing,
network related callouts, other ways of packet injection to the
network stack shall be performed in net_epoch. Any leaf function
that walks network configuration now asserts epoch.

Tricky part is configuration code paths - ioctls, sysctls. They
also call into leaf functions, so some need to be changed.

This patch would introduce more epoch recursions (see EPOCH_TRACE)
than we had before. They will be cleaned up separately, as several
of them aren't trivial. Note, that unlike a lock recursion the
epoch recursion is safe and just wastes a bit of resources.

Reviewed by:	gallatin, hselasky, cy, adrian, kristof
Differential Revision:	https://reviews.freebsd.org/D19111
2019-10-07 22:40:05 +00:00
Michael Tuexen
746c7ae563 In r343587 a simple port filter as sysctl tunable was added to siftr.
The new sysctl was not added to the siftr.4 man page at the time.
This updates the man page, and removes one left over trailing whitespace.

Submitted by:		Richard Scheffenegger
Reviewed by:		bcr@
MFC after:		3 days
Differential Revision:	https://reviews.freebsd.org/D21619
2019-10-07 20:35:04 +00:00
Alan Somers
c43f30ee6b ZFS: fix the redundancy tests
* Fix force_sync_path, which ensures that a file is fully flushed to disk.
  Apparently "zpool history"'s performance has improved, but exporting and
  importing the pool still works.
* Fix file_dva by using undocumented zdb syntax to clarify that we're
  interested in the pool's root file system, not the pool itself. This
  should also fix the zpool_clear_001_pos test.
* Remove a redundant cleanup step

MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D21901
2019-10-07 20:21:23 +00:00
Alan Somers
bbacbaccf9 ZFS: mark hotspare_scrub_002_pos as an expected failure
"zpool scrub" doesn't detect all errors on active spares in raidz arrays

PR:		241069
MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 20:19:05 +00:00
Alan Somers
87015aac27 ZFS: fix the delegate tests
These tests have never worked correctly

* Replace runwattr with sudo
* Fix a scoping bug with the "dtst" variable
* Cleanup user properties created during tests
* Eliminate the checks for refreservation and send support. They will always
  be supported.
* Fix verify_fs_snapshot. It seemed to assume that permissions would not yet
  be delegated, but that's not how it's actually used.
* Combine verify_fs_promote with verify_vol_promote
* Remove some useless sleeps
* Fix backwards condition in verify_vol_volsize
* Remove some redundant cleanup steps in the tests. cleanup.ksh will handle
  everything.
* Disable some parts of the tests that FreeBSD doesn't support:
    * Creating snapshots with mkdir
    * devices
    * shareisci
    * sharenfs
    * xattr
    * zoned

The sharenfs parts could probably be reenabled with more work to remove the
Solarisms.

MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D21898
2019-10-07 20:13:49 +00:00
Alan Somers
063f9d1295 zfs: skip the zfsd tests if zfsd is not running
MFC after:	2 weeks
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D21878
2019-10-07 19:50:22 +00:00
Alan Somers
95becc8898 zfs: fix the zdb_001_neg test
The test needed to be updated for r331701 (MFV illumos 8671400), which added
a "-k" option.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 19:48:18 +00:00
Alan Somers
f8069eab4c ZFS: fix the zpool_get_002_pos test
ZFS has grown some additional properties that hadn't been added to the
config file yet.  While I'm here, improve the error message, and remove a
superfluous command.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 19:24:50 +00:00
Edward Tomasz Napierala
1a13f2e6b4 Introduce stats(3), a flexible statistics gathering API.
This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof).  Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs.  The stats(3) framework can be used in
both userspace and the kernel.

See the stats(3) manual page for details.

This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.

The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.

Reviewed by:	sef (earlier version)
Obtained from:	Netflix
Relnotes:	yes
Sponsored by:	Klara Inc, Netflix
Differential Revision:	https://reviews.freebsd.org/D20477
2019-10-07 19:05:05 +00:00
Alan Somers
d253c454bb zfs: fix the slog_012_neg test
This test attempts to corrupt a file-backed vdev by deleting it and then
recreating it with truncate.  But that doesn't work, because the pool
already has the vdev open, and it happily hangs on to the open-but-deleted
file.  Fix by truncating the file without deleting it.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 18:55:40 +00:00
Alan Somers
ba23dca684 ZFS: fix several zvol_misc tests
* Adapt zvol_misc_001_neg to use dumpon instead of Solaris's dumpadm
* Disable zvol_misc_003_neg, zvol_misc_005_neg, and zvol_misc_006_pos,
  because they involve using a zvol as a dump device, which FreeBSD does not
  yet support.

MFC after:	2 weeks
Sponsored by:	Axcient
2019-10-07 18:21:33 +00:00
Hans Petter Selasky
4715738b12 Compile time assert a valid subsystem for all VNET init and uninit functions.
Using VNET init and uninit functions outside the given range has undefined
behaviour.

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-07 14:24:59 +00:00
Hans Petter Selasky
204e2f30d9 Factor out VNET shutdown check into an own vnet structure field.
Remove the now obsolete vnet_state field. This greatly simplifies the
detection of VNET shutdown and avoids code duplication.

Discussed with:	bz@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-07 14:15:41 +00:00
Hans Petter Selasky
ac66be4122 Make control endpoint quirk for xhci(4) configurable.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2019-10-07 13:40:29 +00:00
Andriy Gapon
3251c5ae51 fix up r353168, add atomic_swap_64 to i386 version of opensolaris_atomic.S
The compatibility code for the atomic operations in ZFS code is a bit
messy.  In some cases the native definitions are directly made
available, in some cases there are emulated operations in
opensolaris_atomic.c and in yet other cases there are atomic operations
implemented in assembly that were obtained from OpenSolaris / illumos.

This commit adds atomic_swap_64 for use with i386 userland.
The code is copied from illumos.

I am not sure why FreeBSD does not provide that operation natively.
Maybe because we try (or pretend) to support processors that did not
have the necessary instructions.

While here I also added atomic_load_64 for the same reasons.
This is original code based on iilumos atomic_swap_64 and FreeBSD
atomic_load_acq_64_i586.

Pointyhat to:	avg
MFC after:	1 week
2019-10-07 12:53:27 +00:00
Andriy Gapon
862c20fd89 MFV r350898, r351075: 8423 8199 7432 Implement large_dnode pool feature
8423 8199 7432 Implement large_dnode pool feature

7432 Large dnode pool feature
8199 multi-threaded dmu_object_alloc()
8423 Implement large_dnode pool feature
10406 large_dnode changes broke zfs recv of legacy stream

llumos/illumos-gate@54811da5ac
54811da5ac
https://www.illumos.org/issues/8423
https://www.illumos.org/issues/8199
https://www.illumos.org/issues/7432

illumos/illumos-gate@811964cd9f
811964cd9f
https://www.illumos.org/issues/10406

  ZoL issues:
  Improved dnode allocation #6564
  Clean up large dnode code #6262
  Fix dnode_hold() freeing dnode behavior #8172
  Fix dnode allocation race #6414, #6439
  Partial: Raw sends must be able to decrease nlevels #6821, #6864
  Remove unnecessary txg syncs from receive_object() Closes #7197

This updates FreeBSD large_dnode code (that was imported from ZoL) to a
version that was committed to illumos.  It has some cleanups,
improvements and fixes comparing to what we have in FreeBSD now.
I think that the most significant update is 8199 multi-threaded
dmu_object_alloc().

This commit reverts r351077 that was a revert of r351074 and r351076 and
restores those changes.  Required atomic operations should be available
now on all platforms where we build ZFS.

Obtained from:	illumos
MFC after:	3 weeks
2019-10-07 08:14:45 +00:00
Emmanuel Vadot
0a457350fd arm: dts: ti: Fix mmc3 instance by setting it to disabled
DTS Import of Linux 5.3 added a patch that rework the L3 mmc instance
in the AM335x SoC but removed the status = 'disabled' on the node.
This cause the kernel to probe the device even if the board doesn't
have this mmc used and since we don't correctly activate the clock
for this module we panic with an external data abort.
Beaglebone(s) don't have this device anyway so simply disabling it.
Patch for the DTS was sent upstream.
https://patchwork.kernel.org/patch/11176921/

PR:		241089
Reported by:	phk
2019-10-07 08:11:49 +00:00
Andriy Gapon
f1fd3654e7 ZFS: unconditionally use atomic_swap_64
Previously, the code used a plain store on platforms that lacked
atomic_swap_64 and possibly some other platforms as the condition worked
only if atomic_swap_64 was a macro.

MFC after:	1 week
X-MFC after:	r353166, r353167
2019-10-07 08:00:54 +00:00
Andriy Gapon
cf78c4beae ZFS: add emulation of atomic_swap_64 and atomic_load_64
Some 32-bit platforms do not provide 64-bit atomic operations that ZFS
requires, either in userland or at all.  We emulate those operations for
those platforms using a mutex.  That is not entirely correct and it's
very efficient.  Besides, the loads are plain loads, so torn values are
possible.

Nevertheless, the emulation seems to work for some definition of work.

This change adds atomic_swap_64, which is already used in ZFS code, and
atomic_load_64 that can be used to prevent torn reads.

MFC after:	1 week
2019-10-07 07:54:34 +00:00
Andriy Gapon
eab7984cfe add atomic_load_64 for mipsn32
It's just an alias for atomic_load_acq_64 (same as on i386).

MFC after:	1 week
2019-10-07 07:42:26 +00:00
Andriy Gapon
80475f9523 align use of cp15_pmccntr_get with its availability
According to ian, the only armv6 cpu we support is the 1176, so this
change is effectively a no-op.
The change is just to make the code more self-consistent.
The issue was noticed by a standalone module build for armv6.

Reviewed by:	ian
MFC after:	3 weeks
2019-10-07 07:37:42 +00:00
Alan Cox
bc285d6a8f Eliminate an unused declaration. The variable in question is only defined
and used on sparc64.

MFC after:	1 week
2019-10-07 04:22:03 +00:00
Alan Cox
e227f09282 Eliminate a redundant bzero(). The l0 page table page was already zeroed
by efi_1t1_page().

MFC after:	1 week
2019-10-07 03:37:28 +00:00
Kyle Evans
605261992c Revert r352557: powerpc/loader: Install ubldr without stripping
This was committed due to what was later diagnosed as an msdosfs bug
preventing in-place strip. This bug was fixed in r352564, and we agreed to
keep the workaround in for a bit to allow the driver fix a suitable amount
of propagation time for folks building/installing powerpc/ubldr, seeing as
how we were not in any hurry to revert.
2019-10-07 03:28:11 +00:00