Commit Graph

241984 Commits

Author SHA1 Message Date
Rick Macklem
0f0869bca3 Modify mountd so that it incrementally updates the kernel exports upon a reload.
Without this patch, mountd would delete/load all exports from the exports
file(s) when it receives a SIGHUP. This works fine for small exports file(s),
but can take several seconds to do when there are large numbers (10000+) of
exported file systems. Most of this time is spent doing the system calls
that delete/export each of these file systems. When the "-S" option
has been specified (the default these days), the nfsd threads are suspended
for several seconds while the reload is done.

This patch changes mountd so that it only does system calls for file systems
where the exports have been changed/added/deleted as compared to the exports
done for the previous load/reload of the exports file(s).
Basically, when SIGHUP is posted to mountd, it saves the exportlist structures
from the previous load and creates a new set of structures from the current
exports file(s). Then it compares the current with the previous and only does
system calls for cases that have been changed/added/deleted.
The nfsd threads do not need to be suspended until the comparison step is
being done. This results in a suspension period of milliseconds for a server
with 10000+ exported file systems.

There is some code using a LOGDEBUG() macro that allow runtime debugging
output via syslog(LOG_DEBUG,...) that can be enabled by creating a file
called /var/log/mountd.debug. This code is expected to be replaced with
code that uses dtrace by cy@ in the near future, once issues w.r.t. dtrace
in stable/12 have been resolved.

The patch should not change the usage of the exports file(s), but improves
the performance of reloading large exports file(s) where there are only a
small number of changes done to the file(s).

Tested by:	pen@lysator.liu.se
PR:		237860
Reviewed by:	kib
MFC after:	1 month
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D20487
2019-06-03 22:58:51 +00:00
Maxim Sobolev
6c4448b394 Put back MTREE_CMD here for now. Doh, not my day, perhaps.
Reported by:	markj, mav
2019-06-03 21:40:42 +00:00
Alexander Motin
0b5319dda0 MFV r348585: 9683 Allow bypassing devid in vdev_disk_open()
illumos/illumos-gate@6fe4f3002c

Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Pavel Zakharov <pavel.zakharov@delphix.com>

This is irrelevant to FreeBSD, just to reduce divergence.
2019-06-03 20:55:52 +00:00
Alexander Motin
43f20e31a6 9683 Allow bypassing devid in vdev_disk_open()
illumos/illumos-gate@6fe4f3002c

Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Pavel Zakharov <pavel.zakharov@delphix.com>
2019-06-03 20:52:19 +00:00
Alexander Motin
9b048dd219 MFV r348583: 9847 leaking dd_clones (DMU_OT_DSL_CLONES) objects
illumos/illumos-gate@17fb938fd6

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Matthew Ahrens <mahrens@delphix.com>
2019-06-03 20:49:20 +00:00
Alexander Motin
e3f8abf29a 9847 leaking dd_clones (DMU_OT_DSL_CLONES) objects
illumos/illumos-gate@17fb938fd6

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Matthew Ahrens <mahrens@delphix.com>
2019-06-03 20:45:48 +00:00
Alexander Motin
eb106113c2 MFV r348580: 9559 zfs diff handles files on delete queue in fromsnap poorly
illumos/illumos-gate@20633e304b

Reviewed by: Joshua M. Clulow <josh@sysmgr.org>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Paul Dagnelie <pcd@delphix.com>
2019-06-03 20:40:32 +00:00
Alexander Motin
3705c6e18f 9559 zfs diff handles files on delete queue in fromsnap poorly
illumos/illumos-gate@20633e304b

Reviewed by: Joshua M. Clulow <josh@sysmgr.org>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author:     Paul Dagnelie <pcd@delphix.com>
2019-06-03 20:39:13 +00:00
Alexander Motin
07a5c938c9 MFV r348578: 9962 zil_commit should omit cache thrash
illumos/illumos-gate@cab3a55e15

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Approved by: Joshua M. Clulow <josh@sysmgr.org>
Author:     Prakash Surya <prakash.surya@delphix.com>
2019-06-03 20:24:40 +00:00
Alexander Motin
0e67d63bd8 9962 zil_commit should omit cache thrash
illumos/illumos-gate@cab3a55e15

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Approved by: Joshua M. Clulow <josh@sysmgr.org>
Author:     Prakash Surya <prakash.surya@delphix.com>
2019-06-03 20:18:02 +00:00
Alexander Motin
a66a7143d4 MFV r348576: 9963 Seperate tunable for disabling ZIL vdev flush
illumos/illumos-gate@f8fdf68125

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Prakash Surya <prakash.surya@delphix.com>
2019-06-03 20:05:43 +00:00
Alexander Motin
b62e8d32da 9963 Seperate tunable for disabling ZIL vdev flush
illumos/illumos-gate@f8fdf68125

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Prakash Surya <prakash.surya@delphix.com>
2019-06-03 19:53:39 +00:00
Cy Schubert
de982ef60d Properly define the fourth argument to ipf_check, the main entry point
into ipfilter. A proper definition simplifies dtrace scripts a little.

MFC after:	1 week
2019-06-03 19:37:14 +00:00
Alexander Motin
c9719c9a6d MFV r348573: 9993 zil writes can get delayed in zio pipeline
illumos/illumos-gate@2258ad0b75

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     George Wilson <george.wilson@delphix.com>
2019-06-03 19:25:53 +00:00
Alexander Motin
5f8c7a6af3 9993 zil writes can get delayed in zio pipeline
illumos/illumos-gate@2258ad0b75

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     George Wilson <george.wilson@delphix.com>
2019-06-03 19:24:40 +00:00
Alexander Motin
84df75c732 9880 Race in ZFS parallel mount
illumos/illumos-gate@bc4c0ff134

Reviewed by: Jason King <jason.king@joyent.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Approved by: Joshua M. Clulow <josh@sysmgr.org>
Author:     Andy Fiddaman <omnios@citrus-it.co.uk>
2019-06-03 19:20:10 +00:00
Tycho Nightingale
88e9fbe568 very large dma mappings can cause integer overflow
Reviewed by:	kib
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20505
2019-06-03 19:19:35 +00:00
Alexander Motin
ce88141b27 MFV r348568: 9466 add JSON output support to channel programs
illumos/illumos-gate@5267591016

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Alek Pinchuk <apinchuk@datto.com>
2019-06-03 19:15:06 +00:00
Warner Losh
63acab6a5b [zfsboot] Fix boot env back compat (#190)
* Fix boot env back compat

zfsboot must try zfsloader before loader in order to remain compatible
with boot environments created prior to zfs functionality being rolled
into loader proper.

* Improve comments in zfsboot

Explain the significance of the load path order, and put the comment
about looping through the paths in the appropriate scope.

Obtained From:  TrueNAS commit 4c60c62fcf0b6b6eac98ee8d46e7bbea64bc86f5
Submitted by: Ryan Moeller <ryan@freqlabs.com>
2019-06-03 19:10:46 +00:00
Alexander Motin
1425be644a 9466 add JSON output support to channel programs
illumos/illumos-gate@5267591016

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Alek Pinchuk <apinchuk@datto.com>
2019-06-03 19:09:39 +00:00
Alexander Motin
4d6afba5e0 MFV r348555: 9690 metaslab of vdev with no space maps was flushed during removal
illumos/illumos-gate@4e75ba6826

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>
2019-06-03 19:03:24 +00:00
Alexander Motin
1b61262505 MFC r348554: 9688 aggsum_fini leaks memory
illumos/illumos-gate@29bf2d68be

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Jorgen Lundman <lundman@lundman.net>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>
2019-06-03 19:00:24 +00:00
Alexander Motin
677ef2563d MFV r348553: 9681 ztest failure in spa_history_log_internal due to spa_rename()
illumos/illumos-gate@6aee0ad769

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>
2019-06-03 18:32:56 +00:00
Alexander Motin
74f7070445 MFV r348552: 9682 page fault in dsl_async_clone_destroy() while opening pool
illumos/illumos-gate@ade2c82828

Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>
2019-06-03 17:56:44 +00:00
Alexander Motin
2eff60e998 MFV r348551: 9862 fix typo in comment in vdev_impl.h
illumos/illumos-gate@84927f52bd

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Allan Jude <allanjude@freebsd.org>
2019-06-03 17:44:47 +00:00
Alexander Motin
bd2ae688a4 MFV r348550: 1700 Add SCSI UNMAP support
illumos/illumos-gate@047c81d31d

Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Saso Kiselkov <saso.kiselkov@nexenta.com>

This is irrelevant to FreeBSD, just a diff reduction.
2019-06-03 17:43:32 +00:00
Alexander Motin
d40f6a585a MFV r348548: 9617 too-frequent TXG sync causes excessive write inflation
illumos/illumos-gate@7928f4baf4

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>
2019-06-03 17:40:11 +00:00
Alexander Motin
b3ed2d08e4 MFV r348537: 8601 memory leak in get_special_prop()
illumos/illumos-gate@e19b450bec

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     John Gallagher <john.gallagher@delphix.com>
2019-06-03 17:29:57 +00:00
Alexander Motin
c066dcc074 MFV r348535: 9677 panic from zio_write_gang_block() when creating dump device on fragmented rpool
illumos/illumos-gate@7341a7de4f

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Brad Lewis <brad.lewis@delphix.com>
2019-06-03 17:27:25 +00:00
Alexander Motin
c4af53b8f1 MFV r348534: 9616 Bogus error when attempting to set property on read-only pool
illumos/illumos-gate@f62db44dbc

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andrew Stormont <astormont@racktopsystems.com>
2019-06-03 17:19:05 +00:00
Alexander Motin
299ed57f93 9738 9112 broke third block copy allocations within one metaslab group
illumos/illumos-gate@b86e7e3f0e

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Alexander Motin <mav@FreeBSD.org>
2019-06-03 17:14:54 +00:00
Alexander Motin
e5cd1b9d07 9690 metaslab of vdev with no space maps was flushed during removal
illumos/illumos-gate@4e75ba6826

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>
2019-06-03 17:13:35 +00:00
Alexander Motin
08bd25db92 9688 aggsum_fini leaks memory
illumos/illumos-gate@29bf2d68be

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Jorgen Lundman <lundman@lundman.net>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>
2019-06-03 17:11:26 +00:00
Alexander Motin
af9de5b9f4 9681 ztest failure in spa_history_log_internal due to spa_rename()
illumos/illumos-gate@6aee0ad769

Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>
2019-06-03 17:08:49 +00:00
Alexander Motin
bb8cd94d19 9682 page fault in dsl_async_clone_destroy() while opening pool
illumos/illumos-gate@ade2c82828

Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>
2019-06-03 17:06:02 +00:00
Alexander Motin
6cc5316344 9862 fix typo in comment in vdev_impl.h
illumos/illumos-gate@84927f52bd

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Allan Jude <allanjude@freebsd.org>
2019-06-03 17:02:35 +00:00
Alexander Motin
86a6a20c48 1700 Add SCSI UNMAP support
illumos/illumos-gate@047c81d31d

Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Reviewed by: Garrett D'Amore <garrett@damore.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Saso Kiselkov <saso.kiselkov@nexenta.com>
2019-06-03 17:01:16 +00:00
Alexander Motin
380d790f2b 9617 too-frequent TXG sync causes excessive write inflation
illumos/illumos-gate@7928f4baf4

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>
2019-06-03 16:57:06 +00:00
Mark Johnston
9afd281187 rpc.yppasswdd: Fix dirname(3) usage after r305952.
PR:		234972
Submitted by:	Edward Fuhr <edward.fuhr@us.fujitsu.com> (original)
MFC after:	3 days
2019-06-03 16:51:07 +00:00
Mark Johnston
8e7299c08b Typo.
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2019-06-03 16:47:00 +00:00
Konstantin Belousov
d852f79b23 hwpmc_intel: List all Silvermont ids.
PR:	238310
Based on submission by:	Masse Nicolas <nicolas.masse@stormshield.eu>
MFC after:	1 week
2019-06-03 16:21:09 +00:00
John Baldwin
e8d8cc0139 Warn about deprecated features on all major OS versions.
Reviewed by:	imp
MFC after:	3 days
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D20490
2019-06-03 15:43:40 +00:00
John Baldwin
37e98e0dda Add 'device cxgbe' explicitly in the synopsis.
ccr depends on symbols exported by the cxgbe driver as well as having
a runtime dependency.  While the runtime depenency was noted in the
manpage already, the compile-time dependency wasn't as clear.

PR:		238265
MFC after:	3 days
Sponsored by:	Chelsio Communications
2019-06-03 15:41:54 +00:00
Konstantin Belousov
b5c45a3e12 efirt efi_enter(): Release acquired locks and restore FPU ownership if
efi_arch_enter() returned an error.

Submitted:	Jan Martin Mikkelsen <janm@transactionware.com>
MFC after:	1 week
2019-06-03 15:41:45 +00:00
Maxim Sobolev
e1216db508 Rollback the rest of the botched r348521. Re-work would be posted to
reviews. Sorry.

Reported by:	Enji Cooper
2019-06-03 15:34:00 +00:00
Konstantin Belousov
185f7e0a9d amd64 ef_rt_arch_call: Preserve %rflags around call into EFI RT service.
If service code faulted, we might end up unwinding with interrupts
disabled.  Top-level kernel code should have interrupts enabled, which
is enforced by checks.

Save %rflags before entering EFI, and restore to the known good value
on return.  This handles situation with disabled interrupts on fault
and perhaps other potential bugs, e.g. invalid value for PSL_D.

Reported and tested by:	Jan Martin Mikkelsen <janm@transactionware.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-06-03 15:32:42 +00:00
Alexander Motin
ba6517c8a4 8601 memory leak in get_special_prop()
illumos/illumos-gate@e19b450bec

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Robert Mustacchi <rm@joyent.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     John Gallagher <john.gallagher@delphix.com>
2019-06-03 15:25:12 +00:00
Konstantin Belousov
f6e5ddff6b Remove dead check.
We already handled the case when symstrindex < 0 at line 680.

Reported by:	danfe using PVS-studio
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-06-03 15:23:37 +00:00
Alexander Motin
740701f1c4 9677 panic from zio_write_gang_block() when creating dump device on fragmented rpool
illumos/illumos-gate@7341a7de4f

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     Brad Lewis <brad.lewis@delphix.com>
2019-06-03 15:23:21 +00:00
Alexander Motin
0c5cf42060 9616 Bogus error when attempting to set property on read-only pool
illumos/illumos-gate@f62db44dbc

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:	Andrew Stormont <astormont@racktopsystems.com>
2019-06-03 15:19:23 +00:00