Commit Graph

235194 Commits

Author SHA1 Message Date
Alexander Motin
d49e9be14f MFV r337184: 9457 libzfs_import.c:add_config() has a memory leak
A memory leak occurs on lines 209 and 213 because the config is not freed
in the error case.  The interface to add_config() seems less than ideal -
it would be better if it copied any data necessary from the config and the
caller freed it.

illumos/illumos-gate@ddfe901b12

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     sara hartse <sara.hartse@delphix.com>
2018-08-02 21:25:32 +00:00
Alexander Motin
732d23f312 9457 libzfs_import.c:add_config() has a memory leak
A memory leak occurs on lines 209 and 213 because the config is not freed
in the error case.  The interface to add_config() seems less than ideal -
it would be better if it copied any data necessary from the config and the
caller freed it.

illumos/illumos-gate@ddfe901b12

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:     sara hartse <sara.hartse@delphix.com>
2018-08-02 21:24:04 +00:00
Alexander Motin
2bce9a5316 MFV r337182: 9330 stack overflow when creating a deeply nested dataset
Datasets that are deeply nested (~100 levels) are impractical. We just put
a limit of 50 levels to newly created datasets. Existing datasets should
work without a problem.

illumos/illumos-gate@5ac95da7d6

Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author:     Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
2018-08-02 21:19:35 +00:00
Alexander Motin
cb97edbcd0 9330 stack overflow when creating a deeply nested dataset
Datasets that are deeply nested (~100 levels) are impractical. We just put
a limit of 50 levels to newly created datasets. Existing datasets should
work without a problem.

illumos/illumos-gate@5ac95da7d6

Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author:     Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
2018-08-02 21:12:52 +00:00
Alexander Motin
3f586250d8 9539 Make zvol operations use _by_dnode routines
Continues what was started in 7801 add more by-dnode routines by fully
converting zvols to avoid unnecessary dnode_hold() calls. This saves a
small amount of CPU time and slightly improves latencies of operations
on zvols.

illumos/illumos-gate@8dfe5547fb

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Richard Yao <richard.yao@prophetstor.com>
2018-08-02 21:07:04 +00:00
Alexander Motin
8398b8e966 9539 Make zvol operations use _by_dnode routines
Continues what was started in 7801 add more by-dnode routines by fully
converting zvols to avoid unnecessary dnode_hold() calls. This saves a
small amount of CPU time and slightly improves latencies of operations
on zvols.

illumos/illumos-gate@8dfe5547fb

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed by: Rick McNeal <rick.mcneal@nexenta.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Richard Yao <richard.yao@prophetstor.com>
2018-08-02 20:49:08 +00:00
Alexander Motin
ac879e61ad 9523 Large alloc in zdb can cause trouble
16MB alloc in zdb_embedded_block() can cause cores in certain situations
(clang, gcc55).

OsX commit: ced236a5da
FreeBSD commit: https://svnweb.freebsd.org/base?view=revision&revision=326150
illumos/illumos-gate@03a4c2f4bf

Reviewed by: Igor Kozhukhov <igor@dilos.org>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Jorgen Lundman <lundman@lundman.net>

This is an update for r326150 (by avg), where this change comes from.
2018-08-02 20:44:07 +00:00
Alexander Motin
903252254b 9523 Large alloc in zdb can cause trouble
16MB alloc in zdb_embedded_block() can cause cores in certain situations
(clang, gcc55).

OsX commit: ced236a5da
FreeBSD commit: https://svnweb.freebsd.org/base?view=revision&revision=326150
illumos/illumos-gate@03a4c2f4bf

Reviewed by: Igor Kozhukhov <igor@dilos.org>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Jorgen Lundman <lundman@lundman.net>
2018-08-02 20:37:53 +00:00
Alexander Motin
5463ca24bf MFV r337175: 9487 Free objects when receiving full stream as clone
All objects after the last written or freed object are not supposed to
exist after receiving the stream. We should free them accordingly, as if
a freeobjects record for them had been included in the stream.

zfsonlinux/zfs@48fbb9ddbf
illumos/illumos-gate@7864b8192b

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>
2018-08-02 20:33:13 +00:00
Alexander Motin
abb5258c34 9487 Free objects when receiving full stream as clone
All objects after the last written or freed object are not supposed to
exist after receiving the stream. We should free them accordingly, as if
a freeobjects record for them had been included in the stream.

zfsonlinux/zfs@48fbb9ddbf
illumos/illumos-gate@7864b8192b

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Paul Dagnelie <pcd@delphix.com>
2018-08-02 20:29:58 +00:00
Alexander Motin
0485d5cba8 MFV r337171:
9464 txg_kick() fails to see that we are quiescing, forcing transactions
to their next stages without leaving them accumulate changes

Ideally we would like txg_kick() to get triggered only when we are sure
that we are not syncing AND not quiescing any txg. This way we can kick
an open TXG to the quiescing state when we are sure that there is nothing
going on and we would benefit from the different states running
concurrently.

illumos/illumos-gate@fa41d87de9

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>
2018-08-02 20:18:49 +00:00
Alexander Motin
fdbc73edd4 9464 txg_kick() fails to see that we are quiescing, forcing transactions
to their next stages without leaving them accumulate changes

Ideally we would like txg_kick() to get triggered only when we are sure
that we are not syncing AND not quiescing any txg. This way we can kick
an open TXG to the quiescing state when we are sure that there is nothing
going on and we would benefit from the different states running
concurrently.

illumos/illumos-gate@fa41d87de9

Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Serapheim Dimitropoulos <serapheim@delphix.com>
2018-08-02 20:17:37 +00:00
Rick Macklem
ac0d649588 Silence newer gcc warnings.
Newer versions of gcc generate "might not be initialized" warnings for
several variables in nfsrpc_doiods().  I have checked and all of these
variables are assigned values before they are used.
In the one case of "tdrpc", it could have passed garbage as an argument
to nfscl_dofflayoutio() when mirrorcnt is one. However nfscl_dofflayoutio() only
uses the argument when mirrorcnt > 1, so it wasn't actually broken.
This patch initializes "tdrpc" to avoid confusion and initializes the rest
to make the compiler happy.

Requested by:	mmacy
2018-08-02 20:10:59 +00:00
Alexander Motin
aba9f1124f MFV r337167: 9442 decrease indirect block size of spacemaps
Updates to indirect blocks of spacemaps can contribute significantly to
write inflation.  Therefore we want to reduce the indirect block size of
spacemaps from 128K to 16K.

illumos/illumos-gate@221813c13b

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Albert Lee <trisk@forkgnu.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>
2018-08-02 20:06:46 +00:00
Navdeep Parhar
ac8ec5fea6 cxgbe(4): Use the tx credit limit for ethofld rather than TOE when
initializing the softc for a per-flow rate limiter.  The limit happens
to be the same for both and the existing code worked by accident for
common configurations.

Reported by:	gallatin@
Sponsored by:	Chelsio Communications
2018-08-02 19:50:12 +00:00
Alexander Motin
58d5696e1c 9442 decrease indirect block size of spacemaps
Updates to indirect blocks of spacemaps can contribute significantly to
write inflation.  Therefore we want to reduce the indirect block size of
spacemaps from 128K to 16K.

illumos/illumos-gate@221813c13b

Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Albert Lee <trisk@forkgnu.org>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author:     Matthew Ahrens <mahrens@delphix.com>
2018-08-02 19:37:13 +00:00
Conrad Meyer
dab6195cd3 FUSE: Bump maximum IO size to enable more performant operation
Various components restrict size of IO passed up to the userspace filesystem
based on the mount's f_iosize value.  The previous default of PAGE_SIZE
is anemic, even for normal filesystems, but especially considering every
FUSE operation involves a kernel <-> userspace IPC upcall.

Bump to DFLTPHYS (currently 64kB) to match other FUSE implementations.

Anecdotally, Jakub reports IO read performance increased from 600 MB/s ->
2700 MB/s with a basic RAM-backed FUSE filesystem.

PR:		230260
Reported by:	Peter (MooseFS) <freebsd AT moosefs.com>
Tested by:	Jakub Kruszona-Zawadzki <acid AT moosefs.com>
MFC after:	3 days
2018-08-02 19:25:43 +00:00
Alexander Motin
c423a5e6b8 MFV r337161: 9512 zfs remap poolname@snapname coredumps
Only filesystems and volumes are valid "zfs remap" parameters: when passed
a snapshot name zfs_remap_indirects() does not handle the EINVAL returned
from libzfs_core, which results in failing an assertion and consequently
crashing.

illumos/illumos-gate@0b2e825398

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     loli10K <ezomori.nozomu@gmail.com>
2018-08-02 19:13:45 +00:00
Alexander Motin
cf8fc527c9 9512 zfs remap poolname@snapname coredumps
Only filesystems and volumes are valid "zfs remap" parameters: when passed
a snapshot name zfs_remap_indirects() does not handle the EINVAL returned
from libzfs_core, which results in failing an assertion and consequently
crashing.

illumos/illumos-gate@0b2e825398

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: John Wren Kennedy <john.kennedy@delphix.com>
Reviewed by: Sara Hartse <sara.hartse@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     loli10K <ezomori.nozomu@gmail.com>
2018-08-02 19:09:13 +00:00
Alexander Motin
7fca1b93c4 Do not blindly include illumos kernel headers instead of user-space.
It is not needed now, and I doubt it much helped at all, creating more
confusions then good.
2018-08-02 18:55:55 +00:00
Glen Barber
768a8d8d44 Fix the ftp-stage target for arm embedded builds.
The images were renamed from KERNCONF to BOARDNAME when
specified, which would result in an image name of:

 12.0-CURRENT-arm-armv7-GENERIC.img

which would then be renamed to use the BOARDNAME for the
SoC the image is targeted to use.  BOARDNAME was specified
for all images as of r336994, which now causes the ftp-stage
target to fail, as the rename is no longer necessary.

Sponsored by:	The FreeBSD Foundation
2018-08-02 18:51:44 +00:00
Ben Widawsky
b7e25196ea Remove spuriously added svn properties 2018-08-02 18:37:02 +00:00
Ben Widawsky
be223cad78 tools: Add a git-svn bootstrap script
codified version of https://wiki.freebsd.org/GitWorkflow/GitSvn#Using_git-svn

Approved by:	emaste (mentor)
Suggested by:	Warner Losh (imp)
Differential Revision:  https://reviews.freebsd.org/D16528
2018-08-02 18:28:02 +00:00
Mark Johnston
0dbdecfd4f Add a rudimentary test for procstat kstack.
MFC after:	1 week
2018-08-02 16:03:47 +00:00
Toomas Soome
bcc7f4c221 loader.efi: efipart_inithandles() failure should not prevent devsw initialization
The efipart_inithandles() can return error code when there are no block devices,
however, we still should walk and initialize devsw.
2018-08-02 12:43:40 +00:00
Ed Maste
7a9e13ddb2 Regen src.conf.5 after r337057, LLD_BOOTSTRAP on armv7 2018-08-02 12:39:23 +00:00
Ruslan Bukin
d8af1b6af6 Implement pt_fpreg_to_ucontext(), pt_ucontext_to_fpreg().
Sponsored by:	DARPA, AFRL
2018-08-02 12:24:34 +00:00
Ruslan Bukin
f00ebc42b5 Save and restore floating-point arguments.
Sponsored by:	DARPA, AFRL
2018-08-02 12:21:39 +00:00
Ruslan Bukin
385a185b43 Don't overwrite tp in set_mcontext().
This makes libthr/swapcontext_test:swapcontext1 happy.

Sponsored by:	DARPA, AFRL
2018-08-02 12:13:52 +00:00
Ruslan Bukin
84154f4b9e o Don't overwrite tp in fork_trampoline().
o Save and restore tp in cpu_switch().
o Restore tp in cpu_throw().
o Save tp in savectx().

This makes libthr tests happy. In particular fork_test:fork.

Sponsored by:	DARPA, AFRL
2018-08-02 12:12:13 +00:00
Ruslan Bukin
7bb4a84ad3 o Correctly set user tls base: consider TP_OFFSET.
o Ensure tp (thread pointer) saved before copying the pcb.

Sponsored by:	DARPA, AFRL
2018-08-02 12:08:52 +00:00
Edward Tomasz Napierala
50cf3bac62 Fix regression tests broken by r337067.
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-08-02 11:55:16 +00:00
Andriy Gapon
a260971458 fix a typo resulting in a wrong variable in kern_syscall_deregister
The difference is between sysent, a global, and sysents, a function
parameter.
2018-08-02 09:41:55 +00:00
Edward Tomasz Napierala
711a4538f8 Make sure the rtld(1) error messages go to stderr, not stdout.
While here fix capitalization of a few nearby strings, add the
rtld's file name prefix so it's obvious where the message come
from, and return zero when "-h" is used.

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D16530
2018-08-02 07:43:28 +00:00
Toomas Soome
17e2c2661b libsa: dereferencing type-punned pointer in cd9660
The warning is given by gcc build, but it is good to fix anyhow.
use bcopy instead of direct assignment.
2018-08-02 06:22:10 +00:00
Alexander Motin
b59e9cd1c0 MFV r316926:
7955 libshare needs to initialize only those datasets being modified by the consumer

illumos/illumos-gate@8a981c3356
8a981c3356

https://www.illumos.org/issues/7955
  Libshare currently initializes all available filesystems when doing any
  libshare operation. This requires iterating through all the filesystem
  multiple times, which is a huge performance problem for sharing and
  unsharing operations.

Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Yuri Pankov <yuri.pankov@gmail.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>
Author: Daniel Hoffman <dj.hoffman@delphix.com>

For FreeBSD this is practically a NOP, just a diff reduction.
2018-08-01 21:51:49 +00:00
Kyle Evans
bba8f1fe0e ubldr: Bump heap size, 1MB -> 2MB
1MB was leaving very little margin in some of the worse-case scenarios with
lualoader. 2MB is still low enough that we shouldn't have any problems with
UBoot-supported boards.

MFC after: 1 week
2018-08-01 20:08:20 +00:00
Mark Johnston
48729f75e1 Fix some nits in the unix_passfd tests.
- Remove return statements in functions with a void return type.
- Allocate enough space for the SCM_CREDS and SCM_RIGHTS messages
  received in the rights_creds_payload test.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-08-01 19:45:04 +00:00
Ed Maste
800ab3e3c7 Use lld as the bootstrap linker on armv7 by default
As of r336972 lld is capable linking the armv7 kernel and userland,
so enable it by default.

PR:		229050
Reviewed by:	kevans
Tested by:	kevans
Relnotes:	Yes
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16528
2018-08-01 19:17:55 +00:00
Hans Petter Selasky
a76c73aadc Don't refer to non-existing atomic functions, even though not compiled,
in the LinuxKPI.

Found by:		rpolka @
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2018-08-01 19:10:46 +00:00
Konstantin Belousov
61b285ac57 Avoid assertion in /dev/ufssuspend when the suspend ioctl is
(incorrectly) called while another suspension is already active.

PR:	230220
Reported by:	dexuan
Tested by:	pho
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-08-01 19:06:55 +00:00
Konstantin Belousov
109de20b5a Add -D option to pciconf(8) to mmap and dump content of the device BAR.
Discussed with:	imp, jhb
Sponsored by:	The FreeBSD Foundation, Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15583
2018-08-01 19:02:05 +00:00
Konstantin Belousov
87842989f8 Add ioctl to conveniently mmap a PCI device BAR into userspace.
Add the ioctl PCIOCBARMMAP on /dev/pci to conveniently create
userspace mapping of a PCI device BAR.  This is enormously superior to
read the BAR value with PCIOCREAD and then try to mmap /dev/mem, and
should allow to automatically activate the mapped BARs when needed in
future.

Current implementation creates new sg pager for each user mmap
request.  If the pointer (and reference) to a managed device pager is
stored in pci_map, we would be able to revoke all mappings on the BAR
deactivation or relocation.  This is related to the unimplemented BAR
activation on mmap, and is postponed for the future.

Discussed with:	imp, jhb
Sponsored by:	The FreeBSD Foundation, Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15583
2018-08-01 18:58:24 +00:00
Mark Johnston
c4c9cd8d68 Remove ARGS_NUM, accidentally left over after r337050. 2018-08-01 18:49:40 +00:00
Konstantin Belousov
e45b89d23d Add pmap_is_valid_memattr(9).
Discussed with:	alc
Sponsored by:	The FreeBSD Foundation, Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D15583
2018-08-01 18:45:51 +00:00
Mark Johnston
322dacf057 Don't hard-code the number of compression utility arguments.
The zstd invocation constructed by newsyslog contains one more parameter
than invocations for the other supported compression utilities.  However,
the maximum number of arguments was hard-coded, leading to an
out-of-bounds array access when using zstd compression.
2018-08-01 18:45:33 +00:00
Mark Johnston
995cd89924 Adjust the number of tests after r336913. 2018-08-01 18:41:43 +00:00
Konstantin Belousov
c9bbc3ef29 Make cache coherency attributes definitions available in machine/vm.h on MIPS.
Move definitions from cpuregs.h into the cca.h, and include cca.h into vm.h.
This is required to make MIPS MD memattr definitions usable in userspace.

Sponsored by:	The FreeBSD Foundation, Mellanox Technologies
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D15583
2018-08-01 18:35:17 +00:00
Alexander Motin
2395a7f52d 8115 parallel zfs mount
Mounting of filesystems in "filesystem/local" is done using `zfs mount -a`,
which mounts each filesystems serially. The bottleneck for each mount is
the I/O done to load metadata for each filesystem. As such, mounting
filesystems using a parallel algorithm should be a big win, and bring down
the runtime of "filesystem/local"'s start method.

illumos/illumos-gate@591e0e133f

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Approved by: Matt Ahrens <mahrens@delphix.com>
Author:     Sebastien Roy <seb@delphix.com>
2018-08-01 18:28:17 +00:00
Ed Maste
bd6313b66e libelf: reload section headers after update with ELF_C_WRITE
These issues (PR 218860 and PR 218861) are still undergoing additional
discussion in the upstream context and there may be additional changes
to come.

PR:		218860
Submitted by:	Eric McCorkle
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D10486
2018-08-01 15:55:14 +00:00