Commit Graph

944 Commits

Author SHA1 Message Date
Ruslan Bukin
ca20f8ec29 o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by:	ngie
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D11901
2017-08-07 14:09:57 +00:00
Mark Johnston
22e406c80b Rework and simplify the ksyms(4) implementation.
- Store the symbol table contents in an anonymous swap-backed object. Have
  mmap(/dev/ksyms) map that object, and stop mapping the symbol table into
  the calling process in ksyms_open(). Previously we would cache a pointer
  to the pmap of the opening process, and mmap(/dev/ksyms) would create a
  mapping using the physical address found by a pmap lookup at the initial
  mapping address. However, this assumes that the cached pmap is valid,
  which may not be the case. [1]
- Remove the ksyms ioctl interface. It appears to have been added to work
  around a limitation in libelf that no longer exists; see r321842.
  Moreover, the interface is difficult to support and isn't present in
  illumos. Since ksyms was added specifically to support lockstat(1), it
  is expected that this removal won't have any real impact.
- Simplify ksyms_read() to avoid unnecessary copying.
- Don't call the device handle destructor if we fail to capture a snapshot
  of the kernel's symbol table. devfs will do that for us.

Reported by:	Ilja van Sprundel <ivansprundel@ioactive.com> [1]
Reviewed by:	kib (previous revision)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D11789
2017-08-03 00:38:13 +00:00
Alexander Motin
f2d3f6918e Add compat shim part missed at r305197.
This fixes compatibility between old kernel and new ZFS tools.
It seems to be tradition to forget it. :(

PR:		221112
MFC after:	3 days
2017-08-02 10:33:47 +00:00
Enji Cooper
4b330699f8 Convert traditional ${MK_TESTS} conditional idiom for including test
directories to SUBDIR.${MK_TESTS} idiom

This is being done to pave the way for future work (and homogenity) in
^/projects/make-check-sandbox .

No functional change intended.

MFC after:	1 weeks
2017-08-02 08:35:51 +00:00
Mark Johnston
3c9308e82b Remove local variables missed in r321842.
X-MFC with:	r321842
2017-08-01 04:52:03 +00:00
Mark Johnston
90ec6fd4ba Let lockstat use ksyms(4)'s mmap interface.
The workaround described in the deleted comment is no longer needed.

MFC after:	1 week
2017-08-01 04:49:54 +00:00
Li-Wen Hsu
ad89d783a6 Add an auxiliary subroutine to generate some events for testing
This test is also timeout on a quiet system because there is nobody triggering
read probefunc while test execution.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11731
2017-07-26 12:07:46 +00:00
Li-Wen Hsu
288ebd813a The test case common.funcs.t_dtrace_contrib.tst_basename_d generates a
verifying script which needs being run to complete the test.

While here, add missing shebang.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11716
2017-07-25 13:18:28 +00:00
Li-Wen Hsu
637ba06516 Modify glob patterns and expected output to match FreeBSD's implementation.
Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11713
2017-07-25 13:14:02 +00:00
Li-Wen Hsu
5604d0f997 Make this test case accepts basename() in D script returns "" or "."
In Solaris, basename(1) and basename(3) both return "." while being given an
empty string (""), while in BSD (and Linux) basename(1) returns "" and
basename(3) returns "."

While here, also change #!/usr/bin/ksh to #!/usr/bin/env ksh to find ksh in
$PATH

Reviewed by:	gnn, markj (earlier version), ngie (earlier version)
Differential Revision:	https://reviews.freebsd.org/D11707
2017-07-25 13:11:20 +00:00
Li-Wen Hsu
4ca0dfa6b0 Explicitly set dynamic variable buffer size.
We added too many variable assignments in BEGIN block, which will run out of
default auto-configured variable buffer space.  The test VM has 4G RAM which
should be enough for most cases so it's reasonable to increase limitation to
these case.

Reviewed by:	gnn
Differential Revision:	https://reviews.freebsd.org/D11676
2017-07-25 13:07:06 +00:00
Li-Wen Hsu
23833df483 Explicitly set dynamic variable buffer size.
We added too many variable assignments in BEGIN block, which will run out of
default auto-configured variable buffer space.  The test VM has 4G RAM which
should be enough for most cases so it's reasonable to increase limitation to
these case.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11674
2017-07-25 13:04:24 +00:00
Li-Wen Hsu
070a148127 Add an auxiliary subroutine to generate read(2) event while testing.
Reviewed by:	gnn, ngie
Differential Revision:	https://reviews.freebsd.org/D11673
2017-07-25 13:01:10 +00:00
Li-Wen Hsu
d83c70758a Add a simple script which calls open(2) and others to generate events for
testing.

This test times-out on a quiet system because there is nobody triggers
syscall::open:entry or syscall::: probe while test execution.

Reviewed by:	gnn, markj (earlier version)
Differential Revision:	https://reviews.freebsd.org/D11671
2017-07-25 12:58:03 +00:00
Li-Wen Hsu
b9de3393dd Add a simple program which calls sigtimedwait(2) to generate events for testing
This test timeout on a quiet system because there is nobody triggers
'syscall::*wait*:entry' probe while test execution.

Reviewed by:	gnn, markj, ngie
Differential Revision:	https://reviews.freebsd.org/D11668
2017-07-25 12:52:32 +00:00
Enji Cooper
31ed01a2de Fix whitespace on a line in fix(..) accidentally missed in r321424
MFC after:	1 month
MFC with:	r321424
2017-07-24 17:29:56 +00:00
Enji Cooper
f3305cae02 Style cleanup: delete spurious trailing whitespace
MFC after:	1 month
2017-07-24 17:27:21 +00:00
Enji Cooper
aa52ad5489 Don't use incorrect hardcoded path to ksh -- use /usr/bin/env
to find ksh instead

MFC after:	1 month
2017-07-23 17:57:00 +00:00
Alan Somers
19d04786c7 zfsd(8): Remove pidfile on shutdown
MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2017-06-20 19:45:02 +00:00
Andriy Gapon
f9cdbaba8d MFV r318946: 8021 ARC buf data scatter-ization
illumos/illumos-gate@770499e185
770499e185

https://www.illumos.org/issues/8021
  The ARC buf data project (known simply as "ABD" since its genesis in the ZoL
  community) changes the way the ARC allocates `b_pdata` memory from using linear
  `void *` buffers to using scatter/gather lists of fixed-size 1KB chunks. This
  improves ZFS's performance by helping to defragment the address space occupied
  by the ARC, in particular for cases where compressed ARC is enabled. It could
  also ease future work to allocate pages directly from `segkpm` for minimal-
  overhead memory allocations, bypassing the `kmem` subsystem.
  This is essentially the same change as the one which recently landed in ZFS on
  Linux, although they made some platform-specific changes while adapting this
  work to their codebase:
  1. Implemented the equivalent of the `segkpm` suggestion for future work
  mentioned above to bypass issues that they've had with the Linux kernel memory
  allocator.
  2. Changed the internal representation of the ABD's scatter/gather list so it
  could be used to pass I/O directly into Linux block device drivers. (This
  feature is not available in the illumos block device interface yet.)

FreeBSD notes:
- the actual (default) chunk size is 4KB (despite the text above saying 1KB)
- we can try to reimplement ABDs, so that they are not permanently
  mapped into the KVA unless explicitly requested, especially on
  platforms with scarce KVA
- we can try to use unmapped I/O and avoid intermediate allocation of a
  linear, virtual memory mapped buffer
- we can try to avoid extra data copying by referring to chunks / pages
  in the original ABD

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Chris Williamson <chris.williamson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Dan Kimmel <dan.kimmel@delphix.com>

MFC after:	3 weeks
2017-06-20 17:39:24 +00:00
Bryan Drewery
c99b67a794 Utilize SYSROOT from r320119 in places where DESTDIR may be wanting WORLDTMP.
Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.

sys/boot/efi/loader/Makefile
        A LIBSTAND hack is no longer required for buildenv.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-06-19 20:47:24 +00:00
Andriy Gapon
b8d341fe26 MFV r319945,r319946: 8264 want support for promoting datasets in libzfs_core
illumos/illumos-gate@a4b8c9aa65
a4b8c9aa65

https://www.illumos.org/issues/8264
  Oddly there is a lzc_clone function, but no lzc_promote function.

Reviewed by: Andriy Gapon <avg@FreeBSD.org>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan McDonald <danmcd@kebe.com>
Approved by: Dan McDonald <danmcd@kebe.com>
Author: Andrew Stormont <astormont@racktopsystems.com>
MFC after:	1 week
2017-06-14 16:31:36 +00:00
Mark Johnston
c645060dbb Override the locale so that file lists get a consistent sort order.
Reported by:	avg
MFC after:	1 week
2017-06-10 14:47:01 +00:00
Andriy Gapon
6bd205a529 follow up to r319746: add the new test files to the make file
Reported by:	markj
MFC after:	2 days
X-MFC with:	r319746
2017-06-10 06:13:52 +00:00
Andriy Gapon
9260925dcd MFV r319740: 8168 NULL pointer dereference in zfs_create()
illumos/illumos-gate@690031d326
690031d326

https://www.illumos.org/issues/8168
  If we manage to export the pool on which we are creating a dataset (filesystem
  or zvol) between entering libzfs`zfs_create() and libzfs`zpool_open() call (for
  which we never check the return value) we end up dereferencing a NULL pointer
  in libzfs`zpool_close().
  This was discovered on ZFS on Linux. The same issue can be reproduced on
  Illumos running in parallel:
    while :; do zpool import -d /tmp testpool ; zpool export testpool ; done
    while :; do zfs create testpool/fs; zfs destroy testpool/fs ; done
  Eventually this will result in several core dumps like this one:
  [root@52-54-00-d3-7a-01 /cores]# mdb core.zfs.4244
  Loading modules: [ libumem.so.1 libc.so.1 libtopo.so.1 libavl.so.1
  libnvpair.so.1 ld.so.1 ]
  > ::stack
  libzfs.so.1`zpool_close+0x17(0, 0, 0, 8047450)
  libzfs.so.1`zfs_create+0x1bb(8090548, 8047e6f, 1, 808cba8)
  zfs_do_create+0x545(2, 8047d74, 80778a0, 801, 0, 3)
  main+0x22c(8047d2c, fef5c6e8, 8047d64, 8055a17, 3, 8047d70)
  _start+0x83(3, 8047e64, 8047e68, 8047e6f, 0, 8047e7b)
  >
  Fix and reproducer (systemtap): https://github.com/zfsonlinux/zfs/pull/6096

Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: loli10K <ezomori.nozomu@gmail.com>
MFC after:	2 weeks
2017-06-09 15:30:41 +00:00
Andriy Gapon
ad2b1a296f MFV r319744,r319745: 8269 dtrace stddev aggregation is normalized incorrectly
illumos/illumos-gate@79809f9cf4
79809f9cf4

https://www.illumos.org/issues/8269
  It seems that currently normalization of stddev aggregation is done
  incorrectly.
  We divide both the sum of values and the sum of their squares by the
  normalization factor. But we should divide the sum of squares by the
  normalization factor squared to scale the original values properly.

FreeBSD note: the actual change was committed in r316853, this commit
adds the test files and record merge information.

Reviewed by: Bryan Cantrill <bryan@joyent.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after:	1 week
Sponsored by:	Panzura
2017-06-09 15:16:39 +00:00
Allan Jude
39b0b876dc New sentences start on new lines, fix two violations
Reviewed by:	bcr
Sponsored by:	BSDCan Dev Summit
2017-06-08 01:39:17 +00:00
Allan Jude
dc379eca14 SHA-512 and Skein have been supported by the boot loader for some time.
Submitted by:	lifanov
Reviewed by:	bcr
Sponsored by:	BSDCan Dev Summit
2017-06-08 01:29:24 +00:00
Andriy Gapon
457711b02b MFV r316922: 5380 receive of a send -p stream doesn't need to try renaming snapshots
illumos/illumos-gate@471a88e499
471a88e499

https://www.illumos.org/issues/5380
  A stream created with zfs send -p -I contains properties of all snapshots of a
  given dataset as opposed to only properties of snapshots in a given range.
  Not only this is suboptimal but the receive code also does not filter
  properties by the range. So, properties of earlier snapshots would be updated
  even though the snapshots themselves are not in the stream (just their
  properties).
  Given that modifying the snapshot properties requires a TXG sync and that the
  snapshots are updated one by one the described behavior may lead to a sever
  performance penalty.

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Andriy Gapon <avg@FreeBSD.org>
MFC after:	3 weeks
2017-05-24 22:30:21 +00:00
Andriy Gapon
b6be31c7ca MFC r316908: 7541 zpool import/tryimport ioctl returns ENOMEM because provided buffer is too small for config
illumos/illumos-gate@8b65a70b76
8b65a70b76

https://www.illumos.org/issues/7541
  When calling zpool import, zpool does a few ioctls to ZFS.
  zpool allocates a buffer in userland and passes it to the kernel so that ZFS
  can copy info into it. ZFS will use it to put the nvlist that describes the
  pool configuration.
  If the allocated buffer is too small, ZFS will return ENOMEM and the call will
  have to be redone. This wastes CPU time and slows down the import process. This
  happens very often for the ZFS_IOC_POOL_TRYIMPORT call.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Pavel Zakharov <pavel.zakharov@delphix.com>
MFC after:	2 weeks
2017-05-24 21:32:35 +00:00
Andriy Gapon
5644318970 MFC r316904: 7729 libzfs_core`lzc_rollback() leaks result nvl
illumos/illumos-gate@ac428481f9
ac428481f9

https://www.illumos.org/issues/7729
  libzfs_core`lzc_rollback() doesn't free the result nvl after lzc_ioctl() call.

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Yuri Pankov <yuri.pankov@nexenta.com>

MFC after:	2 weeks
2017-05-24 20:53:01 +00:00
Andriy Gapon
c65389d367 MFV r316860: 7545 zdb should disable reference tracking
illumos/illumos-gate@4dd77f9e38
4dd77f9e38

https://www.illumos.org/issues/7545
  When evicting from the ARC, we manipulate some refcount_t's, e.g. arcs_size.
  When using zdb to examine a large amount of data (e.g. zdb -bb on a large pool
  with small blocks), the ARC may have a large number of entries. If reference
  tracking is enabled, there will be ~1 reference for each block in the ARC. When
  evicting, we decrement the refcount and have to search all the references to
  find the one that we are removing, which is very slow.
  Since zdb is typically used to find problems with the on-disk format, and not
  with the code it is running, we should disable reference tracking in zdb.

Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>

MFC after:	2 weeks
2017-05-24 20:41:26 +00:00
Konstantin Belousov
6992112349 Commit the 64-bit inode project.
Extend the ino_t, dev_t, nlink_t types to 64-bit ints.  Modify
struct dirent layout to add d_off, increase the size of d_fileno
to 64-bits, increase the size of d_namlen to 16-bits, and change
the required alignment.  Increase struct statfs f_mntfromname[] and
f_mntonname[] array length MNAMELEN to 1024.

ABI breakage is mitigated by providing compatibility using versioned
symbols, ingenious use of the existing padding in structures, and
by employing other tricks.  Unfortunately, not everything can be
fixed, especially outside the base system.  For instance, third-party
APIs which pass struct stat around are broken in backward and
forward incompatible ways.

Kinfo sysctl MIBs ABI is changed in backward-compatible way, but
there is no general mechanism to handle other sysctl MIBS which
return structures where the layout has changed. It was considered
that the breakage is either in the management interfaces, where we
usually allow ABI slip, or is not important.

Struct xvnode changed layout, no compat shims are provided.

For struct xtty, dev_t tty device member was reduced to uint32_t.
It was decided that keeping ABI compat in this case is more useful
than reporting 64-bit dev_t, for the sake of pstat.

Update note: strictly follow the instructions in UPDATING.  Build
and install the new kernel with COMPAT_FREEBSD11 option enabled,
then reboot, and only then install new world.

Credits: The 64-bit inode project, also known as ino64, started life
many years ago as a project by Gleb Kurtsou (gleb).  Kirk McKusick
(mckusick) then picked up and updated the patch, and acted as a
flag-waver.  Feedback, suggestions, and discussions were carried
by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles),
and Rick Macklem (rmacklem).  Kris Moore (kris) performed an initial
ports investigation followed by an exp-run by Antoine Brodin (antoine).
Essential and all-embracing testing was done by Peter Holm (pho).
The heavy lifting of coordinating all these efforts and bringing the
project to completion were done by Konstantin Belousov (kib).

Sponsored by:	The FreeBSD Foundation (emaste, kib)
Differential revision:	https://reviews.freebsd.org/D10439
2017-05-23 09:29:05 +00:00
Mark Johnston
b4a3f67bd6 Add a little helper program for tst.exitcore.ksh.
sleep(1) is capsicumized, which means that we cannot rely on it to dump
core as required by the test.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-05-22 20:34:51 +00:00
Alexander Motin
c9bcbf3800 Fix time handling in cv_timedwait_hires().
pthread_cond_timedwait() receives absolute time, not relative.  Passing
wrong time there caused two threads of zdb to spin in a tight loop.

MFC after:	1 week
2017-05-19 05:12:58 +00:00
Mark Johnston
9ebaf5f802 Remove the EXFAIL annotation for tests which pass as of r309596.
Reported by:	bdrewery
Sponsored by:	Dell EMC Isilon
2017-05-19 00:25:09 +00:00
Josh Paetzel
c78abb8b50 MFV 316894
7252 7628 compressed zfs send / receive

illumos/illumos-gate@5602294fda
5602294fda

https://www.illumos.org/issues/7252
  This feature includes code to allow a system with compressed ARC enabled to
  send data in its compressed form straight out of the ARC, and receive data in
  its compressed form directly into the ARC.

https://www.illumos.org/issues/7628
  We should have longer, more readable versions of the ZFS send / recv options.

7628 create long versions of ZFS send / receive options

Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
Reviewed by: David Quigley <dpquigl@davequigley.com>
Reviewed by: Thomas Caputi <tcaputi@datto.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Dan Kimmel <dan.kimmel@delphix.com>
2017-04-25 17:57:43 +00:00
Josh Paetzel
ef18459108 MFV 316891
7386 zfs get does not work properly with bookmarks

illumos/illumos-gate@edb901aab9
edb901aab9

https://www.illumos.org/issues/7386
  The zfs get command does not work with the bookmark parameter while it works
  properly with both filesystem and snapshot:
  # zfs get -t all -r creation rpool/test
  NAME               PROPERTY  VALUE                  SOURCE
  rpool/test         creation  Fri Sep 16 15:00 2016  -
  rpool/test@snap    creation  Fri Sep 16 15:00 2016  -
  rpool/test#bkmark  creation  Fri Sep 16 15:00 2016  -
  # zfs get -t all -r creation rpool/test@snap
  NAME             PROPERTY  VALUE                  SOURCE
  rpool/test@snap  creation  Fri Sep 16 15:00 2016  -
  # zfs get -t all -r creation rpool/test#bkmark
  cannot open 'rpool/test#bkmark': invalid dataset name
  #
  The zfs get command should be modified to work properly with bookmarks too.

Reviewed by: Simon Klinkert <simon.klinkert@gmail.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Matthew Ahrens <mahrens@delphix.com>
Author: Marcel Telka <marcel@telka.sk>
2017-04-21 19:53:52 +00:00
Alan Somers
07bb15b440 MFV 316855
7900 zdb shouldn't print the path of a znode at verbosity < 5

Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: Matt Ahrens <mahrens@delphix.com>
Approved by: Dan McDonald <danmcd@omniti.com>
Author: Alan Somers <asomers@freebsd.org>

illumos/illumos-gate@e548d2fa41
https://www.illumos.org/issues/7900

MFC after:	3 weeks
Sponsored by:	Spectra Logic Corp
2017-04-14 16:30:37 +00:00
Andriy Gapon
5ad79d9b20 dtrace: fix normalization of stddev aggregation
To be upstreamed.

Discussed with:	Bryan Cantrill <bryancantrill@gmail.com>
MFC after:	2 weeks
Sponsored by:	Panzura
2017-04-14 15:31:04 +00:00
Pedro F. Giffuni
9e98194646 MFV r316693:
8046 Let calloc() do the multiplication in libzfs_fru_refresh

5697e03e6e

https://www.illumos.org/issues/8046

Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Approved by: Robert Mustacchi <rm@joyent.com>
Author:	Pedro Giffuni <pfg@freebsd.org>

MFC after:	3 days
2017-04-10 22:56:38 +00:00
Alexander Motin
3aef5b286a MFV r315290, r315291: 7303 dynamic metaslab selection
illumos/illumos-gate@8363e80ae7
https://github.com/illumos/illumos-gate/commit/8363e80ae72609660f6090766ca8c2c18

https://www.illumos.org/issues/7303

  This change introduces a new weighting algorithm to improve metaslab selection.
  The new weighting algorithm relies on the SPACEMAP_HISTOGRAM feature. As a result,
  the metaslab weight now encodes the type of weighting algorithm used
  (size-based vs segment-based).

  This also introduce a new allocation tracing facility and two new dcmds to help
  debug allocation problems. Each zio now contains a zio_alloc_list_t structure
  that is populated as the zio goes through the allocations stage. Here's an
  example of how to use the tracing facility:

> c5ec000::print zio_t io_alloc_list | ::walk list | ::metaslab_trace
  MSID    DVA    ASIZE      WEIGHT             RESULT               VDEV
     -      0      400           0    NOT_ALLOCATABLE           ztest.0a
     -      0      400           0    NOT_ALLOCATABLE           ztest.0a
     -      0      400           0             ENOSPC           ztest.0a
     -      0      200           0    NOT_ALLOCATABLE           ztest.0a
     -      0      200           0    NOT_ALLOCATABLE           ztest.0a
     -      0      200           0             ENOSPC           ztest.0a
     1      0      400      1 x 8M            17b1a00           ztest.0a

> 1ff2400::print zio_t io_alloc_list | ::walk list | ::metaslab_trace
  MSID    DVA    ASIZE      WEIGHT             RESULT               VDEV
     -      0      200           0    NOT_ALLOCATABLE           mirror-2
     -      0      200           0    NOT_ALLOCATABLE           mirror-0
     1      0      200      1 x 4M            112ae00           mirror-1
     -      1      200           0    NOT_ALLOCATABLE           mirror-2
     -      1      200           0    NOT_ALLOCATABLE           mirror-0
     1      1      200      1 x 4M            112b000           mirror-1
     -      2      200           0    NOT_ALLOCATABLE           mirror-2

  If the metaslab is using segment-based weighting then the WEIGHT column will
  display the number of segments available in the bucket where the allocation
  attempt was made.

Author: George Wilson <george.wilson@delphix.com>
Reviewed by: Alex Reece <alex@delphix.com>
Reviewed by: Chris Siden <christopher.siden@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: Paul Dagnelie <paul.dagnelie@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Don Brady <don.brady@intel.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
2017-03-24 09:37:00 +00:00
Enji Cooper
acc37ca1c1 cddl: normalize paths using SRCTOP-relative paths or :H when possible
This simplifies make logic/output

While here, remove bogus CFLAGS which look for headers in cddl/lib/libumem.
There aren't any source files there (just Makefiles)

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 11:30:04 +00:00
Mark Johnston
d935f34b8f Fix memory leaks in error cases in libdtrace.
Submitted by:	Tom Rix <trix@juniper.net>
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D9705
2017-02-23 17:56:24 +00:00
Mark Johnston
74d9553e43 Fix a memory leak in an error case in libctf.
Submitted by:	Tom Rix <trix@juniper.net>
MFC after:	1 week
2017-02-23 17:54:17 +00:00
Mark Johnston
281e4f2dd4 When patching USDT probes, use non-unique names for aliases of weak symbols.
Aliases are normally given names that include a key that's unique for each
input object file. This, for example, ensures that aliases for identically
named local symbols in different object files don't conflict. However, in
some cases the static linker will leave an undefined alias after merging
identical weak symbols, resulting in a link error. A non-unique name allows
the aliases to be merged as well.

PR:		216871
X-MFC With:	r313262
2017-02-10 02:01:32 +00:00
Mark Johnston
35bf9feb41 Search for _DTRACE_VERSION in sys/sdt.h rather than unistd.h.
MFC after:	1 week
2017-02-05 02:45:35 +00:00
Mark Johnston
55c2fd519f Avoid using Sun compiler-specific flags.
MFC after:	1 week
2017-02-05 02:44:48 +00:00
Mark Johnston
273efb05a2 Fix a double free of libelf data buffers in the USDT link code.
libdtrace needs to append to the input object files' string and symbol
tables. Currently it does so by allocating a larger buffer, copying the
existing sections into them, and swapping pointers in the libelf data
descriptors. However, it also frees those buffers when its processing is
complete, which leads to a double free since the elftoolchain libelf
owns them and also frees them in elf_end(3). Instead, free the buffers
originally allocated by libelf.

MFC after:	2 weeks
2017-02-05 02:44:08 +00:00
Mark Johnston
e801af6fba Use PC-relative relocations for USDT probe sites on i386 and amd64.
When recording probe site addresses in the output DOF file, dtrace -G
needs to emit relocations for the .SUNW_dof section in order to obtain
the addresses of functions containing probe sites. DTrace expects the
addresses to be relative to the base address of the final ELF file,
and the amd64 USDT implementation was relying on some unspecified and
incorrect behaviour in the base system GNU ld to achieve this.

This change reimplements the probe site relocation handling to allow
USDT to be used with lld and newer GNU binutils. Specifically, it
makes use of R_X86_64_PC64/R_386_PC32 relocations to obtain the
probe site address relative to the DOF file address, and adds and uses a
new DOF relocation type which computes the final probe site address using
these relative offsets.

Reported by and discussed with:	Rafael Espíndola
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D9374
2017-02-05 02:39:12 +00:00