Commit Graph

254022 Commits

Author SHA1 Message Date
kevans
2f561e90ad getopt_long(3): Document behavior of leading characters in optstring
Leading '+', '-', and ':' in optstring have special meaning. We briefly
mention that the first two have special meaning in that we say
POSIXLY_CORRECT turns them off, but we don't actually document their
meaning. Add a paragraph to RETURN VALUES explaining how they control
the treatment of non-option arguments.

A leading ':' has no mention; add a note that it suppresses warnings about
missing arguments.

Reviewed by:	jilles
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14142
2018-08-12 00:08:14 +00:00
kevans
68da6ee2cf krb5-config build: Remove gratuitous escaping
MFC after:	1 week
2018-08-12 00:06:21 +00:00
kevans
0a437023fb bectl(8): Rename "index" variable, which shadows a global in some lands 2018-08-12 00:00:13 +00:00
kevans
c506761767 Merge libbe(3)/bectl(8) from projects/bectl into head
bectl(8) is an administrative interface for working with ZFS boot
environments, intended to provide a superset of the functionality provided
by sysutils/beadm.

libbe(3) is the back-end library that the required functionality has been
pulled out into for later reuse.

These were originally written for GSoC 2017 under the mentorship of
allanjude@.

bectl(8) has proven pretty stable in my testing, with the known bug
documented in the man page.

Relnotes:	yes
2018-08-11 23:50:09 +00:00
kevans
511c25b220 libbe(3)/bectl(8): More SYSROOT/GCC build fixes
- Missing include path
- Fully specify libzfs's dependencies (except for deps pulled in by other
  deps) in Makefile.inc1
- Drop WARNS back down to 2 for libbe(3). I do this with much hesitation,
  but the libzfs headers are apparently a hot warning-filled mess as far as
  GCC 4.2 is concerned.
2018-08-11 22:45:39 +00:00
mmacy
b2672ceb1a fix top after arc accounting change 2018-08-11 22:11:12 +00:00
mmacy
df5b1f7f1d Enable balanced arc pruning
Taken from:
ommit f6046738365571bd647f804958dfdff8a32fbde4
Author: Brian Behlendorf <behlendorf1@llnl.gov>
Date:   Sat May 30 09:57:53 2015 -0500

    Make arc_prune() asynchronous

    As described in the comment above arc_adapt_thread() it is critical
    that the arc_adapt_thread() function never sleep while holding a hash
    lock.  This behavior was possible in the Linux implementation because
    the arc_prune() logic was implemented to be synchronous.  Under
    illumos the analogous dnlc_reduce_cache() function is asynchronous.

    To address this the arc_do_user_prune() function is has been reworked
    in to two new functions as follows:

    * arc_prune_async() is an asynchronous implementation which dispatches
    the prune callback to be run by the system taskq.  This makes it
    suitable to use in the context of the arc_adapt_thread().

    * arc_prune() is a synchronous implementation which depends on the
    arc_prune_async() implementation but blocks until the outstanding
    callbacks complete.  This is used in arc_kmem_reap_now() where it
    is safe, and expected, that memory will be freed.

    This patch additionally adds the zfs_arc_meta_strategy module option
    while allows the meta reclaim strategy to be configured.  It defaults
    to a balanced strategy which has been proved to work well under Linux
    but the illumos meta-only strategy can be enabled.

    Signed-off-by: Tim Chase <tim@chase2k.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
2018-08-11 22:01:52 +00:00
np
4c0dd92f54 cxgbe(4): Move all control queues to the adapter.
There used to be one control queue per adapter (the mgmtq) that was
initialized during adapter init and one per port that was initialized
later during port init.  This change moves all the control queues (one
per port/channel) to the adapter so that they are initialized during
adapter init and are available before any port is up.  This allows the
driver to issue ctrlq work requests over any channel without having to
bring up any port.

MFH:		2 weeks
Sponsored by:	Chelsio Communications
2018-08-11 21:10:08 +00:00
netchild
66f3a7f644 Extend the info about the limitations of datasets in jails.
Reviewed by:	allanjude
Sponsored by:	Essen Hackathon
2018-08-11 20:49:19 +00:00
kevans
1ccb08bdae bectl(8): Add missing include path 2018-08-11 20:47:35 +00:00
mmacy
23b616b6bc buildworld fix: private appears to have special meaning on FreeBSD - revert to priv 2018-08-11 20:41:42 +00:00
kevans
55d1987652 Add libzfs to prebuild_libs, libbe dependency on libzfs 2018-08-11 20:32:50 +00:00
mmacy
4ff3104245 Limit the amount of dnode metadata in the ARC
In addition import most recent arc_prune_async implementation as dependency

commit 25458cbef9e59ef9ee6a7e729ab2522ed308f88f
Author: Tim Chase <tim@chase2k.com>
Date:   Wed Jul 13 07:42:40 2016 -0500

    Limit the amount of dnode metadata in the ARC

    Metadata-intensive workloads can cause the ARC to become permanently
    filled with dnode_t objects as they're pinned by the VFS layer.
    Subsequent data-intensive workloads may only benefit from about
    25% of the potential ARC (arc_c_max - arc_meta_limit).

    In order to help track metadata usage more precisely, the other_size
    metadata arcstat has replaced with dbuf_size, dnode_size and bonus_size.

    The new zfs_arc_dnode_limit tunable, which defaults to 10% of
    zfs_arc_meta_limit, defines the minimum number of bytes which is desirable
    to be consumed by dnodes.  Attempts to evict non-metadata will trigger
    async prune tasks if the space used by dnodes exceeds this limit.

    The new zfs_arc_dnode_reduce_percent tunable specifies the amount by
    which the excess dnode space is attempted to be pruned as a percentage of
    the amount by which zfs_arc_dnode_limit is being exceeded.  By default,
    it tries to unpin 10% of the dnodes.

    The problem of dnode metadata pinning was observed with the following
    testing procedure (in this example, zfs_arc_max is set to 4GiB):

        - Create a large number of small files until arc_meta_used exceeds
          arc_meta_limit (3GiB with default tuning) and arc_prune
          starts increasing.

        - Create a 3GiB file with dd.  Observe arc_mata_used.  It will still
          be around 3GiB.

        - Repeatedly read the 3GiB file and observe arc_meta_limit as before.
          It will continue to stay around 3GiB.

    With this modification, space for the 3GiB file is gradually made
    available as subsequent demands on the ARC are made.  The previous behavior
    can be restored by setting zfs_arc_dnode_limit to the same value as the
    zfs_arc_meta_limit.

    Signed-off-by: Tim Chase <tim@chase2k.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Issue #4345
    Issue #4512
    Issue #4773
    Closes #4858
2018-08-11 19:45:04 +00:00
alc
d20c62f3cb Eliminate a redundant assignment.
MFC after:	1 week
2018-08-11 19:21:53 +00:00
pstef
0102f00aa4 indent(1): revert r334640 and r334632
While STACKSIZE macro is indeed problematic on some systems, the commits
were wrong to shrink il[] and cstk[], because they need to be of the same
size as p_stack[] as they're accessed with the same index ps.tos.
2018-08-11 19:20:06 +00:00
brd
db59dff474 Move all NTP related files to usr.sbin/ntp/ntpd.
This helps with pkgbase by using CONFS to tag these as config files.

Approved by:	allanjude (mentor), ian, cy
Sponsored by:	Essen Hackathon
Differential Revision:	https://reviews.freebsd.org/D16661
2018-08-11 17:42:42 +00:00
brd
13ea1f5be7 Move all periodic related config and scripts to usr.sbin/periodic/
This makes pkgbase easier by tagging these as CONFS so they are properly
tagged as config files.

Approved by:	will (mentor)
Sponsored by:	Essen Hackathon
Differential Revision:	https://reviews.freebsd.org/D16553
2018-08-11 17:11:08 +00:00
kp
4af4c25f85 pf tests: Basic test for 'set skip in $groupname'
This tests for the problem reported in PR 229241, where using a group
name in 'set skip on' did not work as expected.

Sponsored by:	Essen Hackathon
2018-08-11 16:41:07 +00:00
kp
264bf696ba pf: Take the IF_ADDR_RLOCK() when iterating over the group list
We did do this elsewhere in pf, but the lock was missing here.

Sponsored by:	Essen Hackathon
2018-08-11 16:37:55 +00:00
kp
7c0a39f95f pf: Fix 'set skip on' for groups
The pfi_skip_if() function sometimes caused skipping of groups to work,
if the members of the group used the groupname as a name prefix.
This is often the case, e.g. group lo usually contains lo0, lo1, ...,
but not always.

Rather than relying on the name explicitly check for group memberships.

Obtained from:	OpenBSD (pf_if.c,v 1.62, pf_if.c,v 1.63)
Sponsored by:	Essen Hackathon
2018-08-11 16:34:30 +00:00
netchild
f21ca1ae2f - Correct the description when jobs are executed related to load avg
to match reality (slightly different to what was submitted in the
   PR: use english word instead of math-symbol).
 - Wrap the corresponding part to below 80 characters per line.

Submitted by:	yamagi@yamagi.org
PR:		202202
Sponsored by:	Essen Hackathon
2018-08-11 16:29:54 +00:00
netchild
ff2edebaa2 Re-enable reading byte swapped NFS_MAGIC dumps.
Fix bug introduced in r98542: previously to this revision the byte-swapped
value was compared at this place. The current check is in a conditional
section where the non-byte-swapped value was already checked to be not
the value which is checked again. As byte-swapping is activated afterwards,
it only makes sense if the byte-swapped value is checked.

Submitted by:	Keith White <kwhite@site.uottawa.ca>
PR:		200059
MFC after:	1 month
Sponsored by:	Essen Hackathon
2018-08-11 16:12:23 +00:00
brd
db6353bafb Fix the build by just installing systop since testing shows it works with:
dwatch -X systop

Reviewed by:	kp
Approved by:	allanjude (mentor)
2018-08-11 16:06:32 +00:00
kevlo
80d86d6706 Remove unused MAPDESCFILE. 2018-08-11 14:45:33 +00:00
brd
83e33a2658 Move pf.os to sbin/pfctl/
Approved by:	will (mentor)
Glanced at by:	kp
Sponsored by:	Essen Hackathon
Differential Revision:	https://reviews.freebsd.org/D16557
2018-08-11 13:58:26 +00:00
brd
fd12366602 Move cron.d/at to usr.bin/at/
This helps with pkgbase as it tags this as a config file so it is handled as
such

Approved by:	allanjude (mentor)
Sponsored by:	Essen Hackathon
Differential Revision:	https://reviews.freebsd.org/D16673
2018-08-11 13:52:23 +00:00
brd
dec06dc11b Move snmpd.config to usr.sbin/bsnmpd/bsnmpd/
This helps with pkgbase as this config file will now be tagged as a config
file

Approved by:	allanjude (mentor)
Sponsored by:	Essen Hackathon
Differential Revision:	https://reviews.freebsd.org/D16674
2018-08-11 13:47:28 +00:00
brd
a4dc1e4521 Move sysctl.conf to sbin/sysctl/ and switch to CONFS.
This helps with pkgbase to tag this config file as a config file.

Approved by:	allanjude (mentor), will (mentor)
Differential Revision:	https://reviews.freebsd.org/D16559
2018-08-11 13:28:03 +00:00
brd
4721a2ec2e Move ddb.conf to sbin/ddb/ and switch to CONFS.
This helps pkgbase as this config file will now be tagged as a config file.

Approved by:	allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D16675
2018-08-11 13:25:39 +00:00
brd
624bc1cd57 Move OpenBSM to CONFS
This helps with pkgbase as these config files will be properly tagged as
config files.

Approved by:	allanjude (mentor), oshogbo
Differential Revision:	https://reviews.freebsd.org/D16679
2018-08-11 13:23:09 +00:00
netchild
48a4ff6884 Add svnlite to places where svn is mentioned.
The Makefile part in the PR is solved already differently, so this
part is skipped form the PR The man page change change is slightly
changed to adapt to the way the Makefile works and to the spirit
of what is intended here.

Submitted by:	Juan Ramón Molina Menor <info@juanmolina.eu>
PR:		194910
Sponsored by:	Essen Hackathon
2018-08-11 13:18:19 +00:00
netchild
c592d5bd90 Add "ESI Juli@ XTe" as a supported device.
Submitted by:	Vladislav Movchan  <vladislav.movchan@gmail.com>
PR:		222025
Sponsored by:	Essen Hackathon
2018-08-11 13:01:46 +00:00
jilles
2259773a12 printf: Fix \c in %b in printf builtin exiting the shell after r337458
SVN r337458 erroneously partially reverted r265885.

This is immediately visible when running the Kyua/ATF tests for
usr.bin/printf, which actually test sh's printf builtin.

PR:		229641
2018-08-11 11:13:34 +00:00
sevan
012b3d1c50 IEEE!
Pointy hat: 	myself
2018-08-11 11:05:22 +00:00
dim
56b9b447a3 Pull in r338481 from upstream llvm trunk (by Chandler Carruth):
[x86] Fix a really subtle miscompile due to a somewhat glaring bug in
  EFLAGS copy lowering.

  If you have a branch of LLVM, you may want to cherrypick this. It is
  extremely unlikely to hit this case empirically, but it will likely
  manifest as an "impossible" branch being taken somewhere, and will be
  ... very hard to debug.

  Hitting this requires complex conditions living across complex
  control flow combined with some interesting memory (non-stack)
  initialized with the results of a comparison. Also, because you have
  to arrange for an EFLAGS copy to be in *just* the right place, almost
  anything you do to the code will hide the bug. I was unable to reduce
  anything remotely resembling a "good" test case from the place where
  I hit it, and so instead I have constructed synthetic MIR testing
  that directly exercises the bug in question (as well as the good
  behavior for completeness).

  The issue is that we would mistakenly assume any SETcc with a valid
  condition and an initial operand that was a register and a virtual
  register at that to be a register *defining* SETcc...

  It isn't though....

  This would in turn cause us to test some other bizarre register,
  typically the base pointer of some memory. Now, testing this register
  and using that to branch on doesn't make any sense. It even fails the
  machine verifier (if you are running it) due to the wrong register
  class. But it will make it through LLVM, assemble, and it *looks*
  fine... But wow do you get a very unsual and surprising branch taken
  in your actual code.

  The fix is to actually check what kind of SETcc instruction we're
  dealing with. Because there are a bunch of them, I just test the
  may-store bit in the instruction. I've also added an assert for
  sanity that ensure we are, in fact, *defining* the register operand.
  =D

Noticed by:	kib
MFC after:	1 week
2018-08-11 10:42:12 +00:00
sevan
e51195670b Drop the ternary operator for calculating ssid display length in list_scan().
Regardless if a verbose scan is required or not, we'd still want to display the
full SSID name by default so use the IEE80211_NWID_LEN constant to set the
value to use instead.

Tested on rene@'s laptop.
Reviewed by:	kp
Sponsored by:	Essen Hackathon
Differential Revision:	https://reviews.freebsd.org/D16566
2018-08-11 10:21:21 +00:00
sevan
cceac591b0 Advise reader to also see mdconfig(8) in mount_cd9660(8).
It's useful for how to mount an iso file via loopback.

Reviewed by:	jilles
Approved by:	bcr (mentor)
Differential Revision:	https://reviews.freebsd.org/D16067
2018-08-11 08:34:24 +00:00
dteske
a22d299259 dwatch(1): Add systop profile
Provides a top-like view of syscall consumers.

MFC after:	3 days
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-08-11 06:32:31 +00:00
dteske
17c4940b00 dwatch(1): Fix syntax error in vop_readdir profile
Reported by:	Arne Ehrlich <ehrlich@consider-it.de>
MFC after:	3 days
X-MFC-to:	stable/11
Sponsored by:	Smule, Inc.
2018-08-11 06:13:11 +00:00
np
8758bd5749 cxgbe(4): Create two variants of service_iq, one for queues with
freelists and one for those without.

MFH:		3 weeks
Sponsored by:	Chelsio Communications
2018-08-11 04:55:47 +00:00
kevans
f25fa3bdae MFH @ r337607, in preparation for boarding 2018-08-11 04:26:29 +00:00
kevans
de698e5e9c Destroy a couple of rogue svn:mergeinfo 2018-08-11 04:23:03 +00:00
kevans
305e8c5705 bectl(8): Also document the import bug. 2018-08-11 04:11:34 +00:00
kevans
4b0660c4b9 libbe(3): Document the import bug... 2018-08-11 04:09:42 +00:00
kevans
b3363cc0c9 libbe(3): Add to bsd.libnames.mk 2018-08-11 04:07:00 +00:00
kevans
1498f9e1e4 bectl(8): Swap the order of ujail/umount documentation...
"Despite my inconsistency, consistency is king." -Anonymous
2018-08-11 04:03:05 +00:00
kevans
b6faf72889 bectl(8): Document export/import 2018-08-11 04:01:24 +00:00
kevans
ece0932ee0 libbe(3): Brain dump... 2018-08-11 03:33:10 +00:00
cem
36a09a6cd5 stat(1): cache id->name resolution
When invoked on a large list of files, it is most common for a small number of
uids/gids to own most of the results.

Like ls(1), use pwcache(3) to avoid repeatedly looking up the same IDs.

Example microbenchmark and non-scientific results:

$ time (find /usr/src -type f -print0 | xargs -0 stat >/dev/null)

BEFORE:
3.62s user 5.23s system 102% cpu 8.655 total
3.47s user 5.38s system 102% cpu 8.647 total

AFTER:
1.23s user 1.81s system 108% cpu 2.810 total
1.43s user 1.54s system 107% cpu 2.754 total

Does this microbenchmark have any real-world significance?  Until a use case
is demonstrated otherwise, I doubt it.  Ordinarily I would be resistant to
optimizing pointless microbenchmarks in base utilities (e.g., recent totally
gratuitous changes to yes(1)).  However, the pwcache(3) APIs actually
simplify stat(1) logic ever so slightly compared to the raw APIs they wrap,
so I think this is at worst harmless.

PR:		230491
Reported by:	Thomas Hurst <tom AT hur.st>
Discussed with:	gad@
2018-08-11 02:56:43 +00:00
kevans
47995699a5 Add libbe(3) to mdoc 2018-08-11 01:54:45 +00:00