Commit Graph

235432 Commits

Author SHA1 Message Date
Conrad Meyer
a04ed7615e stat(1): cache id->name resolution
When invoked on a large list of files, it is most common for a small number of
uids/gids to own most of the results.

Like ls(1), use pwcache(3) to avoid repeatedly looking up the same IDs.

Example microbenchmark and non-scientific results:

$ time (find /usr/src -type f -print0 | xargs -0 stat >/dev/null)

BEFORE:
3.62s user 5.23s system 102% cpu 8.655 total
3.47s user 5.38s system 102% cpu 8.647 total

AFTER:
1.23s user 1.81s system 108% cpu 2.810 total
1.43s user 1.54s system 107% cpu 2.754 total

Does this microbenchmark have any real-world significance?  Until a use case
is demonstrated otherwise, I doubt it.  Ordinarily I would be resistant to
optimizing pointless microbenchmarks in base utilities (e.g., recent totally
gratuitous changes to yes(1)).  However, the pwcache(3) APIs actually
simplify stat(1) logic ever so slightly compared to the raw APIs they wrap,
so I think this is at worst harmless.

PR:		230491
Reported by:	Thomas Hurst <tom AT hur.st>
Discussed with:	gad@
2018-08-11 02:56:43 +00:00
Sevan Janiyan
2bda75506d Fix escaping, otherwise Dx gets translated as the macro for DragonFly.
From 2018 Linuxhotel Hackathon & DevSummit

Approved by:	eadler
Obtained from:	OpenBSD r1.49
Differential Revision:	https://reviews.freebsd.org/D16616
2018-08-11 00:08:59 +00:00
Matt Macy
90df93417e ZFS/MFV: Use cached feature info in spa_add_feature_stats()
commit 417104bdd3
Author: Ned Bass <bass6@llnl.gov>
Date:   Thu Feb 26 12:24:11 2015 -0800

    Use cached feature info in spa_add_feature_stats()

    Avoid issuing I/O to the pool when retrieving feature flags information.
    Trying to read the ZAPs from disk means that zpool clear would hang if
    the pool is suspended and recovery would require a reboot. To keep the
    feature stats resident in memory, we hang a cached nvlist off of the
    spa.  It is built up from disk the first time spa_add_feature_stats() is
    called, and refreshed thereafter using the cached feature reference
    counts. spa_add_feature_stats() gets called at pool import time so we
    can be sure the cached nvlist will be available if the pool is later
    suspended.

    Signed-off-by: Ned Bass <bass6@llnl.gov>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes #3082
2018-08-10 23:42:11 +00:00
Devin Teske
ab9ed8a1bd Fix misspellings of transmitter/transmitted
Reviewed by:	emaste, bcr
Sponsored by:	Smule, Inc.
Differential Revision:	https://reviews.freebsd.org/D16025
2018-08-10 20:37:32 +00:00
Dimitry Andric
33c39ab688 In r308100, an explicit -fexceptions flag was added for the C sources
from LLVM's libunwind, which end up in libgcc_eh.a and libgcc_s.so.
This is because the unwinder needs the unwinder data for its own
functions.

However, for the C++ sources in libunwind, -fexceptions is already the
default, and this can have the side effect of generating a reference to
__gxx_personality_v0, the so-called personality function, which is
normally provided by the C++ ABI library (libcxxrt or libsupc++).

If the reference ends up in the eventual libgcc_s.so, linking any
non-C++ programs against it will fail with "undefined reference to
`__gxx_personality_v0'".

Note that at high optimization levels, the reference is usually
optimized away, which is why we have never noticed this problem before.

With clang 7.0.0 though, higher optimization levels don't help anymore,
since the addition of address-significance tables [1] in
<https://reviews.llvm.org/rL337339>.  Effectively, this always causes a
reference to __gxx_personality_v0.

After discussion with the upstream author of that change, it turns out
that we should compile libunwind sources with the -fno-exceptions
-funwind-tables flags instead.  This ensures unwind tables are
generated, but no references to any personality functions are emitted.

[1] https://lists.llvm.org/pipermail/llvm-dev/2018-May/123514.html

Reported by:	jbeich
PR:		230399
MFC after:	1 week
2018-08-10 19:57:55 +00:00
Mark Johnston
0b56e7a8e9 Disable the D subroutines msgsize() and msgdsize().
They are specific to illumos and the corresponding DIF subroutines are
already disabled on FreeBSD.

Reported by:	gnn
2018-08-10 19:23:20 +00:00
Conrad Meyer
f053ca1f08 Walk back r337554 while discussion continues
The idea was to get the uncontroversial mechanical change out of the way,
then get the meatier functional changes reviewed subsequently.  I had not
realized that the immediately adjacent issue was addressed in a different
direction in r334506 (see Warner's guidance in D15592).

Discussion continues, trying to determine if there is a secondary issue
still[1] and how best to fix it.  With 12-related activities coming up,
while that is ongoing, just take this back for now.

[1]: Shutdown-time eventhandler events fire normally during panic's reboot
path.  Driver callbacks that attempt to issue and wait on interrupt-
completed IO may never complete, hanging the system.  This is particularly
obnoxious in the shutdown/panic path, as the debugger cannot be entered
anymore and the hang prevents reboot restoring availability.

(There's nothing CAM-specific about this problem -- any shutdown
event-triggered driver could do something like this during panic.  But most
NICs, etc.  don't try to send spin-down commands at shutdown. ;-))

Discussed with:	imp, markj
2018-08-10 19:19:07 +00:00
Kyle Evans
0915d9d070 subr_prf: remove think-o that had returned to local patch
Reported by:	cognet
2018-08-10 15:35:02 +00:00
Kyle Evans
170bc29131 boot tagging: minor fixes
msgbufinit may be called multiple times as we initialize the msgbuf into a
progressively larger buffer. This doesn't happen as of now on head, but it
may happen in the future and we generally support this. As such, only print
the boot tag if we've just initialized the buffer for the first time.

The boot tag also now has a newline appended to it for better visibility,
and has been switched to a normal printf, by requesto f bde, after we've
denoted that the msgbuf is mapped.
2018-08-10 15:29:06 +00:00
Warner Losh
3d19db5dfb Update man page to include FreeBSD-specific details.
While this implements a standards-conforming C11 function, there's
implementation details the programmer needs to know. Include those
here. Make changes inspired by comments on the initial review as well,
though mostly this involves stealing the epoch verbage from
gettimeofday(2). Add myself to authors since I've now changed a
substantial amount of this man page.
2018-08-10 15:16:41 +00:00
Warner Losh
de1118b1dd Remove assert.h and commented out _DIAGASSERT.
Remove assert.h and _DIAGASSERT to create a paper-trail of changes
from NetBSD. Specifically didn't fix other style issues since I
don't want this to diverge from the NetBSD original too much and
that's too niggling a change to be worth future merge hassles.

Differential Review: https://reviews.freebsd.org/D16649
2018-08-10 15:16:36 +00:00
Warner Losh
7e299411ac Bring in timespce_get form NetBSD.
Bring in the functionality for timespec_get from NetBSD. I've lightly
edited the .c file to remove _DIAGASSERT because FreeBSD doesn't have
that functionality and the typical #define'ing it to assert isn't
right here. The man page is verbatim from NetBSD, but will be revised
as part of a larger cleanup of the time man pages (they are
inconsistent and vague in all the wrong places).

Differential Review: https://reviews.freebsd.org/D16649
2018-08-10 15:16:30 +00:00
Andrey V. Elsukov
f4d5e7d8b5 Restore the behaviour changed in r337536, when bad ipfw delete command
returns error.

Now -q option only makes it quiet. And when -f flag is specified, the
command will ignore errors and continue executing with next batched
command.

MFC after:	2 weeks
2018-08-10 14:10:22 +00:00
Kyle Evans
84c956df77 ath: Minor style cleanups
device_printf => DPRINTF and two whitespace adjustments

Submitted by:	Augustin Cavalier <waddlesplash@gmail.com>
Obtained from:	Haiku (4a88aa503ad4155a20931e263d24343043994ea9)
MFC after:	1 week
2018-08-10 13:38:23 +00:00
Kyle Evans
8e0cc51b87 ieee8021_node: fix whitespace issues
Submitted by:	Augustin Cavalier <waddlesplash@gmail.com>
Obtained from:	Haiku (dffc3e235360cd7b71261239ee8507b7d62a1471)
MFC after:	1 week
2018-08-10 13:34:23 +00:00
Kyle Evans
58a7c4bfcf net80211: Drain ageq before cleaning it up.
The comment above ieee80211_ageq_cleanup specifically notes that the queue
is assumed to be empty, and in order to make it so, ieee80211_ageq_drain
must be used.

Submitted by:	Augustin Cavalier <waddlesplash@gmail.com>
Obtained from:	Haiku (dffc3e235360cd7b71261239ee8507b7d62a1471)
MFC after:	1 week
2018-08-10 13:32:02 +00:00
Kyle Evans
060b3e4ff1 bwi(4): Set ic->ic_softc before bwi_getradiocaps to avoid bad deref
Submitted by:	François Revol <revol@free.fr>
Obtained from:	Haiku (ba88131cfde64e21bedb4ebedd699cfa5e7fd314)
MFC after:	1 week
2018-08-10 13:06:14 +00:00
Ed Maste
ce0c6340ec readelf: display NT_GNU_PROPERTY_TYPE_0 note name
NT_GNU_PROPERTY_TYPE_0 in a .note.gnu.property section "contains a
program property note which describes special handling requirements
for linker and run-time loader." (from the System V Application Binary
Interface - Linux Extensions")

Intel CET uses two processor-specific program properties in
NT_GNU_PROPERTY_TYPE_0: GNU_PROPERTY_X86_FEATURE_1_IBT to indicate that
all executable sections are compatible with Indirect Branch Tracking,
and GNU_PROPERTY_X86_FEATURE_1_SHSTK to indicate that sections are
compatible with shadow stack.

A later change should add decoding of the individual properties.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2018-08-10 10:37:25 +00:00
Andrey V. Elsukov
16bbf600d9 Remove unneeded ipsec-related includes.
Reviewed by:	rrs
Differential Revision:	https://reviews.freebsd.org/D16637
2018-08-10 07:24:01 +00:00
Matt Macy
648cfe57fd Performance optimization of AVL tree comparator functions
MFV:
commit ee36c709c3
Author: Gvozden Neskovic <neskovic@gmail.com>
Date:   Sat Aug 27 20:12:53 2016 +0200

    perf: 2.75x faster ddt_entry_compare()
        First 256bits of ddt_key_t is a block checksum, which are expected
    to be close to random data. Hence, on average, comparison only needs to
    look at first few bytes of the keys. To reduce number of conditional
    jump instructions, the result is computed as: sign(memcmp(k1, k2)).

    Sign of an integer 'a' can be obtained as: `(0 < a) - (a < 0)` := {-1, 0, 1} ,
    which is computed efficiently.  Synthetic performance evaluation of
    original and new algorithm over 1G random keys on 2.6GHz Intel(R) Xeon(R)
    CPU E5-2660 v3:

    old     6.85789 s
    new     2.49089 s

    perf: 2.8x faster vdev_queue_offset_compare() and vdev_queue_timestamp_compare()
        Compute the result directly instead of using conditionals

    perf: zfs_range_compare()
        Speedup between 1.1x - 2.5x, depending on compiler version and
    optimization level.

    perf: spa_error_entry_compare()
        `bcmp()` is not suitable for comparator use. Use `memcmp()` instead.

    perf: 2.8x faster metaslab_compare() and metaslab_rangesize_compare()
    perf: 2.8x faster zil_bp_compare()
    perf: 2.8x faster mze_compare()
    perf: faster dbuf_compare()
    perf: faster compares in spa_misc
    perf: 2.8x faster layout_hash_compare()
    perf: 2.8x faster space_reftree_compare()
    perf: libzfs: faster avl tree comparators
    perf: guid_compare()
    perf: dsl_deadlist_compare()
    perf: perm_set_compare()
    perf: 2x faster range_tree_seg_compare()
    perf: faster unique_compare()
    perf: faster vdev_cache _compare()
    perf: faster vdev_uberblock_compare()
    perf: faster fuid _compare()
    perf: faster zfs_znode_hold_compare()

    Signed-off-by: Gvozden Neskovic <neskovic@gmail.com>
    Signed-off-by: Richard Elling <richard.elling@gmail.com>
    Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
    Closes #5033
2018-08-10 06:42:08 +00:00
Poul-Henning Kamp
a34b3be4c2 Make distribution now happens from top of source tree.
Silence debugging output
2018-08-10 06:28:25 +00:00
Justin Hibbits
7d849dc1a4 powerpc: Add lwsync and ptesync 'sync' opcode variants to ddb disassembler
The canonical form of sync is:

  sync L, E (if Category Elemental Memory Barriers implemented)

The L bits (2) denote the type of sync:

  0 -- hwsync
  1 -- lwsync
  2 -- ptesync or hwsync

It's been found that most 32-bit CPUs designed prior to the introduction of
lwsync will ignore the L bits.  However, some cores, particularly the e500 core,
will trigger an illegal instruction exception.  Adding these variants will make
it easier to see which sync variant is actually being used in case of a trap.
2018-08-10 03:28:40 +00:00
Cy Schubert
79476a1c3e Correct a comment. Should have been detected by ipf_nat_in() not
ipf_nat_out().

MFC after:	1 week
X-MFC-with:	r337558
2018-08-10 00:30:15 +00:00
Kyle Evans
2b720021aa Makefile.inc1: Add libl to -legacy as well
libl is needed for config(8), which is a bootstrap-tool. It is possible to
build a system WITHOUT_TOOLCHAIN to exclude lex and thus, libl. We still
need to support building from this kind of host, though.

While here, group the config(8) dependencies together and add a small
explanation. These can likely both be scoped more clearly, but this will
need some further investigation.

Reported by:	rgrimes (not WITHOUT_TOOLCHAIN, but provoked investigation)
MFC after:	immediately
2018-08-10 00:10:57 +00:00
Cy Schubert
e6191e11f0 Identify the return value (rval) that led to the IPv4 NAT failure
in ipf_nat_checkout() and report it in the frb_natv4out and frb_natv4in
dtrace probes.

This is currently being used to diagnose NAT failures in PR/208566. It's
rather handy so this commit makes it available for future diagnosis and
debugging efforts.

PR:		208566
MFC after:	1 week
2018-08-10 00:04:32 +00:00
Glen Barber
b534d57f63 Rename head from -CURRENT to -ALPHA1 as part of the
12.0-RELEASE cycle.  This commit marks the start of
the code slush for the 12.0 cycle.

Approved by:	re (implicit)
Sponsored by:	The FreeBSD Foundation
2018-08-10 00:01:21 +00:00
Glen Barber
6ac4304a0f Invoke the growfs rc script for each boot on GCE.
PR:		230275
Submitted by:	gustavo.scalet@collabora.com
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-08-09 23:43:10 +00:00
Glen Barber
0366e18e5a Update and replace old rc daemons for GCE images.
PR:		229000
Submitted by:	helen.koike@collabora.com
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-08-09 23:31:18 +00:00
Conrad Meyer
2077be2b73 cam(4): Add an xpt-neutral flag indicating a valid panic CCB
No functional change.

Note that this change is careful to set the CCB header xflags after
foo_fill_bar() routines, which generally zero existing flags.  An earlier
version of this patch mistakenly set the flag before the fill routines.

Submitted by:	Scott Ferris <sferris AT isilon.com>, jhibbits@
Reviewed by:	bdrewery@, markj@, and non-committer FreeBSD contributor Anton Rang
Sponsored by:	Dell EMC Isilon
2018-08-09 21:53:32 +00:00
Navdeep Parhar
2d73ac5e4a cxgbe(4): Add a sysctl to control the tx credit reclaim mechanism for
netmap tx queues.  There is no change in default behavior.

Sponsored by:	Chelsio Communications
2018-08-09 21:52:51 +00:00
Dimitry Andric
89edb881e6 Add optional LLVM BPF target support
BPF (eBPF) is an independent instruction set architecture which is
introduced in Linux a few years ago. Originally, eBPF execute
environment was only inside Linux kernel. However, recent years there
are some user space implementation (https://github.com/iovisor/ubpf,
https://doc.dpdk.org/guides/prog_guide/bpf_lib.html) and kernel space
implementation for FreeBSD is going on
(https://github.com/YutaroHayakawa/generic-ebpf).

The BPF target support can be enabled using WITH_LLVM_TARGET_BPF, as it
is not built by default.

Submitted by:	Yutaro Hayakawa <yhayakawa3720@gmail.com>
Reviewed by:	dim, bdrewery
Differential Revision: https://reviews.freebsd.org/D16033
2018-08-09 21:28:31 +00:00
Conrad Meyer
bc812246a0 cam_ccb.h: Remove redundant declarations of static inline functions
No functional change.

They're unnecessarily confusing for tools like grep or ctags.

Sponsored by:	Dell EMC Isilon
2018-08-09 21:20:07 +00:00
Navdeep Parhar
518bca2c21 cxgbe(4): Set fl_pktshift to 0 by default.
Sponsored by:	Chelsio Communications
2018-08-09 21:07:32 +00:00
Kyle Evans
50a2c22918 libnv: Remove -I${SRCTOP}/sys
This should have been done as part of r336019 -- including ${SRCTOP}/sys is
not a good business model for something that's build in legacy/bootstrap
stages.

Beyond that, libnv seems to build quite alright as legacy, part of
buildworld, and standalone without. Axe it.

Reported by:	truckman (head building stable/11)
Tested by:	Shawn Webb (HardenedBSD)
MFC after:	3 days
2018-08-09 20:29:44 +00:00
Kyle Evans
240fcda1e8 subr_prf: style(9) the sizeof
Reported by:	jkim, ian
2018-08-09 19:09:06 +00:00
Mark Johnston
b50a4ea646 Account for the lowmem handlers in the inactive queue scan target.
Before r329882 the target would be computed after lowmem handlers run
and free pages.  On some systems a significant amount of page
reclamation happens this way.  However, with r329882 the target is
computed first, which can lead to unnecessary reclamation from the
page cache, and this in turn may result in excessive swapping.

Instead, adjust the target after running lowmem handlers.  Don't
invoke the lowmem handlers before the PID controller, though, since
that would hide the true rate of page allocation.

Reviewed by:	alc, kib (previous version)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D16606
2018-08-09 18:25:49 +00:00
Kyle Evans
4c793b68da subr_prf: Use "sizeof current_boot_tag" instead 2018-08-09 17:53:18 +00:00
Kyle Evans
2a4650cc11 BOOT_TAG: Make a config(5) option, expose as sysctl and loader tunable
BOOT_TAG lived shortly in sys/msgbuf.h, but this wasn't necessarily great
for changing it or removing it. Move it into subr_prf.c and add options for
it to opt_printf.h.

One can specify both the BOOT_TAG and BOOT_TAG_SZ (really, size of the
buffer that holds the BOOT_TAG). We expose it as kern.boot_tag and also add
a loader tunable by the same name that we'll fetch upon initialization of
the msgbuf.

This allows for flexibility and also ensures that there's a consistent way
to figure out the boot tag of the running kernel, rather than relying on
headers to be in-sync.

Prodded super-super-lightly by:	imp
2018-08-09 17:47:47 +00:00
Kyle Evans
21aa6e8345 msgbuf: Light detailing (const'ify and bool'itize) 2018-08-09 17:42:27 +00:00
John Baldwin
a6bb5342e3 Correct default path of kernel modules. 2018-08-09 16:42:13 +00:00
Navdeep Parhar
8a684e1fd1 cxgbe(4): Display pkt-size and burst-size in traffic class parameters. 2018-08-09 14:36:44 +00:00
Navdeep Parhar
6ba813531f cxgbetool(8): Userspace part of support for high priority filters on T6+.
MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-08-09 14:21:27 +00:00
Navdeep Parhar
5fc0f72f3b cxgbe(4): Add support for high priority filters on T6+. They have their
own region in the TCAM starting with T6, unlike previous chips where
they were in the same region as normal filters.

These filters "hit" before anything else in the LE's lookup.  The exact
order is:
a) High priority filters
b) TOE's active region (TCAM and/or hash)
c) Servers (TOE hw listeners)
d) Normal filters

MFC after:	1 week
Sponsored by:	Chelsio Communications
2018-08-09 14:19:47 +00:00
Leandro Lupori
c8e2123b6a [ppc] Fix kernel panic when using BOOTP_NFSROOT
On PowerPC (and possibly other architectures), that doesn't use
EARLY_AP_STARTUP, the config task queue may be used initialized.
This was observed while trying to mount the root fs from NFS, as
reported here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230168.

This patch has 2 main changes:
1- Perform a basic initialization of qgroup_config, similar to
what is done in taskqgroup_adjust, but simpler.
This makes qgroup_config ready to be used during NFS root mount.

2- When EARLY_AP_STARTUP is not used, call inm_init() and
in6m_init() right before SI_SUB_ROOT_CONF, because bootp needs
to send multicast packages to request an IP.

PR:		Bug 230168
Reported by:	sbruno
Reviewed by:	jhibbits, mmacy, sbruno
Approved by:	jhibbits
Differential Revision:	D16633
2018-08-09 14:04:51 +00:00
Andrey V. Elsukov
e327ad3377 If -q flag is specified, do not complain when we are trying to delete
nonexistent NAT instance or nonexistent rule.

This allows execute batched `delete` commands and do not fail when
found nonexistent rule.

Obtained from:	Yandex LLC
MFC after:	2 weeks
Sponsored by:	Yandex LLC
2018-08-09 12:46:30 +00:00
Edward Tomasz Napierala
d1b1fe3a25 Use NULLs instead of casted zeroes, for consistency.
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2018-08-09 12:17:03 +00:00
Edward Tomasz Napierala
f3c4a698df Refactor common code into execute_script().
Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D16627
2018-08-09 12:13:08 +00:00
Olivier Houchard
d8f1ed8d94 Import CK as of commit 08813496570879fbcc2adcdd9ddc0a054361bfde, mostly
to avoid using lwsync on ppc32.
2018-08-09 12:11:49 +00:00
Olivier Houchard
449aa8c641 Import CK as of commit 08813496570879fbcc2adcdd9ddc0a054361bfde, mostly
to avoid using lwsync on ppc32.
2018-08-09 12:07:37 +00:00
Edward Tomasz Napierala
308224833a Make ldconfig(8) atomic, by removing an unneccessary call to unlink(2)
before rename(2).

Reviewed by:	kib
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D16641
2018-08-09 11:46:12 +00:00