Commit Graph

250925 Commits

Author SHA1 Message Date
dteske
18fa609dff dwatch(1): Allow `-E code' to override profile EVENT_DETAILS
This allows quick changes to the formatted output of a profile.

Sponsored by:	Smule, Inc.
2018-05-12 06:01:41 +00:00
dteske
54142a8787 dwatch(1): Export ARGV to profiles loaded via load_profile()
A module that wishes to post-process the output needs to know which
arguments were passed in order to re-execute a child in a pipe-chain.
Further, the expansion of ARGV needs to be such that items are escaped
properly.

Sponsored by:	Smule, Inc.
2018-05-12 05:49:31 +00:00
dteske
88c834f5de dwatch(1): Simplify info message test
The info() function already tests $QUIET

Sponsored by:	Smule, Inc.
2018-05-12 05:43:47 +00:00
dteske
6fd93270fd dwatch(1): Separate default values so `-[BK] num' don't affect usage
If you were to pass an invalid option after `-B num' or `-K num' you
would see that the usage statement would show the value you passed
instead of the actual default.

Moving the default values to separate variables that are unaffected
by the options-parsing allows the usage statement to correctly show
the hard-coded default values if no flags are used.

Sponsored by:	Smule, Inc.
2018-05-12 05:41:28 +00:00
dteske
96dfb00139 dwatch(1): Bugfix, usage displayed with `-1Q'
A return statement should have been an exit in list_profiles().
If the user passed `-Q' to list profiles and asked for one-line
per profile (`-1'), list_profiles() would not exit as should.

Sponsored by:	Smule, Inc.
2018-05-12 05:36:47 +00:00
mmacy
dcb7d046f9 hwpmc(9): clear remaining sample work for hardclock
- fix last minute change in 333509 where by runcount references
  to a pmc would remaining causing us to pause loop forever

Approved by:	sbruno
2018-05-12 03:45:30 +00:00
imp
56cab5772f Remove extra copy of bcopy.c now that we're using the libkern version
of this file.
2018-05-12 01:43:32 +00:00
mmacy
2981a3420c hwpmc(9): Make pmclog buffer pcpu and update constants
On non-trivial SMP systems the contention on the pmc_owner mutex leads
to a substantial number of samples captured being from the pmc process
itself. This change a) makes buffers larger to avoid contention on the
global list b) makes the working sample buffer per cpu.

Run pmcstat in the background (default event rate of 64k):
pmcstat -S UNHALTED_CORE_CYCLES -O /dev/null sleep 600 &

Before:
make -j96 buildkernel -s >&/dev/null 3336.68s user 24684.10s system 7442% cpu 6:16.50 total

After:
make -j96 buildkernel -s >&/dev/null 2697.82s user 1347.35s system 6058% cpu 1:06.77 total

For more realistic overhead measurement set the sample rate for ~2khz
on a 2.1Ghz processor:
pmcstat -n 1050000 -S UNHALTED_CORE_CYCLES -O /dev/null sleep 6000 &

Collecting 10 samples of `make -j96 buildkernel` from each:

x before
+ after

real time:
    N           Min           Max        Median           Avg        Stddev
x  10          76.4        127.62        84.845        88.577     15.100031
+  10         59.71         60.79        60.135        60.179    0.29957192
Difference at 95.0% confidence
        -28.398 +/- 10.0344
        -32.0602% +/- 7.69825%
        (Student's t, pooled s = 10.6794)

system time:
    N           Min           Max        Median           Avg        Stddev
x  10       2277.96       6948.53       2949.47      3341.492     1385.2677
+  10        1038.7       1081.06      1070.555      1064.017      15.85404
Difference at 95.0% confidence
        -2277.47 +/- 920.425
        -68.1574% +/- 8.77623%
        (Student's t, pooled s = 979.596)

x no pmc
+ pmc running
real time:

HEAD:
    N           Min           Max        Median           Avg        Stddev
x  10         58.38         59.15         58.86        58.847    0.22504567
+  10          76.4        127.62        84.845        88.577     15.100031
Difference at 95.0% confidence
        29.73 +/- 10.0335
        50.5208% +/- 17.0525%
        (Student's t, pooled s = 10.6785)

patched:
    N           Min           Max        Median           Avg        Stddev
x  10         58.38         59.15         58.86        58.847    0.22504567
+  10         59.71         60.79        60.135        60.179    0.29957192
Difference at 95.0% confidence
        1.332 +/- 0.248939
        2.2635% +/- 0.426506%
        (Student's t, pooled s = 0.264942)

system time:

HEAD:
    N           Min           Max        Median           Avg        Stddev
x  10       1010.15       1073.31      1025.465      1031.524     18.135705
+  10       2277.96       6948.53       2949.47      3341.492     1385.2677
Difference at 95.0% confidence
        2309.97 +/- 920.443
        223.937% +/- 89.3039%
        (Student's t, pooled s = 979.616)

patched:
    N           Min           Max        Median           Avg        Stddev
x  10       1010.15       1073.31      1025.465      1031.524     18.135705
+  10        1038.7       1081.06      1070.555      1064.017      15.85404
Difference at 95.0% confidence
        32.493 +/- 16.0042
        3.15% +/- 1.5794%
        (Student's t, pooled s = 17.0331)

Reviewed by:	jeff@
Approved by:	sbruno@
Differential Revision:	https://reviews.freebsd.org/D15155
2018-05-12 01:26:34 +00:00
rmacklem
962b30b569 Add support for the TestStateID operation to the NFSv4.1 server.
The Linux client now uses the TestStateID operation, so this patch adds
support for it to the NFSv4.1 server. The FreeBSD client never uses this
operation, so it should not be affected.

MFC after:	2 months
2018-05-11 22:16:23 +00:00
jilles
9a92c3b5eb sh: Test that backslash-newline within single-quotes is not special
This works correctly, but the test may be helpful when modifying the parser.
2018-05-11 21:56:01 +00:00
shurd
210352aeb1 Fix LORs in in6?_leave_group()
r333175 updated the join_group functions, but not the leave_group ones.

Reviewed by:	sbruno
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15393
2018-05-11 21:42:27 +00:00
kib
009a13f92b Remove dead declaration.
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
2018-05-11 20:47:45 +00:00
mmacy
0a7aab5128 iflib(9): Add support for cloning pseudo interfaces
Part 3 of many ...
The VPC framework relies heavily on cloning pseudo interfaces
(vmnics, vpc switch, vcpswitch port, hostif, vxlan if, etc).

This pulls in that piece. Some ancillary changes get pulled
in as a side effect.

Reviewed by:	shurd@
Approved by:	sbruno@
Sponsored by:	Joyent, Inc.
Differential Revision:	https://reviews.freebsd.org/D15347
2018-05-11 20:08:28 +00:00
shurd
44343042d4 Fix mld6query(8) and add a new -g option
The mld6query command relies on KAME behaviour which allows the
ipv6mr_multiaddr member of the request object in a IPV6_JOIN_GROUP
setsockopt() call to be INADDR6_ANY. The FreeBSD stack doesn't allow
this, so mld6query has been non-functional.

Also, add a -g option which sends a General Query (query INADDR6_ANY)

Reviewed by:	sbruno, mmacy
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15384
2018-05-11 19:37:18 +00:00
mmacy
61486c0416 epoch(9): always set inited in epoch_init
- set inited in the !usedomains case

Reported by:	jhibbits
Approved by:	sbruno
2018-05-11 18:37:14 +00:00
sbruno
efee591c25 vxge(4): deprecation notice
This hardware isn't totally ancient, about equal to a mxge(4) or mlx4en(4),
but the company was sold to Exar which then promptly exited the Ethernet
business so the card was commercially available for under 2 years. On deep
search, the only usage of these cards I found was by the importing of the
driver. There are code quality issues identified by Brooks and Hiren and
no visible use nor maintainership that warrant removal from FreeBSD 12.0.

Submitted by:	kbowling
Reviewed by:	gnn brooks
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15363
2018-05-11 17:26:59 +00:00
bdrewery
16095883e9 Revert r333496.
I didn't put the correct paths, nor lib32, and I may have been comparing
against the wrong source tree.
2018-05-11 17:24:04 +00:00
ae
c53ab47acf Apply the change from r272770 to if_ipsec(4) interface.
It is guaranteed that if_ipsec(4) interface is used only for tunnel
mode IPsec, i.e. decrypted and decapsultaed packet has its own IP header.
Thus we can consider it as new packet and clear the protocols flags.
This allows ICMP/ICMPv6 properly handle errors that may cause this packet.

PR:		228108
MFC after:	1 week
2018-05-11 16:50:25 +00:00
bdrewery
1e55e4ac64 Add a bunch of orphaned libraries.
MFC after:	3 days
2018-05-11 16:46:52 +00:00
emaste
f3b0d7717c Strip trailing / from TESTSDIR
Otherwise makefs gets upset:
makefs: ./usr/tests/sys/cddl/zfs/tests/exec/: empty leaf element

Sponsored by:	The FreeBSD Foundation
2018-05-11 16:11:24 +00:00
trasz
4f5da10309 Improve development(7):
- Use Fx when referring to FreeBSD.
 - Use Ql instead of Cm for command invocations.
 - Remove some redundant Pp macros.
 - Use a literal indented Bd instead of a series of Dl macros.

Submitted by:	0mp@
Reviewed by:	eadler@
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D15126
2018-05-11 15:11:53 +00:00
trasz
6f31c71ba1 Set kldxref_enable="YES" for ARM images. Without it, the images are missing
the /boot/kernel/linker.hints file, which breaks loading some of the modules
with dependencies, eg cfiscsi.ko.

This is a minimal fix for ARM images, in order to safely MFC it before
11.2-RELEASE.  Afterwards, however, I believe we should actually just change
the default (as in, etc/defaults/rc.conf).  The reason is that it's required
for every image that's being cross-built, as kldxref(1) cannot handle files
for non-native architectures.  For the one that is not - amd64 - having it
on by default doesn't change anything - the script is noop if the linker.hints
already exists.

The long-term solution would be to rewrite kldxref(1) to handle other
architectures, and generate linker.hints at build time.

Reviewed by:	gjb@
MFC after:	3 days
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14534
2018-05-11 14:52:35 +00:00
ken
7820a12c47 Clear out the entire structure, not just the size of a pointer to it.
sys/dev/ocs/ocs_os.c:
	In ocs_thread_create(), use sizeof(*thread) (instead of
	sizeof(thread)) as the size argument to memset so that we clear
	out the entire thread structure instead of just a few bytes of it.

Submitted by:	jtl
MFC after:	3 days
2018-05-11 14:50:26 +00:00
trasz
006e6f83df Make /etc/rc.d/kldxref not print anything for directories that don't
contain any kernel modules.  This makes the common case completely silent,
as it should be.

Reviewed by:	imp@
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14694
2018-05-11 14:43:21 +00:00
des
13e42418d1 Upgrade to OpenSSH 7.7p1. 2018-05-11 13:22:43 +00:00
emaste
babef78f7a usbdevs: add new Microchip USB-Ethernet device IDs
LAN7800 USB 3.1 to 10/100/1000 Ethernet with PHY
LAN7801 USB 3.1 to 10/100/1000 Ethernet with RGMII interface

Also update manufacturer name for the Vendor ID.  Microchip acquired
SMSC in May 2012.

Sponsored by:	The FreeBSD Foundation
2018-05-11 13:09:21 +00:00
emaste
e3fceb2b36 gpart.8: list all options in table form for each command
Previously gpart's man page listed some command options in prose, and
some in table form, which made it more difficult to use as a reference.

Reviewed by:	bcr
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15135
2018-05-11 12:58:36 +00:00
emaste
d513e28273 gpart.8: sort suboptions per mdoc(7)
Alphabetical order, uppercase before lowercase for each letter and with
no regard to whether an option takes an argument.

Sponsored by:	The FreeBSD Foundation
2018-05-11 12:57:25 +00:00
mjg
7831fe46a1 amd64: align the .data.exclusive_cache_line section to 128
This aligns the section itself compared to other sections, does not change
internal alignment of fields stored inside. This may or may not come later.

The motivation is partially combating adverse effects of the adjacent cache
line prefetcher. Without the annotation part of read_mostly section was on
the line of fire.
2018-05-11 08:56:39 +00:00
mmacy
5f425dce2c epoch(9): callback task fixes
- initialize the pcpu STAILQ in the NUMA case
- don't enqueue the callback task if there isn't sufficient work to be done

Reported by:	pho@
Approved by:	sbruno@
2018-05-11 08:16:56 +00:00
mjg
02e19399b7 uma: increase alignment to 128 bytes on amd64
Current UMA internals are not suited for efficient operation in
multi-socket environments. In particular there is very common use of
MAXCPU arrays and other fields which are not always properly aligned and
are not local for target threads (apart from the first node of course).
Turns out the existing UMA_ALIGN macro can be used to mostly work around
the problem until the code get fixed. The current setting of 64 bytes
runs into trouble when adjacent cache line prefetcher gets to work.

An example 128-way benchmark doing a lot of malloc/frees has the following
instruction samples:

before:
kernel`lf_advlockasync+0x43b            32940
          kernel`malloc+0xe5            42380
           kernel`bzero+0x19            47798
   kernel`spinlock_exit+0x26            60423
         kernel`0xffffffff80            78238
                         0x0           136947
   kernel`uma_zfree_arg+0x46           159594
 kernel`uma_zalloc_arg+0x672           180556
   kernel`uma_zfree_arg+0x2a           459923
 kernel`uma_zalloc_arg+0x5ec           489910

after:
            kernel`bzero+0xd            46115
kernel`lf_advlockasync+0x25f            46134
kernel`lf_advlockasync+0x38a            49078
   kernel`fget_unlocked+0xd1            49942
kernel`lf_advlockasync+0x43b            55392
          kernel`copyin+0x4a            56963
           kernel`bzero+0x19            81983
   kernel`spinlock_exit+0x26            91889
         kernel`0xffffffff80           136357
                         0x0           239424

See the review for more details.

Reviewed by:	kib
Differential Revision:	https://reviews.freebsd.org/D15346
2018-05-11 07:04:57 +00:00
mjg
e6cfcf1248 rmlock: partially depessimize lock/unlock fastpath
Previusly the slow path was folded in and partially jumped over in the
common case.
2018-05-11 06:59:54 +00:00
eadler
fa401e141a [expand] add __dead2 annotation to usage 2018-05-11 06:55:02 +00:00
mmacy
361b54f07a Allow different bridge types to coexist
if_bridge has a lot of limitations that make it scale poorly to higher data
rates. In my projects/VPC branch I leverage the bridge interface between
layers for my high speed soft switch as well as for purposes of stacking
in general.

Reviewed by:	sbruno@
Approved by:	sbruno@
Differential Revision:	https://reviews.freebsd.org/D15344
2018-05-11 05:00:40 +00:00
mmacy
73d042eb34 epoch(9): fix priority handling, make callback lists pcpu, and other fixes
- Lend priority to preempted threads in epoch_wait to handle the case
  in which we've had priority lent to us. Previously we borrowed the
  priority of the lowest priority preempted thread. (pointed out by mjg@)

- Don't attempt allocate memory per-domain on powerpc, we don't currently
  handle empty sockets (as is the case on jhibbits Talos' board).

- Handle deferred callbacks as pcpu lists and poll the lists periodically.
  Currently the interval is 1/hz.

- Drop the thread lock when adaptive spinning. Holding the lock starves
  other threads and can even lead to lockups.

- Keep a generation count pcpu so that we don't keep spining if a thread
  has left and re-entered an epoch section.

- Actually removed the callback from the callback list so that we don't
  double free. Sigh ...

Approved by:	sbruno@
2018-05-11 04:54:12 +00:00
mmacy
2076e8aba5 Test priority handling in epoch test.
- Double the number of test threads to mp_ncpu*2
- Give each thread a different scheduling priority
2018-05-11 04:47:05 +00:00
jhibbits
095a58f17a No need to bzero splpar_vpa entries
splpar_vpa is in the BSS, so is already zeroed when the kernel starts up.

Tested by:	Leandro Lupori
2018-05-11 02:04:01 +00:00
jasone
c9624aad5c Update jemalloc to version 5.1.0. 2018-05-11 00:32:31 +00:00
des
42f7c6ed63 Slight cleanup of interface event logging.
Make if_printf() use vlog() instead of vprintf().  This means it can no
longer return the number of characters printed, as it used to, but every
single call to if_printf() in the entire kernel ignores the return value
anyway; just return 0 so we don't have to change the prototype.

Consistently use if_printf() throughout sys/net/if.c, instead of a
mixture of if_printf() and log().

In ifa_maintain_loopback_route(), don't needlessly log an error if we
either failed to add a route because it already existed or failed to
remove one because it did not.  We still return an error code, though.

MFC after:	1 week
2018-05-11 00:19:49 +00:00
des
58d2db41a5 Reduce <sys/queue.h> pollution.
While <sys/sysctl.h> includes <sys/queue.h> unconditionally, it is only
actually used in code which is conditional on _KERNEL.  Make the #include
itself conditional as well, and fix userland code that uses <sys/queue.h>
for other purposes but relied on <sys/sysctl.h> to bring it in.

MFC after:	1 week
2018-05-11 00:01:43 +00:00
gjb
06e9cdaa58 Add a special GCE_LICENSE variable to Makefile.gce, which when set,
will include license metadata in the resultant GCE image.

GCE_LICENSE is unset by default, as it primarily pertains to images
produced by the FreeBSD Project, but for downstream FreeBSD consumers,
it can be set in the make(1) environment in the format of:

 --licenses="projects/PROJECT_ID/global/licenses/LICENSE_NAME"

The "license" is not a license, per se, but required metadata that
is required by the GCE marketplace.  For the FreeBSD Project, the
license name is simply 'freebsd', with the description of 'FreeBSD'.

MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
2018-05-10 21:46:58 +00:00
np
f702263d0a cxgbe(4): Add fields to support configuration of hardware NAT and
swapmac (SMAC/DMAC switcheroo) from userspace.

Sponsored by:	Chelsio Communications
2018-05-10 20:39:04 +00:00
imp
2da7562b43 Revert r333365
Even though we don't use it, it appears something else requires it to
be != 0 to work. This breaks tftp boot in loader.efi, so revert until
that can be sorted out.
2018-05-10 20:27:12 +00:00
emaste
52fe12515b Error out on attempt to link amd64 kernel with old binutils linker
As of r333461 we require ifunc support to link a working amd64 kernel.
The default in-tree bootstrap linker is lld and it has the required
support, as does any modern out-of-tree binutils linker.  The in-tree
GNU ld is from binutils 2.17.50 and it does not have ifunc support,
so produce an error rather than a broken kernel.

Reviewed by:	kib
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15378
2018-05-10 20:10:02 +00:00
mmacy
0f77b86d64 Allocate epoch for networking at startup
Additionally add CK to include paths for modules

Approved by:	sbruno@
2018-05-10 19:13:00 +00:00
mmacy
b175926a96 Revert accidentally commited local change to bmake to prevent debilitating
excess system time from poor API usage.

Approved by:	sbruno@
2018-05-10 17:57:46 +00:00
mmacy
68b801ac97 Add simple preempt safe epoch API
Read locking is over used in the kernel to guarantee liveness. This API makes
it easy to provide livenes guarantees without atomics.

Includes epoch_test kernel module to stress test the API.

Documentation will follow initial use case.

Test case and improvements to preemption handling in response to discussion
with mjg@

Reviewed by:	imp@, shurd@
Approved by:	sbruno@
2018-05-10 17:55:24 +00:00
lwhsu
a2b0dc578d Fix build for platforms using GCC:
- Remove unused or dead store variable
- Remove unused function ctl_copyin_alloc
- Add missing curly brackets, this seems a regression in r287720

Reviewed by:	jhibbits
Differential Revision:	https://reviews.freebsd.org/D15383
2018-05-10 17:22:04 +00:00
dumbbell
242ffb3648 vt(4): Use default VGA palette
Before this change, the VGA palette was configured to match the shell
palette (e.g. color #1 was red). There was one glitch early in boot when
the vt(4)'s VGA palette was loaded: the loader's logo would switch from
red to blue. Likewise for the "Booting..." message switching from blue
to red. That's because the loader's logo was drawed with the default VGA
palette where a few colors are swapped compared to the shell palette
(e.g. blue <-> red).

This change configures the default VGA palette during initialization and
converts input's colors from shell to VGA palette index.

There should be no visible changes, except the loader's logo which will
keep its original color.

Reviewed by:	eadler
2018-05-10 17:00:33 +00:00
dumbbell
c7886ada60 vt(4): Put for() loop outside switch() in vt_generate_cons_palette()
This makes it more logical:
 1. It checks the requested color format
 2. It fills the palette accordingly

Also vt_palette_init() is only called when needed (i.e. when the format
is `COLOR_FORMAT_RGB`).
2018-05-10 16:41:47 +00:00