Commit Graph

233547 Commits

Author SHA1 Message Date
Matt Macy
9616acde97 hwpmc: don't do EMIT64 on constant 2018-06-07 02:20:27 +00:00
Matt Macy
f992dd4b5c pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
Matt Macy
1b000b505b libpmc: simplify lookup by idx 2018-06-07 00:55:17 +00:00
Matt Macy
41abd7afa3 hwpmc: don't log pid->name more than once 2018-06-07 00:54:43 +00:00
Matt Macy
155046394a cpufunc: add rdtscp for x86 2018-06-07 00:54:11 +00:00
Justin Hibbits
60f18ad0b4 Add partition scheme for mpc85xx devices
Some mpc85xx devices with u-boot need MBR partitioning with a FAT boot
partition.  Since the infrastructure is already in place to have a dedicated
boot partition, this adds the necessary bits to use that infrastructure with
mpc85xx boards.

Reviewed By: nwhitehorn
Differential Revision: https://reviews.freebsd.org/D15664
2018-06-07 00:24:10 +00:00
Baptiste Daroussin
30dc95029e Remove NLS support from BSD grep
GNU grep as in actually in base does not have any translations support
compiled in, so no functionnality loss.

We do support 193 locales in base, we will never catch up on that number of
translation with bsd grep.

Removing NLS support make bsd grep consistent with the other binaries in base
which are not translated, and also reduce a little bit the code.

Reviewed by:	kevans
Approved by:	kevans
Discussed with:	kevans @BSDCan
Differential Revision:	https://reviews.freebsd.org/D15682
2018-06-06 23:12:35 +00:00
Conrad Meyer
9316908f48 strdup.3: Document standardization in POSIX
Sponsored by:	Dell EMC Isilon
2018-06-06 21:32:49 +00:00
Conrad Meyer
b7e471c2b2 strcpy.3: Improve legibility and clarity
In the DESCRIPTION, put the more commonly used functions first in the
corresponding sentence, to help catch the eye.

Pull out the note about overlapping buffers to its own paragraph, as it
applies to all routines documented by this page.

Emphasize the potentially surprising strncpy(3) behavior of zero-filling the
remainder of a buffer larger than the source string.

Encourage strlcpy use; remove portability note about strlcpy(3).  Adapting a
strlcpy-using code base to a platform that does not provide strlcpy in libc
is so trivial as to not be worth mentioning.  (Just copy strlcpy.c out of
any BSD libc, or include and link the pre-packaged libbsd library on non-BSD
platforms.)

Likewise, expand the page's warning about ease of potential misuse to cover
all functions documented herein, and explicitly suggest using strlcpy most
of the time.  The text was mostly cribbed from a similar suggestion in
gets(3).

Finally, document the remaining valid use of strncpy -- the rare
fixed-length record with no expectation of nul-termination.

Sponsored by:	Dell EMC Isilon
2018-06-06 20:09:21 +00:00
Kristof Provost
fa1d4439f9 pf: Return non-zero from 'status' if pf is not enabled
In the pf rc.d script the output of `/etc/rc.d/pf status` or `/etc/rc.d/pf
onestatus` always provided an exit status of zero. This made it fiddly to
programmatically determine if pf was running or not.

Return a non-zero status if the pf module is not loaded, extend pfctl to have
an option to return an error status if pf is not enabled.

PR:		228632
Submitted by:	James Park-Watt <jimmypw AT gmail.com>
MFC after:	1 week
2018-06-06 19:36:37 +00:00
Michael Tuexen
ff34bbe9c2 Improve compliance with RFC 4895 and RFC 6458.
Silently dicard SCTP chunks which have been requested to be
authenticated but are received unauthenticated no matter if support
for SCTP authentication has been negotiated. This improves compliance
with RFC 4895.

When the application uses the SCTP_AUTH_CHUNK socket option to
request a chunk to be received in an authenticated way, enable
the SCTP authentication extension for the end-point. This improves
compliance with RFC 6458.

Discussed with:	Peter Lei
MFC after:	3 days
2018-06-06 19:27:06 +00:00
Romain Tartière
69b96cd300 gpioctl.8: Fix man page section
Reviewed by:	manu
Approved by:	manu
2018-06-06 18:52:33 +00:00
Kyle Evans
7aba5b2f6a lualoader: Add a loaded hook for others to execute upon config load
This will not be executed on reload, though later work could allow for that.
It's intended/expected that later work won't generally need to happen on
every config load, just once (for, e.g., menu initialization) or just when
config is reloaded but not upon the initial load.
2018-06-06 18:28:17 +00:00
Conrad Meyer
cbb009b9fe puc(4): Add provisional support for Exar XR17V352
Reportedly, this is sufficient for 4800bps use, but maybe not any better.

PR:		228781
Submitted by:	peo AT nethead.se
2018-06-06 16:47:33 +00:00
Conrad Meyer
bc5356c5a8 clog.3, complex.3: Fix typos and igor style issues
PR:		228783
Reported by:	Karsten <freebsd-bugzilla AT kkoenig.net>
2018-06-06 16:25:00 +00:00
Hans Petter Selasky
40ddfc7604 Make some list functions RCU safe in the LinuxKPI.
While at it rename hlist_add_after() into hlist_add_behind().

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 15:49:01 +00:00
Sean Bruno
1a43cff92a Load balance sockets with new SO_REUSEPORT_LB option.
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).

This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967.  Thanks to rwatson@
for the substantive feedback that is included in this commit.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Obtained from:	DragonflyBSD
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-06-06 15:45:57 +00:00
Hans Petter Selasky
17e2a84e9a Rewrite code using atomic_fcmpset_int() in the LinuxKPI.
Suggested by:	mjg@
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 15:31:47 +00:00
Hans Petter Selasky
e6e028d01f Implement the __add_wait_queue_entry_tail() function in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 15:19:30 +00:00
Justin Hibbits
3f9e1fc8ee Revert r334708
This is the wrong place to put the barrier.
Requested by:	kib,mjg
2018-06-06 15:12:19 +00:00
Hans Petter Selasky
7e95e98db8 Implement the might_sleep_if() function macro in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 15:10:11 +00:00
Hans Petter Selasky
ab98f1e8d8 Rename two structure field members while keeping backwards compatibility in
the LinuxKPI. Add a comment saying in which Linux version this change was made.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 15:06:21 +00:00
Hans Petter Selasky
1b092623b2 Implement the init_wait_entry() function macro in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 14:59:23 +00:00
Hans Petter Selasky
23dcf4359e Implement the atomic_dec_if_positive() function in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 13:59:51 +00:00
Hans Petter Selasky
9e067b2256 Implement the ktime_compare() and ktime_after() functions in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 13:37:31 +00:00
Hans Petter Selasky
a16387c1d4 Implement the rdmsrl_safe() function macro in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-06 13:29:52 +00:00
Andrey V. Elsukov
590d0a43b6 Make in_delayed_cksum() be similar to IPv6 implementation.
Use m_copyback() function to write checksum when it isn't located
in the first mbuf of the chain. Handmade analog doesn't handle the
case when parts of checksum are located in different mbufs.
Also in case when mbuf is too short, m_copyback() will allocate new
mbuf in the chain instead of making out of bounds write.

Also wrap long line and remove now useless KASSERTs.

X-MFC after:	r334705
2018-06-06 13:01:53 +00:00
Justin Hibbits
32c369f40c Add a memory barrier after taking a reference on the vnode holdcnt in _vhold
This is needed to avoid a race between the VNASSERT() below, and another
thread updating the VI_FREE flag, on weakly-ordered architectures.

On a 72-thread POWER9, without this barrier a 'make -j72 buildworld' would
panic on the assert regularly.

It may be possible to use a weaker barrier, and I'll investigate that once
all stability issues are worked out on POWER9.
2018-06-06 12:57:11 +00:00
Andrey V. Elsukov
4a089e6bc5 Use m_copyback() function to write delayed checksum when it isn't located
in the first mbuf of the chain.

MFC after:	1 week
2018-06-06 10:46:24 +00:00
Eitan Adler
59c8d5dc27 top(1): fix err again
I am amazed at my ability to get a one-line error reporting line wrong
thrice.
2018-06-06 07:13:27 +00:00
Tom Jones
1fdbfb909f Use UDP len when calculating UDP checksums
The length of the IP payload is normally equal to the UDP length, UDP Options
(draft-ietf-tsvwg-udp-options-02) suggests using the difference between IP
length and UDP length to create space for trailing data.

Correct checksum length calculation to use the UDP length rather than the IP
length when not offloading UDP checksums.

Approved by: jtl (mentor)
Differential Revision:	https://reviews.freebsd.org/D15222
2018-06-06 07:04:40 +00:00
Eitan Adler
bca1b45bb1 top(1): make error message happy
We don't use a "size" variable, so just don't report it.

Reported by:	bapt
2018-06-06 06:42:12 +00:00
Andrey V. Elsukov
a41372abd4 Fix LINT-NOINET build.
Use known at build time size for min_length value. Also remove the
check from in6_gre_encapcheck(), now it is done in generic code.
2018-06-06 05:17:21 +00:00
Mateusz Guzik
ffc3ab5d39 malloc: elaborate on r334545 due to frequent questions
While here annotate the NULL check as probably true.
2018-06-06 05:08:05 +00:00
Matt Macy
b2ca2e50b9 hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
Brad Davis
64fe1b5e1d Only create /var/log/sendmail.st if start sendmail.
For those of us that never use or start sendmail, it is unneeded.

Approved by:	bapt (mentor)
2018-06-06 01:51:05 +00:00
Ian Lepore
9f6fd8397c Remove comments and assertions that are no longer valid after r330809.
r330809 replaced duplication of devdesc struct fields with an embedded copy
of the devdesc struct, to avoid fragility.  That means all the scattered
comments indicating that structs must match are no longer valid.  Likewise
asserts that attempted to mitigate some of the old fragility.

Reviewed by:	imp@
2018-06-05 22:13:45 +00:00
Andrey V. Elsukov
b941bc1d6e Rework if_gif(4) to use new encap_lookup_t method to speedup lookup
of needed interface when many gif interfaces are present.

Remove rmlock from gif_softc, use epoch(9) and CK_LIST instead.
Move more AF-related code into AF-related locations.
Use hash table to speedup lookup of needed softc. Interfaces
with GIF_IGNORE_SOURCE flag are stored in plain CK_LIST.
Sysctl net.link.gif.parallel_tunnels is removed. The removal was planed
16 years ago, and actually it could work only for outbound direction.
Each protocol, that can be handled by if_gif(4) interface is registered
by separate encap handler, this helps avoid invoking the handler
for unrelated protocols (GRE, PIM, etc.).

This change allows dramatically improve performance when many gif(4)
interfaces are used.

Sponsored by:	Yandex LLC
2018-06-05 21:24:59 +00:00
Andrey V. Elsukov
73ae9758a1 Constify argument of in6_getscope(). 2018-06-05 20:54:29 +00:00
Andrey V. Elsukov
6d8fdfa9d5 Rework IP encapsulation handling code.
Currently it has several disadvantages:
- it uses single mutex to protect internal structures. It is used by
  data- and control- path, thus there are no parallelism at all.
- it uses single list to keep encap handlers for both INET and INET6
  families.
- struct encaptab keeps unneeded information (src, dst, masks, protosw),
  that isn't used by code in the source tree.
- matches are prioritized and when many tunneling interfaces are
  registered, encapcheck handler of each interface is invoked for each
  packet. The search takes O(n) for n interfaces. All this work is done
  with exclusive lock held.

What this patch includes:
- the datapath is converted to be lockless using epoch(9) KPI.
- struct encaptab now linked using CK_LIST.
- all unused fields removed from struct encaptab. Several new fields
  addedr: min_length is the minimum packet length, that encapsulation
  handler expects to see; exact_match is maximum number of bits, that
  can return an encapsulation handler, when it wants to consume a packet.
- IPv6 and IPv4 handlers are stored in separate lists;
- added new "encap_lookup_t" method, that will be used later. It is
  targeted to speedup lookup of needed interface, when gif(4)/gre(4) have
  many interfaces.
- the need to use protosw structure is eliminated. The only pr_input
  method was used from this structure, so I don't see the need to keep
  using it.
- encap_input_t method changed to avoid using mbuf tags to store softc
  pointer. Now it is passed directly trough encap_input_t method.
  encap_getarg() funtions is removed.
- all sockaddr structures and code that uses them removed. We don't have
  any code in the tree that uses them. All consumers use encap_attach_func()
  method, that relies on invoking of encapcheck() to determine the needed
  handler.
- introduced struct encap_config, it contains parameters of encap handler
  that is going to be registered by encap_attach() function.
- encap handlers are stored in lists ordered by exact_match value, thus
  handlers that need more bits to match will be checked first, and if
  encapcheck method returns exact_match value, the search will be stopped.
- all current consumers changed to use new KPI.

Reviewed by:	mmacy
Sponsored by:	Yandex LLC
Differential Revision:	https://reviews.freebsd.org/D15617
2018-06-05 20:51:01 +00:00
Eric van Gyzen
56009ba0ed Make Coverity more happy with r334545
Coverity complains about:

	if (((flags) & M_WAITOK) || _malloc_item != NULL)

saying:

	The expression
		1 /* (2 | 0x100) & 2 */ || _malloc_item != NULL
	is suspicious because it performs a Boolean operation
	on a constant other than 0 or 1.

Although the code is correct, add "!= 0" to make it slightly
more legible and to silence hundreds(?) of Coverity warnings.

Reported by:	Coverity
Discussed with:	mjg
Sponsored by:	Dell EMC
2018-06-05 20:34:11 +00:00
Alan Somers
c41bbc0acb audit(4): add tests for open(2) and openat(2)
These syscalls are atypical, because each one corresponds to several
different audit events, and they each pass several different audit class
filters.

Submitted by:	aniketp
MFC after:	2 weeks
Sponsored by:	Google, Inc. (GSoC 2018)
Differential Revision:	https://reviews.freebsd.org/D15657
2018-06-05 20:13:24 +00:00
Andrey V. Elsukov
9f855fb282 tcp_lro.h requires <netinet/in.h>, include it directly instead of
indirect inclusion trough if_gif.h
2018-06-05 19:23:23 +00:00
Ed Maste
521ef556a0 Handle -DNO_CLEAN builds across brk/sbrk rewrite in r334626
Sponsored by:	The FreeBSD Foundation
2018-06-05 18:27:09 +00:00
Ian Lepore
3e9c7874ce Make the v*printf() functions in libsa return int instead of void.
This makes them compatible with the C standard signatures, avoiding
spurious mismatch errors in the places where the oddball requirements
of standalone code end up putting two declarations of the same function
in play.
2018-06-05 17:18:10 +00:00
Hans Petter Selasky
7a13eeba18 Declare and set the global "system_highpri_wq" workqueue structure pointer
in the LinuxKPI.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-05 15:49:35 +00:00
Hans Petter Selasky
c6d920309c Implement the INIT_DELAYED_WORK_ONSTACK() function macro in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-05 15:46:16 +00:00
Hans Petter Selasky
7f346854b8 Define the __kernel_size_t type in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-05 15:42:35 +00:00
Hans Petter Selasky
5a1d03bb7c Implement the task_pid_vnr() function macro in the LinuxKPI.
Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-05 15:40:09 +00:00
Hans Petter Selasky
747d9a8165 Add "access" function pointer to the "vm_operations_struct" structure
in the LinuxKPI. While at it document when to use the "virtual_address" or
the "address" field in the "vm_fault" structure.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
MFC after:	1 week
Sponsored by:	Mellanox Technologies
Sponsored by:	Limelight Networks
2018-06-05 15:37:28 +00:00