Commit Graph

19468 Commits

Author SHA1 Message Date
Sean Bruno
1a43cff92a Load balance sockets with new SO_REUSEPORT_LB option.
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).

This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967.  Thanks to rwatson@
for the substantive feedback that is included in this commit.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Obtained from:	DragonflyBSD
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-06-06 15:45:57 +00:00
Matt Macy
b2ca2e50b9 hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
Mark Johnston
ea16e3e1e7 Don't build brk_test on platforms that don't support brk().
X-MFC with:	r334626
2018-06-05 13:06:06 +00:00
Matt Macy
ebfaf69cc0 hwpmc: log name->pid, name->tid mappings
By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log
2018-06-05 04:26:40 +00:00
Matt Macy
12193215c2 libpmc: improve arg checking 2018-06-04 21:17:46 +00:00
Matt Macy
ee62e0b419 libpmc / stat: use westmere supported counter 2018-06-04 21:17:42 +00:00
Mark Johnston
9f9c9b22ec Reimplement brk() and sbrk() to avoid the use of _end.
Previously, libc.so would initialize its notion of the break address
using _end, a special symbol emitted by the static linker following
the bss section.  Compatibility issues between lld and ld.bfd could
cause the wrong definition of _end (libc.so's definition rather than
that of the executable) to be used, breaking the brk()/sbrk()
interface.

Avoid this problem and future interoperability issues by simply not
relying on _end.  Instead, modify the break() system call to return
the kernel's view of the current break address, and have libc
initialize its state using an extra syscall upon the first use of the
interface.  As a side effect, this appears to fix brk()/sbrk() usage
in executables run with rtld direct exec, since the kernel and libc.so
no longer maintain separate views of the process' break address.

PR:		228574
Reviewed by:	kib (previous version)
MFC after:	2 months
Differential Revision:	https://reviews.freebsd.org/D15663
2018-06-04 19:35:15 +00:00
Matt Macy
ce6fc9d779 update pmc_pmu_idx_get_by_event signature for !x86 2018-06-04 05:04:40 +00:00
Matt Macy
bfb46e2ba8 pmc: add filter command
pmc filter allows the user to select event types, threads, and processes from
a sample.

% pmcstat -S unhalted_core_cycles -S llc-misses -S -S  resource_stalls.any -O pmc.log
% pmc filter -e llc-misses pmc.log pmc-llc-misses.log
% pmc filter -e unhalted_core_cycles -t 100339  pmc.log pmc-core-cycles.log
etc...
% pmcstat -R pmc-core-cycles.log -G pmc-core-cycles.stacks
2018-06-04 04:59:48 +00:00
Matt Macy
07d80fd8dc hwpmc: ABI fixes
- increase pmc cpuid field from 8 to 12 bits
- add cpuid version string to initialize entry in the log
  so that filter can identify which counter index an
  event name maps to
- GC unused config flags
- make fixed counter assignment more robust as well as the
  changes needed to be properly identified for filter
2018-06-04 02:05:48 +00:00
Alan Somers
8a46c08a2a pty.3: Add a HISTORY section
These functions were first added in 4.3 BSD-Reno, according to
http://unix.superglobalmegacorp.com/ and the CSRG svn repository.

Reviewed by:	bcr, bjk
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D15652
2018-06-02 22:40:16 +00:00
Justin Hibbits
2e65567500 Added ptrace support for reading/writing powerpc VSX registers
Summary:
Added ptrace support for getting/setting the remaining part of the VSX registers
(the part that's not already covered by FPR or VR registers).

This is necessary to add support for VSX registers in debuggers.

Submitted by:	Luis Pires
Differential Revision: https://reviews.freebsd.org/D15458
2018-06-02 19:17:11 +00:00
Mark Johnston
2e7680c6bf Don't export _end on arm64 and riscv.
These platforms don't support brk() and sbrk(), which are the reason
for exporting _end in the first place.

MFC after:	1 week
2018-06-01 23:42:10 +00:00
Mark Johnston
e2c1730299 Remove an inaccuracy from mincore.2.
Super pages are supported on non-x86 architectures, so just remove the
incorrect note.  While here, change terminology to be consistent with
mmap.2.

MFC after:	1 week
2018-06-01 23:40:43 +00:00
Brooks Davis
0141ef6c07 Remove support for SYS_sys_exit in favor of SYS_exit.
SYS_exit has been defined in the repo since 1994 except for a brief
window when SYS_sys_exit was defined in 2000.
2018-06-01 22:09:27 +00:00
Matt Macy
fbf962e6bb libpmc: allow substring for list and add function for printing event details 2018-06-01 00:45:53 +00:00
Matt Macy
cfb0dfa3c2 libpmc: Intel doesn't require runtime counter table init 2018-06-01 00:45:48 +00:00
Matt Macy
96cbd26a24 libpmc/pmu: update aliases table 2018-06-01 00:45:43 +00:00
Matt Macy
e92a1350b5 hwpmc: remove unused pre-table driven bits for intel
Intel now provides comprehensive tables for all performance counters
and the various valid configuration permutations as text .json files.
Libpmc has been converted to use these and hwpmc_core has been greatly
simplified by moving to passthrough of the table values.

The one gotcha is that said tables don't support pentium pro and and pentium
IV. There's very few users of hwpmc on _amd64_ kernels on new hardware. It is
unlikely that anyone is doing low level optimization on 15 year old Intel
hardware. Nonetheless, if someone feels strongly enough to populate the
corresponding tables for p4 and ppro I will reinstate the files in to the
build.

Code for the K8 counters and !x86 architectures remains unchanged.
2018-05-31 22:41:07 +00:00
Matt Macy
7d1c2b74a0 libpmc/pmu: enable for i386 as well 2018-05-31 22:26:55 +00:00
Matt Macy
785dd70da8 libpmc: add pmu support for uncore events 2018-05-31 22:26:50 +00:00
Dimitry Andric
b451efbedc Resolve conflicts between macros in fenv.h and ieeefp.h
This is a follow-up to r321483, which disabled -Wmacro-redefined for
some lib/msun tests.

If an application included both fenv.h and ieeefp.h, several macros such
as __fldcw(), __fldenv() were defined in both headers, with slightly
different arguments, leading to conflicts.

Fix this by putting all the common macros in the machine-specific
versions of ieeefp.h.  Where needed, update the arguments in places
where the macros are invoked.

This also slightly reduces the differences between the amd64 and i386
versions of ieeefp.h.

Reviewed by:	kib
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D15633
2018-05-31 20:22:47 +00:00
Warner Losh
cf72c10fd2 Update to device enumeration protocol 2
The new protocol from the kernel encodes things as a string table,
extract it into fields. strdup the strings, and free them when we're
done.

Differential Revision: https://reviews.freebsd.org/D15629
2018-05-31 02:58:03 +00:00
Warner Losh
92376fa76c Sanity check the return from the kernel.
We should be getting back as many bytes as we asked for, and we
don't handle shortages at all, so just reject anything that's
not right.

Differential Revision: https://reviews.freebsd.org/D15629
2018-05-31 02:57:51 +00:00
Warner Losh
51a9cfbf60 There's no meaningful errno when there's a version mismatch, so use
warnx. Also, report the mis-matched versions.
2018-05-30 15:08:46 +00:00
Hans Petter Selasky
7cf2653c60 Remove redundant header file from source list in libpcap.
MFC after:	3 weeks
Sponsored by:	Mellanox Technologies
2018-05-30 08:24:57 +00:00
Matt Macy
8ffdc5df52 libpmc: squelch valgrind warnings 2018-05-30 04:12:51 +00:00
Matt Macy
8b20f97570 libpmc: bring pmu_util closer in line with style(9) 2018-05-30 03:40:02 +00:00
Matt Macy
8bed125da7 libpmc: silence scan-build warnings 2018-05-29 21:02:13 +00:00
Eric van Gyzen
4f9b4ac4c2 pmc_annotate: adhere to the API
If the 'mode' parameter was invalid, pmc_annotate() would
return EINVAL instead of setting errno and returning -1.

Sponsored by:	Dell EMC
2018-05-29 20:30:46 +00:00
Matt Macy
1a32aa0ff7 libpmc: don't return -1 on success in pmc_allocate 2018-05-29 20:09:35 +00:00
Matt Macy
ed50b91880 libpmc: don't leak string in error case either
Reported by:	vangyzen@
2018-05-29 19:07:00 +00:00
Matt Macy
c53c63f67e libpmc: remove fixed counter diagnostic 2018-05-29 18:30:37 +00:00
Matt Macy
731ca83529 libpmc: free allocated string on return from pmc_allocate
Reported by:	Coverity
CID: 1391359
2018-05-29 18:26:12 +00:00
Dag-Erling Smørgrav
5f04ebd4d3 Fix an inverted conditional in the netrc code, which would ignore the
value of $HOME and always use the home directory from the passwd
database, unless $HOME was unset, in which case it would use (null).

While there, clean up handling of netrcfd and add debugging aids.

MFC after:	3 weeks
2018-05-29 13:07:36 +00:00
Dag-Erling Smørgrav
f2eac20246 Fix a few (but far from all) style issues.
MFC after:	3 weeks
2018-05-29 10:29:43 +00:00
Dag-Erling Smørgrav
c5712d6da1 Use __VA_ARGS__ to simplify the DEBUG macro.
MFC after:	3 weeks
2018-05-29 10:28:20 +00:00
Matt Macy
a191ed2d9f libpmc: add support for using fixed function counters 2018-05-29 04:23:21 +00:00
Matt Macy
78beed2f81 libpmc: export names of counters for stat mode, make get_by_idx name consistent with others 2018-05-29 04:23:16 +00:00
Eric van Gyzen
23d2383783 Bump the date on man pages in r334306
It seems a shame to ruin the patina of the June 4, 1993 date
on abort.3, especially since it still matched the date of
the SCCS ID, but those are the rules.

Reported by:	araujo
MFC after:	3 days
Sponsored by:	Dell EMC
2018-05-29 02:41:32 +00:00
Eric van Gyzen
4c72d794f0 Cross-reference abort2(2) from a few man pages
I didn't know abort2 existed until it was mentioned on a mailing list.
Mention it in related pages so others can find it easily.

MFC after:	3 days
Sponsored by:	Dell EMC
2018-05-29 01:16:00 +00:00
Ed Maste
0eff530775 strsep.3: don't silently ignore errors
Reported by:	bde
MFC with:	r334275
2018-05-28 18:29:15 +00:00
Hans Petter Selasky
b00ab7548b MFV r333789: libpcap 1.9.0 (pre-release)
MFC after:	1 month
Sponsored by:	Mellanox Technologies
2018-05-28 08:12:18 +00:00
Marcelo Araujo
7e3d9013f2 Remove unnecessary free(string) from EXAMPLES.
Reported by:	se
2018-05-28 06:01:02 +00:00
Marcelo Araujo
4bee02ad1b Update strsep(3) EXAMPLE section regards the usage of assert(3).
As many people has pointed out, using assert(3) shall be not the best approach
to verify if strdup(3) has allocated memory to string.

Reviewed by:	imp
MFC after:	4 weeks.
Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D15594
2018-05-28 05:01:42 +00:00
Eric van Gyzen
0155aa938e fts_stat: fix buffer overrun on error path
Reported by:	Coverity
CID:		1375582
MFC after:	1 week
Sponsored by:	Dell EMC
2018-05-28 02:31:49 +00:00
Eric van Gyzen
9b20744177 libprocstat: fix memory leak
Free the rlimits array on the happy path in procstat_getrlimit_core().

Reported by:	Coverity
CID:		1373328
Sponsored by:	Dell EMC
2018-05-28 02:10:35 +00:00
Matt Macy
959826ca1b pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
Matt Macy
5506ceb87f Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
Matt Macy
4928135658 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00