Commit Graph

145 Commits

Author SHA1 Message Date
Matt Macy
f992dd4b5c pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
Matt Macy
b2ca2e50b9 hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
Matt Macy
5de96e33d6 hwpmc: support sampling both kernel and user stacks when interrupted in kernel
This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.

Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.

Obtained from: jtl
Sponsored by: Juniper Networks, Limelight Networks
2018-06-04 01:10:23 +00:00
Matt Macy
fbf962e6bb libpmc: allow substring for list and add function for printing event details 2018-06-01 00:45:53 +00:00
Matt Macy
ae266f9f94 pmcstat: suppress uninitialized warning of event 2018-05-27 17:49:27 +00:00
Matt Macy
959826ca1b pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
Matt Macy
5506ceb87f Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
Matt Macy
4928135658 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00
Sean Bruno
e22ee9b247 pmcstat(8)
- Document per thread filtering.

Submitted by:	kbowling
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15549
2018-05-25 18:11:13 +00:00
Matt Macy
0a87e3ff64 pmcstat: add usage strings for -L and -I 2018-05-24 07:18:49 +00:00
Matt Macy
4bd2a4b58e pmcstat top mode + -I - use fully qualified address 2018-05-24 06:47:04 +00:00
Matt Macy
357405d985 pmcstat: don't compare signed and unsigned 2018-05-24 04:43:40 +00:00
Matt Macy
e98bbcf9ca libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00
Matt Macy
95104040a3 pmc: bump PMC major version to reflect ABI breakage and make warning not require verbose 2018-05-23 19:55:47 +00:00
Matt Macy
0b5dc7f64f hwpmc: add thread id field to callchain and context switch records to allow filtering
on thread in post-processing.

To generate stacks for just ${THREADID}:

pmcstat -R ${PREFIX}.pmcstat -L ${THREADID} -z100 -G ${PREFIX}.stacks

Sponsored by:	Limelight Networks
2018-05-23 17:25:00 +00:00
Matt Macy
821a352a77 pmcstat: add option to not decode the leaf function in top mode
-I will allow the user to see the hot instruction in question
as opposed getting the name of the function
2018-05-22 04:45:46 +00:00
Fabien Thomas
cb43ed2815 Only call flush in pipe mode.
It fixes a crash with a socket in top mode.

Ex:
# pmcstat -R 127.0.0.1:8080 -T -w1
then
# pmcstat -n1 -Sclock.prof -Slock.failed -O 127.0.0.1:8080

MFC after:	1 week
Sponsored by:	Stormshield
2018-01-17 16:55:35 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Andriy Voskoboinyk
7d41856e52 pmcstat(8): unbreak build with LDFLAGS+=-Wl,--as-needed
libpmcstat.a requires libelf.so; reorder them accordingly.
2017-11-03 00:12:32 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Ruslan Bukin
d27927f731 Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D12718
2017-10-24 16:28:00 +00:00
Konstantin Belousov
e28f262a86 Fix off by one in calculation of the number of buckets for the pc
addresses.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D11864
2017-08-04 18:02:54 +00:00
Pedro F. Giffuni
780154cbaf pmcstat(8); unsign some allocation variables and use reallocarray(3).
Use unsigned values in some internal variables that will be used during
allocation. The variables are used in reduced scope and have no chance of
becoming negative.

Provide bounds checking through reallocarray(3).

MFC after:	2 weeks
2017-04-22 16:02:40 +00:00
Mark Johnston
3176d87c9d Remove an unused field from struct pmcstat_image.
MFC after:	3 days
2016-10-15 23:46:55 +00:00
Stephen J. Kiernan
3ad36a25cf Just remove freeing memory in pmcstat_cleanup, since we only clean up
when we are exiting anyway.

Add NULL checks for all malloc and strdup returns.

Reviewed by:	gnn
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D8002
2016-10-07 21:03:12 +00:00
Ed Schouten
d93c912801 Make code compile when basename() is POSIX compliant.
The POSIX basename() function is allowed to modify its input buffer,
which means its argument is "char *". Pull a copy of the input string
before computing the base.

Reviewed by:	jtl
Differential Revision:	https://reviews.freebsd.org/D6465
2016-05-25 08:45:03 +00:00
Pedro F. Giffuni
d7b1cc0a3c pmcstat: minor spelling fixes.
Mostly comments bur also a user-visible string.

MFC after:	2 weeks
2016-05-01 16:40:56 +00:00
Jonathan T. Looney
cc2ccfbe20 Fix the date on the pmcstat(8) man page from r291016.
Approved by:	gnn (mentor)
MFC after:	1 month
X-MFC with:	r291016
Pointy hat to:	jtl
2015-11-18 15:12:24 +00:00
Jonathan T. Looney
465dadb59b Support a wider history counter in pmcstat(8) gmon output
pmcstat(8) contains an option to output sampling data in a gmon format
compatible with gprof(1). Currently, it uses the default histcounter,
which is an (unsigned short). With large sets of sampling data, it
is possible to overflow the maximum value provided by an (unsigned
short).

This change adds the -e argument to pmcstat. If -e and -g are both
specified, pmcstat will use a histcounter type of uint64_t.

Differential Revision:	https://reviews.freebsd.org/D4151
Reviewed by:	jhb, bjk
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-18 14:52:01 +00:00
Jonathan T. Looney
e64ba55dd2 Change the driver stats to what they really are: unsigned values.
When pmcstat exits after some samples were dropped, give the user an
idea of how many were lost. (Granted, these are global numbers, but
they may still help quantify the scope of the loss.)

Differential Revision:	https://reviews.freebsd.org/D4123
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-16 15:16:09 +00:00
Fabien Thomas
66ca84a7ba Fix for r288176 changes related to debug symbols move.
Reviewed by: emaste
Sponsored by:	Stormshield
2015-10-08 09:46:35 +00:00
John Baldwin
7816cb4a0b Fix an off by one error in r283613: Like regular ffs(), CPU_FFS() returns
1 for CPU 0, etc. so the return value must be decremented to obtain the
first valid CPU ID.

Submitted by:	fabient
MFC after:	1 week
2015-09-01 17:52:43 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
John Baldwin
ef16b9abd4 Use the cpuset API more consistently:
- Fetch the root set from cpuset_getaffinity() instead of assuming all CPUs
  from 0 to hw.ncpu are the root set.
- Use CPU_SETSIZE and CPU_FFS.
- The original notion of halted CPUs the manpage and code refers to is gone.
  Use the term "available" instead.

Differential Revision:	https://reviews.freebsd.org/D2491
Reviewed by:	emaste
MFC after:	1 week
2015-05-27 13:54:37 +00:00
Simon J. Gerraty
98e0ffaefb Merge sync of head 2015-05-27 01:19:58 +00:00
Stanislav Sedov
c21fd5b663 Fix pmcstat symbol resolution for userland processes.
When examining existing processes pmcstat fails to
correctly determine the locations of executable sections
of the process due to a miscalculated virtual load address.
This does not affect the newly launched processes as the
same value passed as a "start address" to the pmcstat_image_link()
thus nullifying the effect of it.  The issue manifests itself
in processes not being reported in the pmcstat(8) output and
"dubious frames" being reported.

Fix it for now by ignoring all the sections except the executable
one.  This won't fix the issue for objects with multiple
executable sections but helps in majority of real world usecases.
The real solution would be to modify the MAP-IN event to include
the appropriate load address so pmcstat(8) won't have to manually
parse object files to try to determine it.

PR:		198147, 198148
Reviewed by:	jhb, rpaulo
MFC after:	2 weeks
2015-05-13 18:52:18 +00:00
Stanislav Sedov
dc599a6bd9 Fix the double space in comment.
While here, change the comment style to match the rest of the file.
2015-05-09 00:48:44 +00:00
John Baldwin
38fd38349a Use the kern.bootfile sysctl to set the default kernel path rather than
hardcoding /boot/kernel. This allows pmcstat(8) to work without -k when
using nextboot -k or 'boot foo' at the loader to boot alternate kernels.

Differential Revision:	https://reviews.freebsd.org/D2425
Reviewed by:	adrian, emaste, gnn
MFC after:	2 weeks
Sponsored by:	Norse Corp, Inc.
2015-05-08 16:02:31 +00:00
Eric van Gyzen
10813ab380 pmcstat.8: fix -a flag description; improve -m flag to match
The -a flag reads a file saved by -O, not -o.

The -m flag requires the -R flag.  Copy that paragraph from -a.

Reviewed by:	adrian
Approved by:	kib (mentor)
MFC after:	1 week
Sponsored by:	Dell Inc
2015-03-28 21:28:22 +00:00
Baptiste Daroussin
c6db8143ed Convert usr.sbin to LIBADD
Reduce overlinking
2014-11-25 16:57:27 +00:00
Simon J. Gerraty
9268022b74 Merge from head@274682 2014-11-19 01:07:58 +00:00
Baptiste Daroussin
f42d8dec6c Fix rendering
Submitted by:	brueffer
2014-10-27 15:50:47 +00:00
Baptiste Daroussin
99e33eab01 Clarify the documentation of pmcstat:
the -d argument should be passed before -p, -s, -P or -S to be taken in account

Differential Revision:	https://reviews.freebsd.org/D1011
Reviewed by:	adrian, gnn
MFC after:	1 week
2014-10-27 15:33:44 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Joel Dahl
f8d7aff9dd mdoc: kill EOL whitespace. 2014-07-29 19:49:27 +00:00
Brooks Davis
80189b3b09 Replace all uses of libncurses and libtermcap with their wide character
variants.  This allows usable file system images (i.e. those with both a
shell and an editor) to be created with only one copy of the curses library.

Exp-run:	antoine
PR:		189842
Discussed with:	bapt
Sponsored by:	DARPA, AFRL
2014-07-17 18:24:34 +00:00
Baptiste Daroussin
01c2b8ac0d use .Mt to mark up email addresses consistently (part2)
PR:		191174
Submitted by:	Franco Fichtner  <franco@lastsummer.de>
2014-06-20 09:57:27 +00:00
Ed Maste
06cc6df79c Update default callchain depth to 16 to match kernel
MFC after:	1 week
2014-05-30 20:12:47 +00:00
George V. Neville-Neil
1cb3ec2eb7 Update the date on the manual page.
Pointed out by: bz
2014-05-18 15:31:53 +00:00