Commit Graph

160 Commits

Author SHA1 Message Date
Elyes Haouas
ca2edebb92 pmcstat: Remove useless return at the end of void function
Signed-off-by: Elyes Haouas <ehaouas@noos.fr>
Reviewed by: imp
Pull Request: https://github.com/freebsd/freebsd-src/pull/656
2023-03-02 08:01:39 -07:00
Andrew Gallatin
0aa1507751 pmcstat: fix log analysis
pmcstat has been broken for analyzing logs since D35342 / b6e28991bf.

This is because the pmc for the first CPU is not added when reading logs
because unlike its clones, its event id is not invalid. That causes us
to fail the assertion at lib/libpmcstat/libpmcstat_logging.c:293
when encountering samples from cpu0.

Fix this by removing the check that the PMC is invalid

Reviewed by: tsoome
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D35709
2022-07-04 12:42:39 -04:00
Aleksandr Rybalko
b6e28991bf System wide and NUMA domain wide counters support. PMC classes for ARM DMC-620 and CMN-600.
Add support for system wide and NUMA domain wide counters support.
Add 3 new PMC classes for ARM DMC-620 and CMN-600 controllers PMU.

Reviewed by:	mhorne
Sponsored By:	ARM
Sponsored By:	Ampere Computing
Differential Revision: https://reviews.freebsd.org/D35342
2022-06-26 08:31:03 +03:00
Baptiste Daroussin
cbc83e378a ncurses: chase dependency changes in the source tree
Differential Revision:	https://reviews.freebsd.org/D32098
2021-10-04 11:38:24 +02:00
Piotr Pawel Stefaniak
0939f965d8 Update a sysctl name to nbuffers_pcpu in hwpmc.4 and pmcstat.c
This change was missed in r333509 (e6b475e0af).

Differential Revision:	https://reviews.freebsd.org/D31704
Reviewed by:	mjg
2021-08-29 21:24:50 +02:00
Mitchell Horne
c4473add1d pmcstat: fix the usage message for -t
A slight mix-up of the flags means this case isn't triggered when it
should be. Now, `pmcstat -s event -t 1234` will print the correct
error message.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2021-04-21 17:44:52 -03:00
Mitchell Horne
edcf9e59c3 pmcstat: set initial counter value to zero
For an infrequent event, pmcstat may report (u_long)-1 for CPUs where
the counter was never incremented. Just set this to zero, instead.

ev->ev_count is passed as the 'count' argument to pmc_allocate(3), but
this wasn't always the case.

Reviewed by:	gnn
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D29887
2021-04-21 17:35:03 -03:00
Andrew Turner
9fd96b416c Add the missing pmcstat log entries
Sponsored by:	Innovate UK
2021-01-05 10:03:05 +00:00
Mateusz Piotrowski
d3231fbdd2 pmcstat: Fix a typo in the usage message
Reviewed by:	emaste
Approved by:	emaste
Differential Revision:	https://reviews.freebsd.org/D26082
2020-11-04 15:44:59 +00:00
Mateusz Piotrowski
51ae4071cb pmcstat.8: Improve description of -T 2020-08-17 10:51:03 +00:00
Mateusz Guzik
ec3b2a79b5 pmcstat: fix build on non-64 bit platforms 2020-08-05 20:42:08 +00:00
Mateusz Guzik
94e9ef85c5 pmcstat: implement showing offsets into symbols in top mode
The -I option (and hotkey) is reused for this. Skipping symbol resolution is
moved to the new -A option (and hotkey).

While arguably this violates POLA I think it's a change for the better.
ALso note the -I option was added in head.

Differential Revision:	https://reviews.freebsd.org/D21658
2020-08-05 19:05:49 +00:00
Emmanuel Vadot
ee55186dfd pmc: Add include path for libpmcstat as it is an internallib
Reviewed by:	bapt
Differential Revision:	https://reviews.freebsd.org/D24173
2020-03-25 01:32:16 +00:00
Ryan Moeller
797711a84f libpmcstat: Try /boot/modules if module not found
Modules from ports/pkg are commonly installed to /boot/modules rather than to
the same directory the kernel resides in.  Look there if a module is not found
next to the kernel.

Submitted by:	mmacy
Reported by:	Nick Principe <nap@iXsystems.com>
Approved by:	mmacy (mentor)
MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2020-03-12 23:04:40 +00:00
Warner Losh
52467047aa Regularize the Netflix copyright
Use recent best practices for Copyright form at the top of
the license:
1. Remove all the All Rights Reserved clauses on our stuff. Where we
   piggybacked others, use a separate line to make things clear.
2. Use "Netflix, Inc." everywhere.
3. Use a single line for the copyright for grep friendliness.
4. Use date ranges in all places for our stuff.

Approved by: Netflix Legal (who gave me the form), adrian@ (pmc files)
2019-02-04 21:28:25 +00:00
Matt Macy
f992dd4b5c pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
Matt Macy
b2ca2e50b9 hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
Matt Macy
5de96e33d6 hwpmc: support sampling both kernel and user stacks when interrupted in kernel
This adds the -U options to pmcstat which will attribute in-kernel samples
back to the user stack that invoked the system call. It is not the default,
because when looking at kernel profiles it is generally more desirable to
merge all instances of a given system call together.

Although heavily revised, this change is directly derived from D7350 by
Jonathan T. Looney.

Obtained from: jtl
Sponsored by: Juniper Networks, Limelight Networks
2018-06-04 01:10:23 +00:00
Matt Macy
fbf962e6bb libpmc: allow substring for list and add function for printing event details 2018-06-01 00:45:53 +00:00
Matt Macy
ae266f9f94 pmcstat: suppress uninitialized warning of event 2018-05-27 17:49:27 +00:00
Matt Macy
959826ca1b pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
Matt Macy
5506ceb87f Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
Matt Macy
4928135658 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00
Sean Bruno
e22ee9b247 pmcstat(8)
- Document per thread filtering.

Submitted by:	kbowling
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D15549
2018-05-25 18:11:13 +00:00
Matt Macy
0a87e3ff64 pmcstat: add usage strings for -L and -I 2018-05-24 07:18:49 +00:00
Matt Macy
4bd2a4b58e pmcstat top mode + -I - use fully qualified address 2018-05-24 06:47:04 +00:00
Matt Macy
357405d985 pmcstat: don't compare signed and unsigned 2018-05-24 04:43:40 +00:00
Matt Macy
e98bbcf9ca libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00
Matt Macy
95104040a3 pmc: bump PMC major version to reflect ABI breakage and make warning not require verbose 2018-05-23 19:55:47 +00:00
Matt Macy
0b5dc7f64f hwpmc: add thread id field to callchain and context switch records to allow filtering
on thread in post-processing.

To generate stacks for just ${THREADID}:

pmcstat -R ${PREFIX}.pmcstat -L ${THREADID} -z100 -G ${PREFIX}.stacks

Sponsored by:	Limelight Networks
2018-05-23 17:25:00 +00:00
Matt Macy
821a352a77 pmcstat: add option to not decode the leaf function in top mode
-I will allow the user to see the hot instruction in question
as opposed getting the name of the function
2018-05-22 04:45:46 +00:00
Fabien Thomas
cb43ed2815 Only call flush in pipe mode.
It fixes a crash with a socket in top mode.

Ex:
# pmcstat -R 127.0.0.1:8080 -T -w1
then
# pmcstat -n1 -Sclock.prof -Slock.failed -O 127.0.0.1:8080

MFC after:	1 week
Sponsored by:	Stormshield
2018-01-17 16:55:35 +00:00
Pedro F. Giffuni
1de7b4b805 various: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:37:16 +00:00
Andriy Voskoboinyk
7d41856e52 pmcstat(8): unbreak build with LDFLAGS+=-Wl,--as-needed
libpmcstat.a requires libelf.so; reorder them accordingly.
2017-11-03 00:12:32 +00:00
Bryan Drewery
ea825d0274 DIRDEPS_BUILD: Update dependencies.
Sponsored by:	Dell EMC Isilon
2017-10-31 00:07:04 +00:00
Ruslan Bukin
d27927f731 Extract a set of pmcstat functions and interfaces to the new internal
library -- libpmcstat.

This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.

This allows to reuse code while building new hwpmc(4)-based applications.

Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.

Reviewed by:	kib
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D12718
2017-10-24 16:28:00 +00:00
Konstantin Belousov
e28f262a86 Fix off by one in calculation of the number of buckets for the pc
addresses.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D11864
2017-08-04 18:02:54 +00:00
Pedro F. Giffuni
780154cbaf pmcstat(8); unsign some allocation variables and use reallocarray(3).
Use unsigned values in some internal variables that will be used during
allocation. The variables are used in reduced scope and have no chance of
becoming negative.

Provide bounds checking through reallocarray(3).

MFC after:	2 weeks
2017-04-22 16:02:40 +00:00
Mark Johnston
3176d87c9d Remove an unused field from struct pmcstat_image.
MFC after:	3 days
2016-10-15 23:46:55 +00:00
Stephen J. Kiernan
3ad36a25cf Just remove freeing memory in pmcstat_cleanup, since we only clean up
when we are exiting anyway.

Add NULL checks for all malloc and strdup returns.

Reviewed by:	gnn
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D8002
2016-10-07 21:03:12 +00:00
Ed Schouten
d93c912801 Make code compile when basename() is POSIX compliant.
The POSIX basename() function is allowed to modify its input buffer,
which means its argument is "char *". Pull a copy of the input string
before computing the base.

Reviewed by:	jtl
Differential Revision:	https://reviews.freebsd.org/D6465
2016-05-25 08:45:03 +00:00
Pedro F. Giffuni
d7b1cc0a3c pmcstat: minor spelling fixes.
Mostly comments bur also a user-visible string.

MFC after:	2 weeks
2016-05-01 16:40:56 +00:00
Jonathan T. Looney
cc2ccfbe20 Fix the date on the pmcstat(8) man page from r291016.
Approved by:	gnn (mentor)
MFC after:	1 month
X-MFC with:	r291016
Pointy hat to:	jtl
2015-11-18 15:12:24 +00:00
Jonathan T. Looney
465dadb59b Support a wider history counter in pmcstat(8) gmon output
pmcstat(8) contains an option to output sampling data in a gmon format
compatible with gprof(1). Currently, it uses the default histcounter,
which is an (unsigned short). With large sets of sampling data, it
is possible to overflow the maximum value provided by an (unsigned
short).

This change adds the -e argument to pmcstat. If -e and -g are both
specified, pmcstat will use a histcounter type of uint64_t.

Differential Revision:	https://reviews.freebsd.org/D4151
Reviewed by:	jhb, bjk
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-18 14:52:01 +00:00
Jonathan T. Looney
e64ba55dd2 Change the driver stats to what they really are: unsigned values.
When pmcstat exits after some samples were dropped, give the user an
idea of how many were lost. (Granted, these are global numbers, but
they may still help quantify the scope of the loss.)

Differential Revision:	https://reviews.freebsd.org/D4123
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-16 15:16:09 +00:00
Fabien Thomas
66ca84a7ba Fix for r288176 changes related to debug symbols move.
Reviewed by: emaste
Sponsored by:	Stormshield
2015-10-08 09:46:35 +00:00
John Baldwin
7816cb4a0b Fix an off by one error in r283613: Like regular ffs(), CPU_FFS() returns
1 for CPU 0, etc. so the return value must be decremented to obtain the
first valid CPU ID.

Submitted by:	fabient
MFC after:	1 week
2015-09-01 17:52:43 +00:00
Simon J. Gerraty
ccfb965433 Add META_MODE support.
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.

Still need to add real targets under targets/ to build packages.

Differential Revision:       D2796
Reviewed by: brooks imp
2015-06-13 19:20:56 +00:00
Simon J. Gerraty
44d314f704 dirdeps.mk now sets DEP_RELDIR 2015-06-08 23:35:17 +00:00
John Baldwin
ef16b9abd4 Use the cpuset API more consistently:
- Fetch the root set from cpuset_getaffinity() instead of assuming all CPUs
  from 0 to hw.ncpu are the root set.
- Use CPU_SETSIZE and CPU_FFS.
- The original notion of halted CPUs the manpage and code refers to is gone.
  Use the term "available" instead.

Differential Revision:	https://reviews.freebsd.org/D2491
Reviewed by:	emaste
MFC after:	1 week
2015-05-27 13:54:37 +00:00