Commit Graph

271 Commits

Author SHA1 Message Date
Simon J. Gerraty
b114e8fcd7 Tweaks for DIRDEPS_BUILD
libmagic only depend on mkmagic if not DIRDEPS_BUILD

libpmc fix -I for libpmcstat

local.dirdeps.mk be even more careful about adding gnu/lib/csu to DIRDEPS

Reviewed by:	bdrewery
Differential Revision:	https://reviews.freebsd.org/D22872
2019-12-19 02:40:04 +00:00
Ed Maste
aab65fc55e libpmc: add MIT SPDX tag to header file
The jevents tool includes a copy of the jsmn json parser which is MIT
licensed.  Upstream the MIT license appears in the jsmn.c source and a
standalone LICENSE file, but the latter is not included in the copy
contained in libpmc and the jsmn.h header carried no license information.
Add an SPDX tag to clarify the situation.
2019-12-12 20:55:43 +00:00
Ed Maste
75d286742d libpmc: convert s390 events data to proper json 2019-12-12 19:37:10 +00:00
Ed Maste
3a3deb00a5 libpmc: convert powerpc event files to proper json 2019-12-12 19:33:16 +00:00
Ed Maste
8b238f4126 libpmc: sort some amdfam17h entries to make valid json 2019-12-12 19:23:38 +00:00
Ed Maste
e641f557bd libpmc: remove undesired prefix from two s390 counters
Two counters included a prefix 'Counter:###\tName:XXX' in their
descriptions that appears to be a leftover from some conversion
process.  Remove them.

Found because a json validator tripped over the tab in the description.
2019-12-12 14:52:37 +00:00
Ed Maste
c039e9596f libpmc: build json event support also on arm64 2019-12-12 00:14:01 +00:00
Ed Maste
1760167c2b libpmc: convert arm64 data files to proper json
jevents includes a very permissive json parser that accepts invalid
json, of which there are many examples in libpmc (typically extra or
missing commas).  Convert the arm64 files to proper json so other tools
can parse them.

Sponsored by:	The FreeBSD Foundation
2019-12-11 22:09:22 +00:00
Simon J. Gerraty
312809fe7f Update dirdeps.mk and gendirdeps.mk
The env space consumed by exporting all libc's .meta files
left little room for command line,
so unexport when done.

Update dirdeps.mk to latest and add
dirdeps-targets.mk to simplify/update targets/Makefile

Makefile changes to go with Makefile.depend changes in D22494

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22495
2019-12-11 17:38:15 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Emmanuel Vadot
f3dbece807 libpmc: Forgot regex.h
Reported by:	ci
MFC after:	1 week
X-MFC-With:	r354549
2019-11-08 17:27:20 +00:00
Emmanuel Vadot
a0ac5706af libpmc: Match on the cpuid with a regex
The CPUID is, or can be, a regex to be matched.
Use regex from libc instead of strcmp

Tested-by:	gallatin
MFC after:	1 week
2019-11-08 16:56:48 +00:00
Ed Maste
6a2a926d5f libpmc: jevents: handle empty descriptoin
PR:		241258
Reported by:	sigsys @ gmail.com
Obtained from:	github.com/andikleen/pmu-tools commit bb3c77ed61
MFC after:	3 days
2019-11-04 22:52:02 +00:00
Dimitry Andric
8b3bc70a2b Merge ^/head r352764 through r353315. 2019-10-08 18:17:02 +00:00
Kyle Evans
036d2e814b Re-add ALLOW_MIPS_SHARED_TEXTREL, sprinkle it around
Diff partially stolen from CheriBSD; these bits need -Wl,-z,notext in order
to build in an LLVM world. They are needed for all flavors/sizes of MIPS.
This will eventually get fixed in LLVM, but it's unclear when.

Reported by:	arichardson, emaste
Differential Revision:	https://reviews.freebsd.org/D21696
2019-10-06 04:19:49 +00:00
Dimitry Andric
0f80acb965 Merge ^/head r352436 through r352536. 2019-09-19 19:26:12 +00:00
Ruslan Bukin
230754ccca Add support for BERI statcounters.
BERI stands for Bluespec Extensible RISC Implementation, based on MIPS.

BERI has not implemented standard MIPS perfomance monitoring counters,
instead it provides statistical counters.

BERI statcounters have a several limitations:
- They can't be written
- They don't support start/stop operation
- None of hardware interrupt is provided on a counter overflow.

So make it separate to hwpmc_mips module and support process/system
counting mode only.

Sponsored by:	DARPA, AFRL
2019-09-18 16:13:50 +00:00
Dimitry Andric
2c3f47a727 Another round of attempting to squelch -Wdeprecated-declarations, which
has become very trigger-happy with libc++ 9.0.0.

It does not help that gcc's implementation of this warning is even more
trigger-happy, in the sense that it already warns on the declaration
itself, not when you are using it.  This is very annoying with our use
of -Wsystem-headers.  That should really be disabled for gcc.
2019-09-17 06:07:08 +00:00
Emmanuel Vadot
4c1a82cea5 pkgbase: Create a FreeBSD-utilities package and make it the default one
The default package use to be FreeBSD-runtime but it should only contain
binaries and libs enough to boot to single user and repair the system, it
is also very handy to have a package that can be tranform to a small mfsroot.
So create a new package named FreeBSD-utilities and make it the default one.
Also move a few binaries and lib into this package when it make sense.
Reviewed by:	bapt, gjb
Differential Revision:	https://reviews.freebsd.org/D21506
2019-09-05 14:15:47 +00:00
Matt Macy
86b5e013ed pmc: restore "unhalted-cycles" alias
Reported by:	mav@
2019-08-15 21:51:11 +00:00
Enji Cooper
b3d01a2ad7 Fix warnings with lib/libpmc
* Use `MIN` instead of similar hand rolled macro.
* Sort headers.
* Use `errno.h` instead of `sys/errno.h`.
* Wrap the argument to sizeof in parentheses for clarity.
* Remove `__BSD_VISIBLE` and `_XOPEN_SOURCE` #defines to mute warnings about
  incompatible snprintf definitions.

This fixes a number of warnings I've been seeing lately in my builds.

Sort makefile variables per style.Makefile(9) (`CFLAGS`/`CWARNFLAG.gcc`) and
bump `WARNS` to 3.

MFC after:	2 weeks
Reviewed by:	jtl
Approved by:	jtl (mentor)
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D19851
2019-04-14 00:06:49 +00:00
Matt Macy
334fd3dabc Fix deterministic builds by sorting input to fts in jevents
Reported by: emaste@
2019-02-05 00:31:25 +00:00
Matt Macy
dacc43df34 Add aditional counter descriptions to AMD 0x17
Submitted by:	Somalapuram Amaranath
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D17401
2018-11-04 06:24:27 +00:00
Alex Richardson
2fa224b17d Fix get_maxfds() in jevents
If RLIM_INFINITY == -1ULL (such as on macOS) the min() call will result
in a value of less than 1 being returned. This causes nftw() to fail
with EINVAL.

While touching this file also fix includes to work on Linux/macOS and don't
declare snprintf since it may have different attributes in the system
headers there.

Reviewed By:	mmacy
Approved By:	brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D17682
2018-10-29 21:08:34 +00:00
Matt Macy
0204d85a62 hwpmc: set default rate if event description lacks one / filter rate against misuse
Not all event descriptions have a sample rate (such as inst_retired.any)
this will restore the legacy behavior of using 65536 in that case. It also
prevents accidental API misuse that could lead to panic.

PR:	230985
Reported by:	markj
Reviewed by:	markj
Approved by:	re (gjb)
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D16958
2018-09-14 01:30:05 +00:00
Matt Macy
3b9156971f re-enable pmcstat, pmccontrol, and pmcannotate for gcc4 builds
I had disabled building of the aforementioned targets due to warnings breaking
tinderbox. This silences the warning and restores them to the build.

Reported by:	jhibbits
Reviewed by:	jhibbits
Approved by:	re (gjb)
2018-09-14 01:11:10 +00:00
Matt Macy
81eb4dcf9e Add library and kernel support for AMD Family 17h counters
NB: lacks default sample rate for most counters
2018-08-14 05:18:43 +00:00
Matt Macy
455d10cc4c pmc amd17h: fix inputs to jevents 2018-08-13 23:46:44 +00:00
Matt Macy
6044931335 Add PMC support for AMD Family CPUs 2018-08-13 22:34:57 +00:00
Warner Losh
c81b12e0d7 Revert r336773: it removed too much.
r336773 removed all things xscale. However, some things xscale are
really armv5. Revert that entirely. A more modest removal will follow.

Noticed by: andrew@
2018-07-27 21:25:01 +00:00
Warner Losh
626930c2fd Remove xscale support
The OLD XSCALE stuff hasn't been useful in a while. The original
committer (cognet@) was the only one that had boards for it. He's
blessed this removal. Newer XSCALE (GUMSTIX) is for hardware that's
quite old. After discussion on arm@, it was clear there was no support
for keeping it.

Differential Review: https://reviews.freebsd.org/D16313
2018-07-27 18:33:09 +00:00
Bryan Drewery
b6b193e9d8 Put in temporary hacks for jevents.
- Handle 'make clean'
- Don't try building it locally, just fail instead.

jevents should really be merged into lib/libpmc/Makefile.
2018-06-15 19:19:36 +00:00
Ryan Libby
2ef5e36473 pmc gcc fixups
Fix the build of lib/libpmc and usr.sbin/pmc for gcc on amd64.

Reviewed by:    mmacy
Sponsored by:   Dell EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D15723
2018-06-11 16:09:54 +00:00
Matt Macy
a72a9036d3 libpmc: add missed (uintmax_t) casts 2018-06-07 03:19:21 +00:00
Matt Macy
f992dd4b5c pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
Matt Macy
1b000b505b libpmc: simplify lookup by idx 2018-06-07 00:55:17 +00:00
Matt Macy
b2ca2e50b9 hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
Matt Macy
ebfaf69cc0 hwpmc: log name->pid, name->tid mappings
By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log
2018-06-05 04:26:40 +00:00
Matt Macy
12193215c2 libpmc: improve arg checking 2018-06-04 21:17:46 +00:00
Matt Macy
ee62e0b419 libpmc / stat: use westmere supported counter 2018-06-04 21:17:42 +00:00
Matt Macy
ce6fc9d779 update pmc_pmu_idx_get_by_event signature for !x86 2018-06-04 05:04:40 +00:00
Matt Macy
bfb46e2ba8 pmc: add filter command
pmc filter allows the user to select event types, threads, and processes from
a sample.

% pmcstat -S unhalted_core_cycles -S llc-misses -S -S  resource_stalls.any -O pmc.log
% pmc filter -e llc-misses pmc.log pmc-llc-misses.log
% pmc filter -e unhalted_core_cycles -t 100339  pmc.log pmc-core-cycles.log
etc...
% pmcstat -R pmc-core-cycles.log -G pmc-core-cycles.stacks
2018-06-04 04:59:48 +00:00
Matt Macy
07d80fd8dc hwpmc: ABI fixes
- increase pmc cpuid field from 8 to 12 bits
- add cpuid version string to initialize entry in the log
  so that filter can identify which counter index an
  event name maps to
- GC unused config flags
- make fixed counter assignment more robust as well as the
  changes needed to be properly identified for filter
2018-06-04 02:05:48 +00:00
Matt Macy
fbf962e6bb libpmc: allow substring for list and add function for printing event details 2018-06-01 00:45:53 +00:00
Matt Macy
cfb0dfa3c2 libpmc: Intel doesn't require runtime counter table init 2018-06-01 00:45:48 +00:00
Matt Macy
96cbd26a24 libpmc/pmu: update aliases table 2018-06-01 00:45:43 +00:00
Matt Macy
e92a1350b5 hwpmc: remove unused pre-table driven bits for intel
Intel now provides comprehensive tables for all performance counters
and the various valid configuration permutations as text .json files.
Libpmc has been converted to use these and hwpmc_core has been greatly
simplified by moving to passthrough of the table values.

The one gotcha is that said tables don't support pentium pro and and pentium
IV. There's very few users of hwpmc on _amd64_ kernels on new hardware. It is
unlikely that anyone is doing low level optimization on 15 year old Intel
hardware. Nonetheless, if someone feels strongly enough to populate the
corresponding tables for p4 and ppro I will reinstate the files in to the
build.

Code for the K8 counters and !x86 architectures remains unchanged.
2018-05-31 22:41:07 +00:00
Matt Macy
7d1c2b74a0 libpmc/pmu: enable for i386 as well 2018-05-31 22:26:55 +00:00
Matt Macy
785dd70da8 libpmc: add pmu support for uncore events 2018-05-31 22:26:50 +00:00
Matt Macy
8ffdc5df52 libpmc: squelch valgrind warnings 2018-05-30 04:12:51 +00:00