Commit Graph

259 Commits

Author SHA1 Message Date
Pedro F. Giffuni
b790c1938d etc: minor spelling fixes.
Mostly comments but also some user-visible strings.

MFC after:	2 weeks
2016-05-02 16:47:28 +00:00
Pedro F. Giffuni
323b076e9c sys: use our nitems() macro when param.h is available.
This should cover all the remaining cases in the kernel.

Discussed in:	freebsd-current
2016-04-21 19:40:10 +00:00
Pedro F. Giffuni
8dfea46460 Remove slightly used const values that can be replaced with nitems().
Suggested by:	jhb
2016-04-21 15:38:28 +00:00
Pedro F. Giffuni
198e7845ee Remove unused e500_event_codes_size.
Found by:	jhb
2016-04-20 20:37:58 +00:00
Pedro F. Giffuni
74b8d63dcc Cleanup unnecessary semicolons from the kernel.
Found with devel/coccinelle.
2016-04-10 23:07:00 +00:00
Justin Hibbits
c4d7f6ab97 Fix a masking bug for e500 PMC.
No idea how this slipped through my regression testing.  pe_code is the event to
count, pe_cpu is the CPU family mask.
2016-04-09 01:02:17 +00:00
Konstantin Belousov
411c83ccd6 If full width writes to the performance monitoring counters are
supported, use full-width aliases MSRs for writes.  This fixes the
"[pmc,X] negative increment" assertion on the context switch when
clipped counter value is sign-extended.

Add definitions for the MSR IA32_PERF_CAPABILITIES needed to detect
the feature.

PR:	207068
Submitted by:	joss.upton@yahoo.com
MFC after:	2 weeks
2016-02-12 07:27:24 +00:00
Konstantin Belousov
0c8cc7b076 Remove tautological cast.
PR:	207068
Submitted by:	joss.upton@yahoo.com
MFC after:	2 weeks
2016-02-12 07:19:59 +00:00
Konstantin Belousov
db57c70a5b Rename P_KTHREAD struct proc p_flag to P_KPROC.
I left as is an apparent bug in ntoskrnl_var.h:AT_PASSIVE_LEVEL()
definition.

Suggested by:	jhb
Sponsored by:	The FreeBSD Foundation
2016-02-09 16:30:16 +00:00
Konstantin Belousov
0fb2c5d60c Do not call vn_fullpath(9) (through the pmc_getfilename() wrapper)
when its result is immediately ignored, i.e. for kernel processes
forked from the user process.  Do not test for non-null before freeing
string.

Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2016-02-06 15:39:04 +00:00
Ruslan Bukin
28029b68c0 Welcome the RISC-V 64-bit kernel.
This is the final step required allowing to compile and to run RISC-V
kernel and userland from HEAD.

RISC-V is a completely open ISA that is freely available to academia
and industry.

Thanks to all the people involved! Special thanks to Andrew Turner,
David Chisnall, Ed Maste, Konstantin Belousov, John Baldwin and
Arun Thomas for their help.
Thanks to Robert Watson for organizing this project.

This project sponsored by UK Higher Education Innovation Fund (HEIF5) and
DARPA CTSRD project at the University of Cambridge Computer Laboratory.

FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv

Reviewed by:	andrew, emaste, kib
Relnotes:	Yes
Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D4982
2016-01-29 15:12:31 +00:00
Justin Hibbits
cdf9344e50 e5500 HWPMC is identical to e500mc, so add support check for it. 2016-01-17 00:14:22 +00:00
Randall Stewart
34d659d314 More fixes in the various intel processors, fixing missing
IAP_F_FM's as well as incorrect umask specifications for
some of the new Broadwell/Skylake PMC's. Also silvermont
had a *lot* of missing IAP_F_FM.

Sponsored by:	Netflix Inc.
2015-12-11 01:21:32 +00:00
Randall Stewart
b01c40f171 Fix the tunable in logging so that if its pre-11 we have the proper
line so the tunable is present.

Sponsored by:	Netflix Inc.
2015-12-09 22:46:40 +00:00
Randall Stewart
f19bae413c Add support for Intel Skylake and Intel Broadwell PMC's. The Broadwell PMC's have been
tested on the Broadwell-Xeon with a hacked up version of pmcstudy -T. I still need
to circle back and add in to pmcstudy all the new tests from the Broadwell Vtune
guide (for the hacked up version I just made it so I could run the -T option). The
Skylake CPU is not yet available (even though Intel is advertising it .. imagine that).
The Skylake PMC's will need to be tested once we can get a sample skylake CPU :-)

Sponsored by: Netflix Inc.
2015-11-30 17:35:49 +00:00
Jonathan T. Looney
5eaa6f01f5 Improve accuracy of PMC sampling frequency
The code tracks a counter which is the number of events until the next
sample. On context switch in, it loads the saved counter. On context
switch out, it tries to calculate a new saved counter.

Problems:

1. The saved counter was shared by all threads in a process. However, this
means that all threads would be initially loaded with the same saved
counter. However, that could result in sampling more often than once every
X number of events.

2. The calculation to determine a new saved counter was backwards. It
added when it should have subtracted, and subtracted when it should have
added. Assume a single-threaded process with a reload count of 1000 events.
Assuming the counter on context switch in was 100 and the counter on context
switch out was 50 (meaning the thread has "consumed" 50 more events), the
code would calculate a new saved counter of 150 (instead of the proper 50).

Fix:

1. As soon as the saved counter is used to initialize a monitor for a
thread on context switch in, set the saved counter to the reload count.
That way, subsequent threads to use the saved counter will get the full
reload count, assuring we sample at least once every X number of events
(across all threads).

2. Change the calculation of the saved counter. Due to the change to the
saved counter in #1, we simply need to add (modulo the reload count) the
remaining counter time we retrieve from the CPU when a thread is context
switched out.

Differential Revision:	https://reviews.freebsd.org/D4122
Approved by:	gnn (mentor)
MFC after:	1 month
Sponsored by:	Juniper Networks
2015-11-16 15:22:15 +00:00
Jonathan T. Looney
c66ea2ee5c Optimizations to the way hwpmc gathers user callchains
Changes to the code to gather user stacks:
* Delay setting pmc_cpumask until we actually have the stack.
* When recording user stack traces, only walk the portion of the ring
  that should have samples for us.

Sponsored by:	Juniper Networks
Approved by:	gnn (mentor)
MFC after:	1 month
2015-11-14 01:45:55 +00:00
Jonathan T. Looney
a39249680f Fix hwpmc "stalled" behavior
Currently, there is a single pm_stalled flag that tracks whether a
performance monitor was "stalled" due to insufficent ring buffer
space for samples. However, because the same performance monitor
can run on multiple processes or threads at the same time, a single
pm_stalled flag that impacts them all seems insufficient.

In particular, you can hit corner cases where the code fails to stop
performance monitors during a context switch out, because it thinks
the performance monitor is already stopped. However, in reality,
it may be that only the monitor running on a different CPU was stalled.

This patch attempts to fix that behavior by tracking on a per-CPU basis
whether a PM desires to run and whether it is "stalled". This lets the
code make better decisions about when to stop PMs and when to try to
restart them. Ideally, we should avoid the case where the code fails
to stop a PM during a context switch out.

Sponsored by:	Juniper Networks
Reviewed by:	jhb
Approved by:	gnn (mentor)
Differential Revision:	https://reviews.freebsd.org/D4124
2015-11-14 01:40:12 +00:00
Bjoern A. Zeeb
71f7442233 Now that we can detect the Cortex-A8 properly, fix the event list
according to the Cortex-A8 TRM r3p2 section 3.2.49.
The A8 list differs from the "ARM-v7 common" list, given the A8
was an earlier model.

There is still more work to be done for other Cortex-Ax version as
andrew points out, but I am just trying to fix A8 for now for teaching.

MFC after:		2 weeks
Sponsored by:		DARPA/AFRL
Obtained from:		Cambridge/L41
Reviewed by:		andrew
Differential Revision:	https://reviews.freebsd.org/D3876
2015-10-14 17:20:19 +00:00
Bjoern A. Zeeb
e6f4757735 When forking a child process with PMC_F_DESCENDANTS set in pmc_attach()
in the parent, we will inherit the pmcids but cannot execute any operations
on them in the child.  The reason for this is that pmc_find_pmc() only
tries to find the current process on the owners hash list, but given the
child does not own the attachment, we cannot find it.
Thus, in case the initial lookup fails, try to find the pmc_process state
affiliated with the child process, lookup the pmc from there using the
row index, and get the owner process from that pmc.
Then continue as normal and lookup the pmc context of the owner (process).

This allows us to call, e.g., pmc_start() in the child process before we
start the work there, but to collect the accumulated results later in
the parent.

Sponsored by:		DARPA,AFRL
Obtained from:		L41
Tested by:		rwatson, L41
MFC after:		4 weeks
Reviewed by:		gnn
Differential Revision:	https://reviews.freebsd.org/D2052
2015-08-24 18:57:32 +00:00
Ruslan Bukin
3e0bfdd882 o Rework ARMv7 events list using aliases - same way as we have for arm64.
o Extend it with Cortex A9-specific events.
2015-06-10 12:42:30 +00:00
Eric van Gyzen
63e4c6cdf9 Provide vnode in memory map info for files on tmpfs
When providing memory map information to userland, populate the vnode pointer
for tmpfs files.  Set the memory mapping to appear as a vnode type, to match
FreeBSD 9 behavior.

This fixes the use of tmpfs files with the dtrace pid provider,
procstat -v, procfs, linprocfs, pmc (pmcstat), and ptrace (PT_VM_ENTRY).

Submitted by:   Eric Badger <eric@badgerio.us> (initial revision)
Obtained from:  Dell Inc.
PR:             198431
MFC after:      2 weeks
Reviewed by:    jhb
Approved by:    kib (mentor)
2015-06-02 18:37:04 +00:00
John Baldwin
a1febbf667 Fix two bugs that could result in PMC sampling effectively stopping.
In both cases, the the effect of the bug was that a very small positive
number was written to the counter. This means that a large number of
events needed to occur before the next sampling interrupt would trigger.
Even with very frequently occurring events like clock cycles wrapping all
the way around could take a long time. Both bugs occurred when updating
the saved reload count for an outgoing thread on a context switch.

First, the counter-independent code compares the current reload count
against the count set when the thread switched in and generates a delta
to apply to the saved count. If this delta causes the reload counter
to go negative, it would add a full reload interval to wrap it around to
a positive value. The fix is to add the full reload interval if the
resulting counter is zero.

Second, occasionally the raw counter value read during a context switch
has actually wrapped, but an interrupt has not yet triggered. In this
case the existing logic would return a very large reload count (e.g.
2^48 - 2 if the counter had overflowed by a count of 2). This was seen
both for fixed-function and programmable counters on an E5-2643.
Workaround this case by returning a reload count of zero.

PR:		198149
Differential Revision:	https://reviews.freebsd.org/D2557
Reviewed by:	emaste
MFC after:	1 week
Sponsored by:	Norse Corp, Inc.
2015-05-19 19:15:19 +00:00
John Baldwin
2b1df86c17 Use the proper mask when reloading sampling PMCs for Core CPUs.
Differential Revision:	https://reviews.freebsd.org/D2492
Reviewed by:	emaste
MFC after:	1 month
2015-05-19 19:01:22 +00:00
John Baldwin
0ceb54c2cf Use fixed enum values for PMC_CLASSES().
This removes one of the frequent causes of ABI breakage when new CPU
types are added to hwpmc(4).

Differential Revision:	https://reviews.freebsd.org/D2586
Reviewed by:	davide, emaste, gnn (earlier version)
MFC after:	2 weeks
2015-05-19 18:58:18 +00:00
Ruslan Bukin
bc88bb2bf3 Add Performance Monitoring Counters support for AArch64.
Family-common and CPU-specific counters implemented.

Supported CPUs: ARM Cortex A53/57/72.

Reviewed by:	andrew, bz, emaste, gnn, jhb
Sponsored by:	ARM Limited
Differential Revision:	https://reviews.freebsd.org/D2555
2015-05-19 15:25:47 +00:00
Bjoern A. Zeeb
62699f3424 Convert remaining hwpmc(4) debug printfs over to KTR to unbreak the build
for at least powerpc kernels.   Missed in r282658.

MFC after:	10 days
2015-05-09 09:21:59 +00:00
John Baldwin
4a3690dfa1 Convert hwpmc(4) debug printfs over to KTR.
Differential Revision:	https://reviews.freebsd.org/D2487
Reviewed by:	davide, emaste
MFC after:	2 weeks
Sponsored by:	Norse Corp, Inc.
2015-05-08 19:40:00 +00:00
John Baldwin
680f1afd94 Move hwpmc(4) debugging code under a new HWPMC_DEBUG option instead of
the broader DEBUG option.

Reviewed by:	emaste
MFC after:	2 weeks
Sponsored by:	Norse Corp, Inc.
2015-05-08 15:57:23 +00:00
Justin Hibbits
a745246822 Implement hwpmc(4) for Freescale e500 core.
This supports e500v1, e500v2, and e500mc. Tested only on e500v2, but the
performance counters are identical across all, with e500mc having some
additional events.

Relnotes:	Yes
2015-04-18 21:39:17 +00:00
Rui Paulo
bc3464096a hwpmc: add initial Intel Broadwell support.
The full list of aliases and events will follow in a subsequent
commit.

MFC after:	1 month
2015-04-05 05:14:20 +00:00
Rui Paulo
03a24b7026 Remove whitespace. 2015-04-05 05:09:38 +00:00
Adrian Chadd
f6e6460dfc Add support for the MIPS74K SoC family performance counters events.
These are similar to the mips24k performance counters - some are
available on perfcnt0/3, some are available on perfcnt1/4.
However, the events aren't all the same.

* Add the events, named the same as from Linux oprofile.
* Verify they're the same as  "MIPS32(R) 74KTM Processor Core Family
  Software User's Manual"; Document Number: MD00519; Revision 01.05.
* Rename INSTRUCTIONS to something else, so it doesn't clash with
  the alias INSTRUCTIONS.  I'll try to tidy this up later; there
  are a few other aliases to add and shuffle around.

Tested:

* QCA9558 SoC (AP135 board) - MIPS74Kc core (no FPU.)
* make universe; where it didn't fail for other reasons.

TODO:

* It'd be nice to support the four performance counters
  in at least this hardware, rather than just two.

Reviewed by:	bsdimp ("looks good; don't break world".)
2015-04-05 02:57:02 +00:00
Bjoern A. Zeeb
54384e56c9 Remove all the handcrafted assembly in hwpmc_armv7.c and use the
common (autogenerated) versions.   Removes extra vertical space,
and makes it easier to grep for usage throughout the tree.
Conditionally compile only for arm6 [1] (yes sounds odd but is right).

Submitted by:	andrew [1]
Reviewed by:	gnn, andrew (ian earlier version I think)
Differential Revision: https://reviews.freebsd.org/D2159
Obtained from:	Cambridge/L41
Sponsored by:	DARPA, AFRL
2015-03-28 18:57:13 +00:00
Bjoern A. Zeeb
0ede88a413 Rather than defining our own magic checks here use INKERNEL() for
the PMC_IN_KERNEL() macro definition.

Add missing macros to extract the return address (LR) from the trapframe.

Discussed with:	andrew
Obtained from:	Cambridge/L41
Sponsored by:	DARPA, AFRL
MFC after:	2 weeks
2015-03-27 08:47:16 +00:00
Ryan Stone
67e51766bd hwpmc: Fix event number to match enum name
Differential revision:	https://reviews.freebsd.org/D1592
Reviewed by:	Joseph Kong
MFC after:	1 month
2015-03-12 23:44:28 +00:00
Randall Stewart
de8d8ca4c8 You need to have the capabilities and not skip it if you are
not on head.. otherwise the file pointer will be NULL and when
you try to do something with it you will crash. Make the #else
be the old capabilites, and then remove the erroneous ifdefs for
11.

MFC after:	1 week (with the other MFC I was going to do until the panic)
2015-03-11 20:15:49 +00:00
Ryan Stone
1761699194 Add missing counter definitions
Differential Revision:	https://reviews.freebsd.org/D1591
MFC after:	1 month
Sponsored by:	Sandvine Inc
2015-03-10 01:24:16 +00:00
Ryan Stone
bc0ad9a99d Fix Ivy Bridge+ MEM_UOPS_RETIRED counters
The MEM_UOPS_RETIRED actually work the same way as the Sandy
Bridge counters, but the counters were documented in a different
way and that seemed to cause the Ivy Bridge counters to be
implemented incorrectly.  Use the same counter definitions as
Sandy Bridge.  While I'm here, rename the counters to match
what's documented in the datasheet.

Differential Revision:	https://reviews.freebsd.org/D1590
MFC after:	1 month
Sponsored by:	Sandvine Inc.
2015-03-10 01:24:08 +00:00
Ryan Stone
e31c323f95 Support architectural events on Haswell/Ivy Bridge
Differential Revision:	https://reviews.freebsd.org/D1589
MFC after:	1 month
Sponsored by:	Sandvine Inc
2015-03-10 01:24:02 +00:00
Ryan Stone
9e60f3acd2 Fix Sandy Bridge+ hwpmc branch counters
On Sandy Bridge and later, to count branch-related events you
have to or together a mask indicating the type of branch
instruction to count (e.g. direct jump, branch, etc) and a bits
indicating whether to count taken and not-taken branches.  The
current counter definitions where defining this bits individually,
so the counters never worked and always just counted 0.

Fix the counter definitions to instead contain the proper
combination of masks.  Also update the man pages to reflect the
new counters.

Differential Revision:	https://reviews.freebsd.org/D1587
MFC after:	1 month
Sponsored by:	Sandvine Inc.
2015-03-10 01:23:47 +00:00
Ryan Stone
89d0633b86 Fix pmc unit restrictions to match documentation
A couple of pmc counters did not work because there were being
restricted to the wrong PMC unit.  I've verified that these
counters now work and match the documented restrictions.

Differential Revision:	https://reviews.freebsd.org/D1586
MFC after:	1 month
Sponsored by:	Sandvine Inc
2015-03-10 01:23:40 +00:00
Ryan Stone
d444c14119 Fix various bugs in Haswell counter definitions
1) The "WALK_COMPLETED_2M_4M" event incorrectly referenced 4K pages.
2) The umask for RING0 and RING123 events was reversed.

Differential Revision:	https://reviews.freebsd.org/D1585
MFC after:	1 month
Sponsored by:	Sandvine Inc
2015-03-10 01:23:34 +00:00
Andrew Turner
90e8f62386 The cpu_id macro was renamed in r278529, catch up with this new name. 2015-02-11 10:37:55 +00:00
Ruslan Bukin
6411d14d62 Add ARMv7 performance monitoring counters.
Differential Revision:	https://reviews.freebsd.org/D1687
Reviewed by:	rpaulo
Sponsored by:	DARPA, AFRL
2015-01-28 16:08:07 +00:00
Ryan Stone
6a429fa5d7 style(9) cleanup 2015-01-22 03:56:23 +00:00
Randall Stewart
d95b3509e1 Update the hwpmc driver to have the new type HASWELL_XEON. Also
go back through HASWELL, IVY_BRIDGE, IVY_BRIDGE_XEON and SANDY_BRIDGE
to straighten out all the missing PMCs. We also add a new pmc tool
pmcstudy, this allows one to run the various formulas from
the documents "Using Intel Vtune Amplifier XE on XXX Generation platforms" for
IB/SB and Haswell. The tool also allows one to postulate your own
formulas with any of the various PMC's. At some point I will enahance
this to work with Brendan Gregg's flame-graphs so we can flamegraph
various PMC interactions. Note the manual page also needs some
work (lots of work) but gnn has committed to help me with that ;-)
Reviewed by: gnn
MFC after:1 month
Sponsored by:	Netflix Inc.
2015-01-14 12:46:58 +00:00
Justin Hibbits
409062f166 Fix hwpmc sampling for ppc970 (G5-class) processors.
With this, hwpmc sampling now works on these processors.

MFC after:	3 weeks
Relnotes:	yes
2014-11-27 18:41:14 +00:00
Justin Hibbits
bd52e21d55 Fix hwpmc sampling for MPC74xxx (G4) processors.
With this, hwpmc sampling now works correctly on these processors.

MFC after:	3 weeks
Relnotes:	yes
2014-11-27 06:42:34 +00:00
Ed Maste
201b654e38 Clamp too-large hwpmc callchaindepth to the maximum
If the depth requested by the user is too large, it's better to provide
the maximum than the smaller default.

Sponsored by:	The FreeBSD Foundation
2014-11-20 23:16:19 +00:00