Commit Graph

1470 Commits

Author SHA1 Message Date
mmacy
0224710b2f pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.
2018-05-26 19:29:19 +00:00
mmacy
274069bbf6 Revert r334242 "pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the"
because of squash commit messages
2018-05-26 19:26:19 +00:00
mmacy
2e56e89840 pmc(3)/hwpmc(4): update supported Intel processors to rely fully on the
vendor provided pmu-events tables and sundry cleanups.

The vendor pmu-events tables provide counter descriptions, default
sample rates, event, umask, and flag values for all the counter
configuration permutations. Using this gives us:

- much simpler kernel code for the MD component
- helpful long and short event descriptions
- simpler user code
- sample rates that won't overload the system

Update man page with newer sample types and remove unused sample type.

Squashed commit of the following:

commit 4459d43eff815bec08ccc5533dbe5de846f03128
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Sat May 26 00:06:31 2018 -0700

    libpmc: fix pmu function signatures for non amd64

commit a2cb8bbc586c65d41f9b291430a2261ec67b59fe
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:38:11 2018 -0700

    pmcstat: fix indentation of usage

commit f686954b15ff56a833ac80404898977cb80a265b
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:49 2018 -0700

    pmclog(3): add callchain and pmcallocatedyn, remove pcsample

commit 73e13a0d2e9498c81c150d14d022050cee7511bb
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:19:00 2018 -0700

    pmclog.h: GC pcsample field

commit 3e93ffd65da641fa657539dad3c48e281f8b5798
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:05:57 2018 -0700

    hwpmc: make Intel core CPUs use external event tables

commit 634f5fae1e1644ac324003136c66cd9c619d1c93
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 22:00:06 2018 -0700

    pmclog: update log record types, bump PMC_MAJOR
    - explicitly make log record types a multiple of 8 bytes
    - hook in pmu event types for pmc_allocate records
    - remove references to no longer PCSAMPLE record

commit 83d84fcd2d65bdf6ddcb2e155a22f0cfa2a9c225
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 21:52:10 2018 -0700

    libpmc: add support for having vendor table driven pmc_allocate

commit 9e6ad63c40c2fce8404847ace5078ca6cb33a736
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:11:33 2018 -0700

    hwpmc_core: add accessors for EVSEL & UMASK, make IAP_UMASK useful to user

commit 859dceb93daa6419a48c794db99b6758e5b041c9
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 19:09:45 2018 -0700

    pmcstat: update usage and man page as well as make -L consistent with pmccontrol

commit 79c7d8597e28c2eb13f5f9113e65ec2792ca57b1
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 18:07:03 2018 -0700

    pmu_util: add support for all current intel event keywords

commit d8089c7f6a6c8527f38324252b1ffb47004694c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:45:00 2018 -0700

    add description for new arguments

commit 058336740bab53c62ec88a3a026ea848cf3878c6
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 17:38:15 2018 -0700

    libpmc: move pmu_events table and pmu_utils out of libpmcstat so that they can be used by pmc_allocate

commit 049b66b382e2f833c3f47bc8df9e750cb265709f
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:12:41 2018 -0700

    pmcstat: hook pmu_events counter description utility routines in

commit f5e01e7b37a691dc045e1aa16b3ebdd162515de8
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:11:59 2018 -0700

    pmu_events: add utility routines for listing counters and their descriptions

commit cba4d4f8907f772279f86f18f915e0d74d33ac56
Author: Matt Macy <mmacy@mattmacy.io>
Date:   Fri May 25 16:09:50 2018 -0700

    pmu-events: expand out skylake regex to simplify string matches
2018-05-26 18:12:50 +00:00
brooks
d904009e53 Support -DNO_CLEAN builds across r334223+r334224. 2018-05-25 20:42:28 +00:00
mmacy
b6f849107a libpmcstat: Don't build pmu tables on !amd64 until the corresponding
util routines have been written and tested. Currently building them
breaks the build on power64

Reported by:	emaste
2018-05-24 21:22:03 +00:00
brooks
e9b5265e7a exect() was removed in 2018, not 2017. 2018-05-24 17:05:41 +00:00
mmacy
da844acd6d libpmcstat: compile in events based on json description 2018-05-24 04:30:06 +00:00
emaste
56f5b3139a Bump min supported release for building HEAD to 10.3
r307825 and r333017 disallow building FreeBSD-HEAD from 9.x or 10.x
versions that have some clang issues.  The minimum supported Subversion
revisions and osreldates containing the fix are:

Version  Min Rev  osreldate
    9.x  r286035     903509
   10.x  r286033    1002501

9.3 is the final 9.x release and does not contain the r286035 fix.
10.3 is the first 10.x release with the fix.  Thus, in practice 10.3 is
the oldest release that can build HEAD.

Although it may still be possible to build HEAD from an up-to-date
stable/9 it's not worth maintaining the special case when the branch
itself is unsupported and there are no usable releases from that branch.

Old UPDATING entries can be removed and the Clang warning in UPDATING
may be updated, in a future commit.

Approved by:	imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15209
2018-04-27 19:50:30 +00:00
emaste
ddbd4530dd Correct r307825 BOOTSTRAPPING test for unsupported upgrade
__FreeBSD_version has five digits after the major number, but the min
versions in the test had an extra 0 (so the test was always false).
2018-04-26 13:18:24 +00:00
brooks
cf97cfc92d Allow -DNO_CLEAN builds across r332443. 2018-04-12 18:24:00 +00:00
brooks
d99229a822 Support -DNO_CLEAN builds across r332119. 2018-04-06 17:23:47 +00:00
brooks
198d9fcdd8 Allow -DNO_CLEAN buildworld after r332080. 2018-04-05 18:19:48 +00:00
emaste
a5aa6a91a2 remove special handling for stale ptrace dependencies
r318957 added special handling for stale ptrace dependency files to
support a -DNO_CLEAN build in a tree last built before r305012.  That
revision is now over a year and a half old, so retire the special case.

Sponsored by:	The FreeBSD Foundation
2018-04-05 03:26:42 +00:00
brooks
d2b366f384 Allow -DNO_CLEAN builds over r332048. 2018-04-04 22:46:30 +00:00
emaste
703d2dbd1f Correct comment date in r331255
Reported by:	jhb
2018-03-20 23:08:04 +00:00
emaste
1cfbfa1fae Remove stale generated assym.s files after r331254
So that -DNO_CLEAN builds work.

This should be reverted after a sufficient transition period (perhaps 3
or 6 months).

Sponsored by:	The FreeBSD Foundation
2018-03-20 18:02:21 +00:00
kevans
1bbebcb5fb pkgbase: Fix post-install script for kernel packages
kernel.ucl uses a hardcoded boot/kernel for kldxref, which is the incorrect
directory when we're installing extra kernels that aren't the "default"
kernel (placed at boot/kernel).

Fix this by instead using a new %KERNELDIR% that we now replace in
Makefile.inc1 with "kernel" for the default kernel and "kernel.${_kernel}"
for these extra kernels so that, e.g. /boot/kernel.SHIVA, will get properly
kldxref'd upon update and avoid outdated linker.hints.

Reviewed by:	gjb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14689
2018-03-14 14:45:57 +00:00
brooks
2144307d26 Improve wording of error message when CROSS_TOOLCHAIN is not found.
Reported by:	emaste, jhb
2018-02-26 19:02:11 +00:00
brooks
2405f0db88 Allow CROSS_TOOLCHAIN to be a path to a file.
This allows working with custom cross toolchains without the need to
create files in /usr/local/share/toolchains.

Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14178
2018-02-25 20:21:30 +00:00
bdrewery
0f6e4bdd0f Properly lookup values if they were empty.
Spotted by:	rpokala
X-MFC-With:	r329676
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 22:03:08 +00:00
bdrewery
c820d3befe Only lookup SVN once and rename to SVN_CMD.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 22:03:04 +00:00
bdrewery
554b0c9dcb Avoid looking up VCS_REVISION for every sub-make call to Makefile.inc1.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-02-20 21:53:39 +00:00
bdrewery
5d18971cc6 Move SVNVERSION_CMD into the one place that uses it.
This code, which is basically `svnversion || svnliteversion`, generates
2 fstatat(2) for every directory in PATH for every Makefile parsed that
includes bsd.own.mk.  This can add up for things like generating a Ports
index (Poudriere) or building a dependency graph for base.

Sponsored by:	Dell EMC
MFC after:	2 weeks
2018-02-20 21:48:16 +00:00
imp
a4b6eaee01 sort needs to be in ITOOLS since find -s was changed to find | sort. 2018-02-09 18:47:00 +00:00
imp
e3dfab111c Add a note about why we have the conditional before including
bsd.compiler.mk. It's so fmake from older 9.x systems still
works (still a supported build config, and having the note here
will let us know when we can cull it more easily).

Also pull in a related change from include to sinclude from
arichardson@'s cross building work, as well as it's companion in
Makefile.inc1 with a note about why we do the odd thing there.

Submitted by: archardson
Differential Revision: https://reviews.freebsd.org/D14241
2018-02-07 16:28:26 +00:00
emaste
384cc05095 add retpoline compiler and linker feature flags
These features indicate that the compiler and linker support the
retpoline speculative execution vulnerability (CVE-2017-5715)
mitigation.

Reviewed by:	dim, imp
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14228
2018-02-07 14:50:06 +00:00
arichardson
20f24e68c5 Don't hardcode /usr/bin as the path for mktemp in build tools
It won't work e.g. when crossbuilding from Ubuntu Linux as mktemp is in
/bin there.

Reviewed By:	bdrewery
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13937
2018-02-06 15:41:35 +00:00
wosch
96e8c3c6da `make installkernel' should display a completed message if done
PR:		225159
Reviewed by:	bdrewery
Approved by: 	cem (mentor)
Differential Revision:	https://reviews.freebsd.org/D13940
2018-01-25 21:36:26 +00:00
bdrewery
25a5722685 test-system-compiler: Display X_ variants for compiler/linker.
Sponsored by:	Dell EMC
2018-01-24 18:09:44 +00:00
bdrewery
3e626dc309 X_COMPILER_* may not be defined.
Sponsored by:	Dell EMC
2018-01-24 18:08:37 +00:00
arichardson
55c1d10911 Don't build share/syscons in build-tools stage if MK_SYSCONS == "no"
Reviewed By:	emaste, jhb
Approved By:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12926
2018-01-16 21:43:36 +00:00
bdrewery
1817cbc328 Ensure MK_LDD for bootstrap-tools matches MK_LLD_BOOTSTRAP.
This fixes building libllvmminimal with MK_LLD support when WITHOUT_LLD
or WITHOUT_TOOLCHAIN are set, which are install flags not build tools
flags.

Reported by:	sbruno
Tested by:	sbruno
Sponsored by:	Dell EMC
2018-01-12 20:40:27 +00:00
bdrewery
a0acbbee94 Cache LINKER_FEATURES to fix the wrong ones being used.
Submitted by:	Dan McGregor <dan.mcgregor@usask.ca>
2018-01-12 01:09:30 +00:00
ian
b2e803acfb Allow use of .WAIT in the LOCAL_DIRS and LOCAL_LIB_DIRS lists.
A comment in Makefile.inc1 has long stated that LOCAL_DIRS are built last,
after the base system.  Incremental improvements in parallel building over
the years have led to LOCAL_DIRS being built in parallel with base system
directories.  This change allows the .WAIT directive to appear in LOCAL_DIRS
and LOCAL_LIB_DIRS lists to give the user some control over parallel
building of local additions.

Differential Revision:	https://reviews.freebsd.org/D13622
2017-12-31 18:53:13 +00:00
emaste
03eda03958 Makefile.inc1: map mips MACHINE_/TARGET_ARCH to triples
This helps Clang MIPS builds.

Reviewed by:	bdrewery, jhb (earlier version)
Differential Revision:	https://reviews.freebsd.org/D12171
2017-12-05 21:42:09 +00:00
bdrewery
ee72627d36 native-xtools: Ensure GCC files are cleaned up.
Because we force enable MK_GCC when building we need to also force
enable it for the cleaning phase.  Otherwise the NXB_TARGET files
are found in the next build's kernel-toolchain phase and cause
an error.

Reported by:	sbruno
X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC
2017-12-05 21:36:08 +00:00
bdrewery
9ec717aaa6 native-xtools: Fix build without META_MODE for GCC archs.
The initial kernel-toolchain is built with TARGET=MACHINE but
we want GCC to have files generated for TARGET=NXB_TARGET
instead later on.  Just clean the files between building of
the toolchain and nxb binaries which will let it rebuild
when needed.

Reported by:	sbruno
X-MFC-With:	r325001
MFC after:	1 month
Sponsored by:	Dell EMC
2017-12-05 02:23:36 +00:00
hselasky
8a118b6cc3 Fix for multi-threaded lib32 compat build on amd64.
Sponsored by:	Mellanox Technologies
2017-11-24 14:29:32 +00:00
hselasky
47c7d18f7b Simplify the build dependencies when building the OFED libraries.
Suggested by:	bdrewery@
Sponsored by:	Mellanox Technologies
2017-11-24 09:01:08 +00:00
hselasky
c6f05b2594 Merge ^/head r325842 through r325998. 2017-11-19 12:36:03 +00:00
bdrewery
540eade088 Fix 'make -n' for {reinstall,distribute}kernel.
PR:		201779
Sponsored by:	Dell
2017-11-18 20:01:02 +00:00
hselasky
8db3ed22bf Update list of hardcoded OFED build order dependencies for
the buildworld target.

Sponsored by:	Mellanox Technologies
2017-11-16 15:13:28 +00:00
emaste
0dfd9255c7 Sort pkgbase mtree metadata, for reproducible builds
Packaged base packages are created by running the stageworld and
stagekernel targets with -DNO_ROOT, and converting the resulting mtree
file into a set of pkg plists.  If stage* is run with multiple processes
the order of entries in the mtree file may be nondeterministic, and the
resulting package tbz also had nondeterministic file ordering.

The mtree file generated by -DNO_ROOT builds consists of one line per
file, with the filename starting in the first column, so is easily
sorted.  There's one exception: the first line of the mtree file is a
comment, but the # character sorts before the filenames anyhow and needs
no special treatment.

PR:		223673
Reviewed by:	bapt, gjb
Sponsored by:	The Linux Foundation, Core Infrastructure Initiative
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D13103
2017-11-15 18:03:31 +00:00
imp
433bd38e3a Move sys/boot to stand. Fix all references to new location
Sponsored by:	Netflix
2017-11-14 23:02:19 +00:00
arichardson
759461bcf2 Only build the libmagic build-tools if MK_FILE != no
Before this patch libmagic was always built in the build-tools phase
even if -DWITHOUT_FILE was specified.

Reviewed by:	emaste, jhb
Approved by:	jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D12925
2017-11-13 20:07:08 +00:00
bdrewery
cc21738f11 cleanworld only needs a tree walk if the expected .OBJDIR is .CURDIR.
Sponsored by:	Dell EMC Isilon
2017-11-11 01:10:08 +00:00
bdrewery
ace01063e0 native-xtools: Fix for GCC archs.
- This also adds in a _cleanobj step as needed.
- This redirects TARGET/TARGET_ARCH to NXB_TARGET/NXB_TARGET_ARCH in
  Makefile.inc1 as the main build needs to be for MACHINE rather
  than TARGET.

  First build the toolchain and then use that as an external toolchain
  to build the needed directories and NXB_TARGET-toolchain, all as
  MACHINE files though via TARGET_TRIPLE=MACHINE_TRIPLE.

  The NXBDIRS is evaluated in the 'everything' submake as it needs to be
  based on TARGET's src.opts.mk values, such as MK_GCC=yes when building
  on a MK_CLANG=yes MACHINE.  This can likely be changed to a specific
  _native-xtools-everything target later and the funky late evaluation
  of SUBDIR_OVERRIDE removed.

X-MFC-With:	r325001
Pointyhat to:	bdrewery
MFC after:	2 months
Sponsored by:	Dell EMC Isilon
2017-11-10 19:53:07 +00:00
bdrewery
fe700bc1e5 rescue ipf: Remove hacks and link in libipf directly.
Sponsored by:	Dell EMC Isilon
2017-11-10 07:52:58 +00:00
bdrewery
e10157c915 Handle some .OBJDIR == .CURDIR cases.
- If OBJROOT is SRCTOP then don't add on TARGET.TARGET_ARCH.  This
  only happens at the top-level, and for sub-directories when the
  user is clever with MAKEOBJDIRPREFIX=/.
- Don't bother checking 'test -w' on .CURDIR.
- Properly set OBJTOP/OBJROOT to SRCTOP in various needed cases.
- Check if the OBJDIR is writable even for *clean* targets since it
  determines which .OBJDIR the user gets;  If they cannot write to an
  existing eligible .OBJDIR then it needs to clean in .CURDIR instead.
- Add guard to cleanworld/cleanuniverse from removing SRCTOP.
- Ensure OBJTOP is proper for .OBJDIR=.CURDIR which fixes finding
  libraries since src.libnames.mk is based on OBJTOP.
- Avoid some chdir(2) for modifying .OBJDIR

Sponsored by:	Dell EMC Isilon
2017-11-10 02:09:33 +00:00
imp
78b05aee6c Note interactive shell errors for make buildenv and add a warning for
people tempted to add back the || true to get rid of them.

Sponsored by: Netflix
2017-11-07 15:01:38 +00:00