Commit Graph

19451 Commits

Author SHA1 Message Date
bdrewery
ee7076ae8a Don't use CCACHE for linking.
MFC after:	2 weeks
Sponsored by:	Dell EMC
2018-06-27 19:29:15 +00:00
bdrewery
f0a89858e1 Rework check for libclang_rt to see if the needed library exists.
Currently libclang_rt is not provided for cross-building and as such
is not connected to cross-tools.  For building clang once in universe
it is likely that libclang_rt won't exist for the universe toolchain
but even if it did it would not support anything but the native arch.
So explicitly check for support before enabling h_raw.

MFC after:	1 week
Reviewed by:	dim
Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D16012
2018-06-27 16:56:46 +00:00
oshogbo
b9a00c5d58 [libcasper] Use explicit_bzero instead of memset to clear pwd struct
Submitted by:	David Carlier <devnexen@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D16015
2018-06-26 19:57:47 +00:00
woodsb02
4fcfcecfa8 geli attach multiple providers
Allow attaching of multiple geli providers at once if they use same
passphrase and keyfiles.

This is helpful when the providers being attached are not used for boot,
and therefore the existing code to first try the cached password when
tasting the providers during boot does not apply.

Multiple providers with the same passphrase and keyfiles can be attached
at the same time during system start-up by adding the following to
rc.conf:
  geli_groups="storage backup"
  geli_storage_flags="-k /etc/geli/storage.keys"
  geli_storage_devices="ada0 ada1"
  geli_backup_flags="-j /etc/geli/backup.passfile -k /etc/geli/backup.keys"
  geli_backup_devices="ada2 ada3"

Reviewed by:	wblock, delphij, jilles
Approved by:	sobomax (src), bcr (doc)
Differential Revision:	https://reviews.freebsd.org/D12644
2018-06-26 18:07:16 +00:00
hselasky
43ee0168f3 Improve the userspace USB string reading function in LibUSB.
Some USB devices does not allow a partial descriptor readout.

Found by:		bz @
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2018-06-26 16:00:16 +00:00
brd
a550f635ac Add a file missed in r335645.
Approved by:	brooks, allanjude (mentor)
Differential Revision:	https://reviews.freebsd.org/D15360
2018-06-25 21:54:39 +00:00
brooks
9fa168b01a Normalize the g(eom,cache,part,...) build.
Rather then combining hardlink creation for the geom(8) binary with
shared library build, move libraries to src/lib/geom so they are
built and installed normally.  Create a common Makefile.classes
which is included by both lib/geom/Makefile and sbin/geom/Makefile
so the symlink and libraries stay in sync.

The relocation of libraries allows libraries to be build for 32-bit
compat.  This also reduces the number of non-standard builds in
the system.

This commit is not sufficent to run a 32-bit /sbin/geom on a 64-bit
system out of the box as it will look in the wrong place for libraries
unless GEOM_LIBRARY_PATH is set appropriatly in the environment.

Reviewed by:	bdrewery
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15360
2018-06-25 19:55:15 +00:00
eadler
b75bfde83d users(1): Use capsicum helpers for users
In doing so also fix the libcasper.h header to work in C++.
2018-06-25 11:44:41 +00:00
oshogbo
cfe9f44871 Document libcasper_service.
Reviewed by:	bcr@
Differential Revision:	https://reviews.freebsd.org/D15766
2018-06-23 20:12:30 +00:00
bdrewery
363a71e8bc Revert r335449 and add needed MK_LLD_BOOTSTRAP check for SRCS_MIW.
This effectively reverts r335449 and changes the previous MK_LLD_IS_LD
to a MK_LLD_BOOTSTRAP check.  If !TOOLS_PREFIX then these sources are
always built for llvm-objdump, lld, and llvm-cov.  When TOOLS_PREFIX
is set then they are only needed if lld is being bootstrapped.

Reported by:	dim
Pointyhat to:	bdrewery
Sponsored by:	Dell EMC
2018-06-22 17:58:56 +00:00
dim
6ee1d5b597 Add support for selectively enabling LLVM targets
This makes it possible, through src.conf(5) settings, to select which
LLVM targets you want to build during buildworld.  The current list is:

* (WITH|WITHOUT)_LLVM_TARGET_AARCH64
* (WITH|WITHOUT)_LLVM_TARGET_ARM
* (WITH|WITHOUT)_LLVM_TARGET_MIPS
* (WITH|WITHOUT)_LLVM_TARGET_POWERPC
* (WITH|WITHOUT)_LLVM_TARGET_SPARC
* (WITH|WITHOUT)_LLVM_TARGET_X86

To not influence anything right now, all of these are on by default, in
situations where clang is enabled.

Selectively turning a few targets off manually should work.  Turning on
only one target should work too, even if that target does not correspond
to the build architecture.  (In that case, LLVM_NATIVE_ARCH will not be
defined, and you can only use the resulting clang executable for
cross-compiling.)

I performed a few measurements on one of the FreeBSD.org reference
machines, building clang from scratch, with all targets enabled, and
with only the x86 target enabled.  The latter was ~12% faster in real
time (on a 32-core box), and ~14% faster in user time.  For a full
buildworld the difference will probably be less pronounced, though.

Reviewed by:	bdrewery
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D11077
2018-06-22 15:00:00 +00:00
ian
f1d7fa0336 Add some words clarifying that rename(2) does nothing when the 'from' and
'to' args are the same file.  Wording borrowed from POSIX.1-2017, but
the freebsd code to implement this behavior was added in 2002 (r103180).
2018-06-21 15:21:17 +00:00
bdrewery
025d9a8386 Fix sources needed for lld.
lld always needs these DWARF sources, as well as other default and extra
tools. XDL seems to be the best fit list.

Remove MK_LLD_IS_LD check from SRCS_MIW which is now reduced to just a
few files for llvm-objdump.

Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D15915
2018-06-20 16:10:10 +00:00
bdrewery
dd80b24c72 Rework WITHOUT_LLD/TOOLCHAIN fix from r327892 for cross-tools.
MK_LLD is for the installed lld while MK_LLD_BOOTSTRAP is for the build
tool.  For WITH_SYSTEM_LINKER it is necesarry to separate the logic of
these two.  When building libllvm TOOLS_PREFIX will be defined and
MK_LLD_BOOTSTRAP should be checked instead.

Sponsored by:	Dell EMC
Differential Revision:	https://reviews.freebsd.org/D15837
2018-06-20 16:10:07 +00:00
stevek
fa4fef13c6 This library allows for user space applications to check file descriptors
or paths to see if they can be verified by MAC/veriexec.

Reviewed by:	jtl, wblock
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D8562
2018-06-20 00:55:18 +00:00
oshogbo
7e49a74ead Fix typo. 2018-06-19 22:19:42 +00:00
oshogbo
e554f01d79 libnv: Add nvlist_append_*_array() family of functions.
The nvlist_append_{bool,number,string,nvlist,descriptor}_array() functions
allows to dynamically extend array stored in the nvlist.

Submitted by:	Mindaugas Rasiukevicius <rmind@netbsd.org>
2018-06-18 22:57:32 +00:00
oshogbo
b977ef9915 libnv: add regression test for r335344. 2018-06-18 22:22:06 +00:00
oshogbo
0838a645fc libnv: Remove nvlist argument from cnvlist_{take,free}_* functions.
All information which are need for those operations is already stored in
the cookie.

We decided not to bump libnv version because this API is not used yet in the
base system.

Reviewed by:	pjd
2018-06-18 21:26:58 +00:00
eadler
d5977ae2f0 libcapsicum: adding missing man page links 2018-06-18 04:58:48 +00:00
eadler
b8e5f98342 liby: rewrite yyerror.h
In order to prevent confusion about copyright, rewrite from scratch
yyerror.h. See discussion on r335270 for details.
2018-06-17 18:05:27 +00:00
eadler
d03aadf7dc liby: build with WARNS=6
Tested with amd64, arm64, i386, mips
2018-06-16 23:50:34 +00:00
dteske
3d99be4e82 Fix comparison between pointer and char literal
PR:		misc/204252
Reported by:	David Binderman <dcb314@hotmail.com>
MFC after:	3 days
X-MFC-to:	stable/11, stable/10
2018-06-16 20:00:41 +00:00
kib
de56a5903d Remove unused file.
The thr_new(2) syscall is called through libc stub.  r154212 stopped
generating it for libthr but did not cleaned the unused remnants.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-06-16 17:11:23 +00:00
kib
2f5b4e5085 Rework ofed build.
Aligns the build with the FreeBSD traditional approach to not build in
contrib/, and to track inter-dependencies between libraries.

With help from:	bdrewery
Reviewed by:	bdrewery, hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15648
2018-06-16 15:05:05 +00:00
bdrewery
984e340bf1 Put in temporary hacks for jevents.
- Handle 'make clean'
- Don't try building it locally, just fail instead.

jevents should really be merged into lib/libpmc/Makefile.
2018-06-15 19:19:36 +00:00
araujo
9117ec7c1d Fix style(9) space vs tab.
Reviewed by:	jhb
MFC after:	3 weeks.
Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D15774
2018-06-14 01:28:55 +00:00
trasz
dfe9559a29 Get rid of references to /usr/share/doc/ from ports(7) and getosreldate(3).
The handbooks are not installed there anymore. While here, improve the
URLs markup a bit.

Reviewed by:	allanjude@
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15793
2018-06-13 18:34:49 +00:00
eadler
af86ebebd7 atoi.3: bump .Dd
I touched the man page. Bump Dd by just over 15 years.
2018-06-13 09:07:24 +00:00
eadler
56f7be9651 libc: remove explicit cast NULL in atoi
There isn't any reason to cast NULL so just remove it. Noticed when
cleaning up top.

Reviewed by:	pstef
2018-06-13 08:52:17 +00:00
rlibby
68ad8264a2 pmc gcc fixups
Fix the build of lib/libpmc and usr.sbin/pmc for gcc on amd64.

Reviewed by:    mmacy
Sponsored by:   Dell EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D15723
2018-06-11 16:09:54 +00:00
kib
26c8a03901 libc qsort(3): stop aliasing.
Qsort swap code aliases the sorted array elements to ints and longs in
order to do swap by machine words.  Unfortunately this breaks with the
full code optimization, e.g. LTO.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=83201 which seems to
reference code directly copied from libc/stdlib/qsort.c.

PR:	228780
Reported by:	mliska@suse.cz
Reviewed by:	brooks
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D15714
2018-06-10 17:54:44 +00:00
jhb
d00bc0730a Use __SCSSID() for SCCS IDs.
While here, fix $FreeBSD$ ID in ftime.c to use __FBSDID instead of a static
array.
2018-06-10 02:34:41 +00:00
mmacy
a7768ee751 pmc: fix logic in skipping riscv 2018-06-09 02:25:18 +00:00
mmacy
194dfe17e4 pmc: don't build on riscv where there's no kmod support 2018-06-09 02:15:45 +00:00
mmacy
9d6eb32525 libpmc: add missed (uintmax_t) casts 2018-06-07 03:19:21 +00:00
mmacy
b418d2ef1e pmc: convert native to jsonl and track TSC value of samples
- add '-j' options to filter to enable converting native pmc
  log format to json lines format to enable the use of scripts
  and external tooling

% pmc filter -j pmc.log pmc.jsonl

- Record the tsc value in sampling interrupts as opposed to
  recording nanotime when the sample is copied to a global log
  in hardclock - potentially many milliseconds later.

- At initialize record the tsc_freq and the time of day to give
  us an offset for translating the tsc values in callchain records
2018-06-07 02:03:22 +00:00
mmacy
c8995bf4d0 libpmc: simplify lookup by idx 2018-06-07 00:55:17 +00:00
cem
25d0013e73 strdup.3: Document standardization in POSIX
Sponsored by:	Dell EMC Isilon
2018-06-06 21:32:49 +00:00
cem
cdc7a46eeb strcpy.3: Improve legibility and clarity
In the DESCRIPTION, put the more commonly used functions first in the
corresponding sentence, to help catch the eye.

Pull out the note about overlapping buffers to its own paragraph, as it
applies to all routines documented by this page.

Emphasize the potentially surprising strncpy(3) behavior of zero-filling the
remainder of a buffer larger than the source string.

Encourage strlcpy use; remove portability note about strlcpy(3).  Adapting a
strlcpy-using code base to a platform that does not provide strlcpy in libc
is so trivial as to not be worth mentioning.  (Just copy strlcpy.c out of
any BSD libc, or include and link the pre-packaged libbsd library on non-BSD
platforms.)

Likewise, expand the page's warning about ease of potential misuse to cover
all functions documented herein, and explicitly suggest using strlcpy most
of the time.  The text was mostly cribbed from a similar suggestion in
gets(3).

Finally, document the remaining valid use of strncpy -- the rare
fixed-length record with no expectation of nul-termination.

Sponsored by:	Dell EMC Isilon
2018-06-06 20:09:21 +00:00
cem
1c0f42528c clog.3, complex.3: Fix typos and igor style issues
PR:		228783
Reported by:	Karsten <freebsd-bugzilla AT kkoenig.net>
2018-06-06 16:25:00 +00:00
sbruno
d0aeaa5af7 Load balance sockets with new SO_REUSEPORT_LB option.
This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.

Most of the code was copied from a similar patch for DragonflyBSD.

However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.

Required changes to structures:
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.

Limitations:
As DragonflyBSD, a load balance group is limited to 256 pcbs (256 programs or
threads sharing the same socket).

This is a substantially different contribution as compared to its original
incarnation at svn r332894 and reverted at svn r332967.  Thanks to rwatson@
for the substantive feedback that is included in this commit.

Submitted by:	Johannes Lundberg <johalun0@gmail.com>
Obtained from:	DragonflyBSD
Relnotes:	Yes
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D11003
2018-06-06 15:45:57 +00:00
mmacy
810fccd58a hwpmc: add summary command and further metadata extensions
metadata changes:
- log pmc sample rate with pmcallocate
- log proc flags with thread / process logging
  to identify user vs kernel threads

fixes:
- use log cpuid to translate event id to event name

Implement rudimentary summary command to track sample
counts by thread and process name within a pmc log.

% make -j4 buildkernel >& /dev/null &
% sudo pmcstat -S unhalted_core_cycles -S llc-misses -O foo sleep 15
% pmc summary foo
cpu_clk_unhalted.thread_p_any:
        idle: 138108207162
        clang-6.0: 105336158004
        sh: 72340108510
        make: 8642012963
        kernel: 7754011631
longest_lat_cache.miss:
        clang-6.0: 87502625
        sh: 40901227
        make: 5500165
        kernel: 3300099
        awk: 2000060

%  pmc summary -f ~/foo
idx: 278 name: cpu_clk_unhalted.thread_p_any rate: 2000003
idle: 69054
clang-6.0: 52668
sh: 36170
make: 4321
kernel: 3877
hwpmc: proc(7445): 3319
awk: 1289
xargs: 357
rand_harvestq: 181
mtree: 102
intr: 53
zfskern: 31
usb: 7
pagedaemon: 4
ntpd: 3
syslogd: 1
acpi_thermal: 1
logger: 1
syncer: 1
snmptrapd: 1
sleep: 1
idx: 17 name: longest_lat_cache.miss rate: 100003
clang-6.0: 875
sh: 409
make: 55
kernel: 33
awk: 20
hwpmc: proc(7445): 14
xargs: 9
idle: 8
intr: 3
zfskern: 2
2018-06-06 02:48:09 +00:00
markj
c72e60e430 Don't build brk_test on platforms that don't support brk().
X-MFC with:	r334626
2018-06-05 13:06:06 +00:00
mmacy
2b8bb3bc50 hwpmc: log name->pid, name->tid mappings
By logging all threads and processes 'pmc filter'
can now filter on process or thread name, relieving
the user of the burden of determining which tid or
pid was which when the sample was taken.

% pmc filter -T if_io_tqg -P nginx pmc.log pmc-iflib.log

% pmc filter -x -T idle pmc.log pmc-noidle.log
2018-06-05 04:26:40 +00:00
mmacy
c693dcccda libpmc: improve arg checking 2018-06-04 21:17:46 +00:00
mmacy
794e361402 libpmc / stat: use westmere supported counter 2018-06-04 21:17:42 +00:00
markj
9d9fd255d6 Reimplement brk() and sbrk() to avoid the use of _end.
Previously, libc.so would initialize its notion of the break address
using _end, a special symbol emitted by the static linker following
the bss section.  Compatibility issues between lld and ld.bfd could
cause the wrong definition of _end (libc.so's definition rather than
that of the executable) to be used, breaking the brk()/sbrk()
interface.

Avoid this problem and future interoperability issues by simply not
relying on _end.  Instead, modify the break() system call to return
the kernel's view of the current break address, and have libc
initialize its state using an extra syscall upon the first use of the
interface.  As a side effect, this appears to fix brk()/sbrk() usage
in executables run with rtld direct exec, since the kernel and libc.so
no longer maintain separate views of the process' break address.

PR:		228574
Reviewed by:	kib (previous version)
MFC after:	2 months
Differential Revision:	https://reviews.freebsd.org/D15663
2018-06-04 19:35:15 +00:00
mmacy
a0a00fe3c2 update pmc_pmu_idx_get_by_event signature for !x86 2018-06-04 05:04:40 +00:00
mmacy
537ef3a60f pmc: add filter command
pmc filter allows the user to select event types, threads, and processes from
a sample.

% pmcstat -S unhalted_core_cycles -S llc-misses -S -S  resource_stalls.any -O pmc.log
% pmc filter -e llc-misses pmc.log pmc-llc-misses.log
% pmc filter -e unhalted_core_cycles -t 100339  pmc.log pmc-core-cycles.log
etc...
% pmcstat -R pmc-core-cycles.log -G pmc-core-cycles.stacks
2018-06-04 04:59:48 +00:00