Commit Graph

4730 Commits

Author SHA1 Message Date
Rebecca Cran
9a2b7c5fce The efifat files are no longer used: remove the code to build them
Reviewed by:	imp, tsoome, emaste
Differential Revision:	https://reviews.freebsd.org/D20562
2019-09-04 20:55:48 +00:00
Kris Moore
59e50df3cd - Retire pc-sysinstall(8)
https://reviews.freebsd.org/D21094

Submitted by: kmoore@FreeBSD.org
Approved by: imp@FreeBSD.org
2019-09-03 19:42:04 +00:00
Matt Macy
14113f123e Use makefs -t msdos in make_esp_file
With this last piece in place, make -C /usr/src/release release.iso is
finally able to run in a jail. This was not possible before because
msdosfs cannot be mounted inside a jail.

Submitted by:	ryan@ixsystems.com
Reviewed by:	emaste@, imp@, gjb@
MFC after:	1 week
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D21385
2019-09-03 18:37:55 +00:00
John Baldwin
b2e60773c6 Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets.  KTLS only supports
offload of TLS for transmitted data.  Key negotation must still be
performed in userland.  Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option.  All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.

Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type.  Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.

At present, rekeying is not supported though the in-kernel framework
should support rekeying.

KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer.  Each TLS frame is described by a single
ext_pgs mbuf.  The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.

KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.

Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame().  ktls_enqueue() is then
called to schedule TLS frames for encryption.  In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed.  For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().

A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue().  Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.

(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)

KTLS supports pluggable software encryption backends.  Internally,
Netflix uses proprietary pure-software backends.  This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames.  As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.

Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready().  At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.

ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation.  In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session.  TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted.  The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface.  If so, the packet is tagged
with the TLS send tag and sent to the interface.  The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation.  If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped.  In addition, a task is scheduled to refresh the TLS send
tag for the TLS session.  If a new TLS send tag cannot be allocated,
the connection is dropped.  If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag.  (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another.  As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)

ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8).  ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.

Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option.  They can also use this socket
option to toggle between software and ifnet TLS modes.

In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax.  However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.

Various sysctls and counters are available under the kern.ipc.tls
sysctl node.  The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default).  The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.

KTLS is enabled via the KERN_TLS kernel option.

This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.

Reviewed by:	gallatin, hselasky, rrs
Obtained from:	Netflix
Sponsored by:	Netflix, Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
Vladimir Kondratyev
7f32f0e227 iwmbtfw: Firmware loader for Intel Wireless 8260 based Bluetooth USB devices
Currently supported models are: 8260, 8265, 9560, 9260 and 22161.
Firmware files can be installed with comms/iwmbt-firmware port.

PR:			237083
Reviewed by:		hps, emax
X-MFC with:		r351196
Differential Revision:	https://reviews.freebsd.org/D21071
2019-08-18 22:20:28 +00:00
Ed Maste
029d6957cc Remove rlogin/rsh references from src.conf(5) WITHOUT_BLACKLIST_SUPPORT
rcmds were removed in r324351

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-13 14:51:16 +00:00
Conrad Meyer
162022d77d tools/tools/crypto: cryptokeytest: Fix build with newer OpenSSL
Also, drag into this decade.
2019-08-09 02:11:47 +00:00
Leandro Lupori
403c9d52eb [PPC64] Don't mark ld.bfd as obsolete
PowerPC64 still needs ld.bfd for 32-bit binaries/libraries.

This will be needed when ELFv2 becomes default, but there is no harm in
committing it already.

Reviewed by:	jhibbits
Differential Revision:	https://reviews.freebsd.org/D21136
2019-08-05 13:28:21 +00:00
Mark Johnston
fbe12f2085 Include caph_rights_limit() in libegacy if need be.
Reported by:	jenkins
Sponsored by:	The FreeBSD Foundation
2019-08-01 19:26:16 +00:00
Ed Maste
a5ada504b3 remove obsolete kernel debugging script
For quite some time kgdb has been internally handling FreeBSD kernel
module state; add-on scripts and tools are not needed.  asf(8) served
a similar purpose to this script and was removed in r335222.

PR:		229046
Reported by:	jhb
Sponsored by:	The FreeBSD Foundation
2019-08-01 17:02:58 +00:00
Eric van Gyzen
8c5a9161d1 Save the last callout function executed on each CPU
Save the last callout function pointer (and its argument) executed
on each CPU for inspection by a debugger.  Add a ddb `show callout_last`
command to show these pointers.  Add a kernel module that I used
for testing that command.

Relocate `ce_migration_cpu` to reduce padding and therefore preserve
the size of `struct callout_cpu` (320 bytes on amd64) despite the
added members.

This should help diagnose reference-after-free bugs where the
callout's mutex has already been freed when `softclock_call_cc`
tries to unlock it.

You might hope that the pointer would still be available, but it
isn't.  The argument to that function is on the stack (because
`softclock_call_cc` uses it later), and that might be enough in
some cases, but even then, it's very laborious.  A pointer to the
callout is saved right before these newly added fields, but that
callout might have been freed.  We still have the pointer to its
associated mutex, and the name within might be enough, but it might
also have been freed.

Reviewed by:	markj jhb
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D20794
2019-07-03 19:22:44 +00:00
Tijl Coosemans
330014fd65 Also remove lib32 versions of libradius.
MFC after:	1 week
2019-07-03 09:14:39 +00:00
Marcel Moolenaar
da4f5bdaff Add support for Python 3 and make it the default.
Python 2.7 will retire on Januari 1, 2020.
2019-06-30 02:29:12 +00:00
Cy Schubert
015cb6cde4 Create a link to the ipmon.conf.5 man page as documented in ipmon.5.
Add its corresponding optional removal entry.

PR:		238816
MFC after:	1 week
2019-06-27 12:37:44 +00:00
Cy Schubert
74bc7fc0b4 Add the ipmon.5 man page.
PR/238816 initially addressed updates to usage() however the PR has
morphed into a shopping list of updates to usage() and man pages.

PR:		238816 (I added to the list during discussion)
MFC after:	1 week
2019-06-27 02:43:26 +00:00
Warner Losh
804b78634d Mount and unmount devfs around calls to add packages.
pkg now uses /dev/null for some of its operations. NanoBSD's packaging
stuff didn't mount that for the chroot it ran in, so any config that
added packages would see the error:
	pkg: Cannot open /dev/null:No such file or directory
when trying to actually add those packages. It's easy enough for
nanobsd to mount /dev and it won't hurt anything that was already
working and may help things that weren't (like this). I moved the
mount/unmount pair to be in the right push/pop order from the
submitted patch.

PR: 238727
Submitted by: mike tancsa
Tested by: Karl Denninger
2019-06-21 03:49:36 +00:00
Conrad Meyer
c363b16c63 sys: Remove DEV_RANDOM device option
Remove 'device random' from kernel configurations that reference it (most).
Replace perhaps mistaken 'nodevice random' in two MIPS configs with 'options
RANDOM_LOADABLE' instead.  Document removal in UPDATING; update NOTES and
random.4.

Reviewed by:	delphij, markm (previous version)
Approved by:	secteam(delphij)
Differential Revision:	https://reviews.freebsd.org/D19918
2019-06-21 00:16:30 +00:00
Marius Strobl
c36f427632 Add <sys/dnv.h> required for libnv to SYSINCS, too, apparently missed
in r336335.
2019-06-15 09:45:00 +00:00
Dimitry Andric
efc5c4420a Upgrade our copies of clang, llvm, lld, lldb, compiler-rt, libc++,
libunwind and openmp to the upstream release_80 branch r363030
(effectively, 8.0.1 rc2).  The 8.0.1 release should follow this within a
week or so.

MFC after:	2 weeks
2019-06-12 21:10:37 +00:00
Dimitry Andric
687a64222b Vendor import of llvm release_80 branch r363030:
https://llvm.org/svn/llvm-project/llvm/branches/release_80@363030
2019-06-11 18:16:27 +00:00
Jung-uk Kim
f02125cdfd Move llvm-ranlib to appropriate location in OptionalObsoleteFiles.inc.
Note llvm-ar is linked to llvm-ranlib since r311565.  r348677 fixed
"make delete-old" issue with llvm-ar but missed it somehow.

Discussed with:	emaste, jhb
2019-06-06 21:24:40 +00:00
Michael Zhilin
5568e57d72 [ath] [fix] athani compilation was broken by recent ath change
* This commit is going to fix build of AR724x on 13-CURRENT with additional
tools like athani. See r344841
2019-06-05 19:55:30 +00:00
Ed Maste
e4e92b4d5f Use CLANG knob to remove llvm-symbolizer man page
r348504 moved llvm-symbolizer from the CLANG_EXTRAS knob to CLANG, but
the man page was still in the CLANG_EXTRAS section in
OptionalObsoleteFiles.inc.

Reported by:	jhb
MFC after:	3 days
MFC with:	r348504
2019-06-05 14:08:39 +00:00
Ed Maste
7120683d7f move llvm-ar and llvm-nm to appropriate location in OptionalObsoleteFiles.inc
After r348610 `make delete-old` was still removing llvm-ar and llvm-nm
(and associated man pages).

Reported by:	jhb
Sponsored by:	The FreeBSD Foundation
2019-06-05 00:55:22 +00:00
Kyle Evans
71259635f7 llvm-symbolizer: Move out of CLANG_EXTRAS, into CLANG
ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.

Reviewed by:	emaste
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D20484
2019-06-02 04:04:21 +00:00
Ed Maste
b7ebd3eff2 nanobsd: exclude .git (and .hg) in the same places we exclude .svn
Allow support of other VCSes.  Note that two other nanobsd files already
had a similar case, excluding .git and .hg in addition to CVS and .svn.

Sponsored by:	The FreeBSD Foundation
2019-05-24 15:21:23 +00:00
Dimitry Andric
001ec17e68 To avoid unnecessarily modifying ports, add a -lgomp symlink, since GCC
does not ship a -lomp symlink.  Also update OptionalObsoleteFiles for
this, and add 32-bit variants while here.

Submitted by:	jbeich
PR:		237975
MFC after:	3 days
2019-05-19 20:13:55 +00:00
Dimitry Andric
6da8876256 Fix OptionalObsoleteFiles copy/paste mistake from r345236, which
connected libomp to the build.  The comparison should not have been
against ${MK_OPENSSH}, but against ${MK_OPENMP}, obviously.

MFC after:	3 days
2019-05-19 19:42:35 +00:00
Brooks Davis
ffda67901e Change ed(4), ep(4), and fxp(4) examples to em(4).
ed(4) and ep(4) have been removed. fxp(4) remains popular in older
systems, but isn't as future proof as em(4).

Reviewed by:	bz, jhb
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D20311
2019-05-18 21:01:36 +00:00
Brad Davis
1935b8f92e Update beinstall to use the mergemaster/etcupdate from the source tree instead
of the installed one.

Approved by:	will (mentor)
Differential Revision:	https://reviews.freebsd.org/D20291
2019-05-18 17:13:08 +00:00
Ed Maste
dc93026776 revert QEMU q35 platform use from r346748
It seems to be incompatible with the OVMF.fd (of unknown provenance)
in use by the Cirrus-CI config.  We will soon have a known OVMF build
via a port/package (see review D19869) and we can switch back to q35
once packages are available.

Discussed with:	bcran
2019-04-30 15:28:52 +00:00
Alan Somers
f9fffe9e2b fsx: seed more randomly with the -S0 option
When using -S0, seed the PRNG with the current time in nanoseconds, not
seconds, so consecutive runs don't accidentally use the same seed.

Also, rename some variables for clarity.

Reviewed by:	ngie
MFC after:	2 weeks
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D20078
2019-04-28 17:27:06 +00:00
Rebecca Cran
c1ffbf2da0 Fix tools/boot/ci-qemu-test.sh and make some improvements
Update ci-qemu-test.sh

o Update the path to the OVMF file, which is now in /usr/local/share/uefi-edk2-qemu.
o Use the more modern q35, pc-q35-3.0 (Q35 + ICH9, 2009) QEMU machine
  instead of the default, obsolete pc, pc-i440fx-3.0 (i440FX + PIIX, 1996).
  For example this adds ACPI support.
o Specify the system firmware using the newer pflash drive syntax instead
  of bios.
o Remove extra, unneeded devices by passing -nodefaults.
o Change text to talk about 'firmware' instead of 'bios', since UEFI
  isn't a BIOS.

Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D20074
2019-04-26 14:44:46 +00:00
Ian Lepore
0c0ffba5d6 For the geli-gpt-zfs test images, both bios and uefi flavors, add a dummy
ufs partition as p2, and put the zfs partition at p3, to test the ability
of the zfs probe code to find a zfs pool on something other than the first
partition.
2019-04-25 00:08:15 +00:00
Ian Lepore
827f1973cb The zfs module has grown a dependency on zcl_nfs4.ko, so copy it into all
the test images.
2019-04-24 23:52:38 +00:00
Ian Lepore
f576caac25 Complain and exit the script if the 'make install' phase fails. Also,
there is no need to install any debug files.
2019-04-24 23:51:12 +00:00
Rebecca Cran
0661cd79ca Fix install-boot.sh and rootgen.sh UEFI bugs
tools/boot/install-boot.sh was assuming that if a device was passed in,
it should operate on the current system and run efibootmgr etc. to
update the boot manager. However, rootgen.sh passes a md(4) device and
not a fixed disk.

Add a -u option to install-boot.sh to tell it to update the system
in-place and run efibootmgr etc.

Also, source install-boot.sh in rootgen.sh to allow it to find and
call make_esp_file etc. And pass the loader file to make_esp_file instead
of a directory name.

Reported by:	ian
Reviewed by:	ian,imp,tsoome
Differential Revision:	https://reviews.freebsd.org/D19992
2019-04-24 22:35:29 +00:00
Cy Schubert
6389bef8f1 As an interim measure until a more permanent solution is implemented
workaround the following error:

/usr/src/contrib/elftoolchain/strings/strings.c:198:55: error: use of
undeclared identifier
'FA_OPEN' fa = fileargs_init(argc, argv, O_RDONLY, 0, &rights, FA_OPEN);

Reported by:	O. Hartmann <ohartmann@walstatt.org>
Reported by:	Michael Butler <imb@protected-networks.net>
Reported by:	gjb@ & cy@ (implicit)
Reviewed by:	emaste@
Noted by:	rgrimes@
2019-04-18 01:02:00 +00:00
Ed Maste
79b02e7017 Put QEMU CI smoke test boot log in /tmp if TMPDIR not set
Sponsored by:	The FreeBSD Foundation
2019-04-17 19:24:41 +00:00
Ed Maste
297ce4ce2d Install some entropy for QEMU CI smoke test
See r346250 and followup commits and mailing list discussion.
We currently fail to boot properly in the absense of boot-time entropy.

Sponsored by:	The FreeBSD Foundation
2019-04-17 19:16:26 +00:00
Ian Lepore
2614c4835f Allow this test script to be run from within src/tools/boot dir, and create
the temporary image in $TMPDIR.

Allow the script to be run from the src/tools/boot directory by using make
-V SRCTOP to find the top of the tree, because this script is handy for
quick smoke-testing of loader changes, as well as being useful in CI testing.

Also, use a temp directory in $TMPDIR to assemble the boot image, and write
the boot log file to $TMPDIR.  Arrange to have the temporary image clean
itself up, but leave the log file in $TMPDIR for post-mortem analysis of
failures when the script is run interactively.

Differential Revision:	https://reviews.freebsd.org/D19876
2019-04-17 16:27:43 +00:00
Ed Maste
cc2e55c7ea Add a smoke test QEMU boot script for CI
Reviewed by:	imp, lwhsu (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D19661
2019-04-10 13:41:34 +00:00
Tycho Nightingale
e92a9a8c52 ioatcontrol(8) crc-copy flag bug and misc usage tweak
Reviewed by:	cem
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19855
2019-04-09 10:33:18 +00:00
Mitchell Horne
b2689b12b1 Add option to build LLVM RISC-V target
Reviewed by:	emaste, dim
Approved by:	markj (mentor)
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D19759
2019-04-07 18:24:26 +00:00
Mariusz Zaborski
d5f4dd1be7 Add cap_fileargs.h to -legacy if needed.
Reviewed by:	arichardson
Differential Revision:	https://reviews.freebsd.org/D19685
2019-04-04 04:14:20 +00:00
Marcin Wojtas
b0fefb25c5 Create kernel module to parse Veriexec manifest based on envs
The current approach of injecting manifest into mac_veriexec is to
verify the integrity of it in userspace (veriexec (8)) and pass its
entries into kernel using a char device (/dev/veriexec).
This requires verifying root partition integrity in loader,
for example by using memory disk and checking its hash.
Otherwise if rootfs is compromised an attacker could inject their own data.

This patch introduces an option to parse manifest in kernel based on envs.
The loader sets manifest path and digest.
EVENTHANDLER is used to launch the module right after the rootfs is mounted.
It has to be done this way, since one might want to verify integrity of the init file.
This means that manifest is required to be present on the root partition.
Note that the envs have to be set right before boot to make sure that no one can spoof them.

Submitted by: Kornel Duleba <mindal@semihalf.com>
Reviewed by: sjg
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D19281
2019-04-03 03:57:37 +00:00
Tycho Nightingale
a8d9ee9c50 ioatcontrol(8) could exercise 8k-aligned copy with page-break, crc and
crc-copy modes.

Reviewed by:	cem
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D19780
2019-04-02 19:06:25 +00:00
Ed Maste
2a965e2836 revert r302146: makeroot: zero out subsecond component of time= keywords
After r345281 by mhorne subsecond components are supported.

PR:		194703
Sponsored by:	The FreeBSD Foundation
2019-03-27 17:28:23 +00:00
John Baldwin
608289394f Add special psuedo-options for the base/{binutils,gcc} ports.
The WITH_PORT_BASE_{BINUTILS,GCC} options are used to prevent 'make check-old'
and 'make delete-old' from deleting files installed by the base/binutils
and base/gcc packages as normally one disables the in-tree variants
(e.g. WITHOUT_BINUTILS) when using these packages.

Reviewed by:	bapt (earlier version)
Differential Revision:	https://reviews.freebsd.org/D19483
2019-03-26 20:56:12 +00:00
Ed Maste
876d02f135 Add description for WITHOUT_RETPOLINE
Not used by default at the moment, but added for the benefit of
downstream projects/branches with different options.
2019-03-24 23:26:03 +00:00