Commit Graph

19889 Commits

Author SHA1 Message Date
Emmanuel Vadot
a7b5a3d486 pkgbase: Put a lot of binaries and lib in FreeBSD-runtime
All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
FreeBSD-runtime.

Reviewed by:    bapt, gjb
Differential Revision:  https://reviews.freebsd.org/D21503
2019-09-05 14:13:08 +00:00
Bryan Drewery
5d0866bcea PROGS: Build common sources before recursed PROGS_TARGETS as well when building.
MFC after:	2 weeks
Sponsored by:	DellEMC
2019-09-04 18:32:11 +00:00
Jason Helfman
f4d7986ce1 - correct HISTORY section
- while here clarify wording

PR:		240260 (based on)
Submitted by:	gbergling@gmail.com
MFC after:	after 1 week
2019-09-04 17:52:30 +00:00
Kris Moore
59e50df3cd - Retire pc-sysinstall(8)
https://reviews.freebsd.org/D21094

Submitted by: kmoore@FreeBSD.org
Approved by: imp@FreeBSD.org
2019-09-03 19:42:04 +00:00
Mark Johnston
e46cfc2542 Revert a portion of r351628 that I did not mean to commit.
Reported by:	mjg
MFC with:	r351628
2019-09-03 14:39:36 +00:00
Ian Lepore
385a0efed3 Fix the name of the devicetree bindings document file cited in the manpage.
Reported by:	thj@
2019-09-02 18:32:08 +00:00
Mark Johnston
08cfa56ea3 Extend uma_reclaim() to permit different reclamation targets.
The page daemon periodically invokes uma_reclaim() to reclaim cached
items from each zone when the system is under memory pressure.  This
is important since the size of these caches is unbounded by default.
However it also results in bursts of high latency when allocating from
heavily used zones as threads miss in the per-CPU caches and must
access the keg in order to allocate new items.

With r340405 we maintain an estimate of each zone's usage of its
(per-NUMA domain) cache of full buckets.  Start making use of this
estimate to avoid reclaiming the entire cache when under memory
pressure.  In particular, introduce TRIM, DRAIN and DRAIN_CPU
verbs for uma_reclaim() and uma_zone_reclaim().  When trimming, only
items in excess of the estimate are reclaimed.  Draining a zone
reclaims all of the cached full buckets (the previous behaviour of
uma_reclaim()), and may further drain the per-CPU caches in extreme
cases.

Now, when under memory pressure, the page daemon will trim zones
rather than draining them.  As a result, heavily used zones do not incur
bursts of bucket cache misses following reclamation, but large, unused
caches will be reclaimed as before.

Reviewed by:	jeff
Tested by:	pho (an earlier version)
MFC after:	2 months
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D16667
2019-09-01 22:22:43 +00:00
Ed Maste
6c30aa54c3 Remove CLANG_NO_IAS definition
CLANG_NO_IAS is not used anywhere in the tree.

Sponsored by:	The FreeBSD Foundation
2019-09-01 16:47:48 +00:00
Santhosh Raju
47b9005348 Add myself (fox) and update the mentor information.
Approved by:	philip (mentor)
2019-09-01 15:39:28 +00:00
Jung-uk Kim
1c9c1f5903 Add support for TP-Link Archer T2U Nano.
MFC after:	2 weeks
2019-09-01 06:40:58 +00:00
Sevan Janiyan
e12102f114 Earliest reference to /dev/null I can find is in v4 sh(1) and nulldev in
nsys/ken/subr.c
via TUHS archive
https://minnie.tuhs.org/cgi-bin/utree.pl?file=V4

MFC after:	5 days
2019-08-30 21:49:00 +00:00
Mark Johnston
d794b3a3c2 Update and clean up the UMA man page.
- Fix warnings from igor and mandoc.
- Provide a brief description of the separation between zones and their
  backend slab allocators.
- Document cache zones and secondary zones.
- Document the kernel config options added in r350659.
- Document the uma_zalloc_pcpu() and uma_zfree_pcpu() wrappers.
- Document uma_zone_reserve(), uma_zone_reserve_kva() and
  uma_zone_prealloc().
- Document uma_zone_alloc() and uma_zone_freef().
- Add some missing MLINKs and Xrefs.

MFC after:	2 weeks
2019-08-30 19:35:44 +00:00
Alexander Motin
f4330e6155 Man page for AMD Non-Transparent Bridge (ntb_hw_amd) driver.
Submitted by:	Rajesh Kumar <rajesh1.kumar@amd.com>
Reviewed by:	bcr
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D21462
2019-08-29 13:46:54 +00:00
Edward Tomasz Napierala
5a38af1fb8 Introduce <sys/qmath.h>, a fixed-point math library from Netflix.
This makes it possible to perform mathematical operations
on
fractional values without using floating point. It operates on Q
numbers, which are integer-sized, opaque structures initialized
to hold a chosen number of integer and fractional
bits.


For a general description of the Q number system, see the "Fixed Point
Representation & Fractional Math" whitepaper[1]; for the actual
API see the qmath(3) man page.

This is one of dependencies for the upcoming stats(3) framework[2]
that will be applied to the TCP stack in a later commit.

1. https://www.superkits.net/whitepapers/Fixed%20Point%20Representation%20&%20Fractional%20Math.pdf
2. https://reviews.freebsd.org/D20477

Reviewed by:	bcr (man pages, earlier version), sef (earlier version)
Discussed with:	cem, dteske, imp, lstewart
Sponsored By:	Klara Inc, Netflix
Obtained from:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20116
2019-08-27 11:46:22 +00:00
John Baldwin
b2e60773c6 Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets.  KTLS only supports
offload of TLS for transmitted data.  Key negotation must still be
performed in userland.  Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option.  All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.

Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type.  Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.

At present, rekeying is not supported though the in-kernel framework
should support rekeying.

KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer.  Each TLS frame is described by a single
ext_pgs mbuf.  The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.

KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.

Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame().  ktls_enqueue() is then
called to schedule TLS frames for encryption.  In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed.  For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().

A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue().  Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.

(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)

KTLS supports pluggable software encryption backends.  Internally,
Netflix uses proprietary pure-software backends.  This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames.  As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.

Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready().  At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.

ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation.  In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session.  TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted.  The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface.  If so, the packet is tagged
with the TLS send tag and sent to the interface.  The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation.  If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped.  In addition, a task is scheduled to refresh the TLS send
tag for the TLS session.  If a new TLS send tag cannot be allocated,
the connection is dropped.  If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag.  (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another.  As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)

ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8).  ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.

Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option.  They can also use this socket
option to toggle between software and ifnet TLS modes.

In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax.  However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.

Various sysctls and counters are available under the kern.ipc.tls
sysctl node.  The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default).  The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.

KTLS is enabled via the KERN_TLS kernel option.

This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.

Reviewed by:	gallatin, hselasky, rrs
Obtained from:	Netflix
Sponsored by:	Netflix, Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
Edward Tomasz Napierala
375409b48b .Xr ports(7) and development(7).
MFC after:	2 weeks
2019-08-26 10:16:46 +00:00
Hiroki Sato
0e31d5241d Document soft updates journaling and GEOM-based UFS journaling.
Reviewed by:	mckusick
2019-08-25 18:52:10 +00:00
Konstantin Belousov
d2757137b8 Add nvdimm(4) man page.
Reviewed by:	emaste
Discussed with:	scottl, scottph
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
Differential revision:	https://reviews.freebsd.org/D21386
2019-08-24 15:38:07 +00:00
Warner Losh
7574e8572d Turn off -Werror for gcc 4.2.1 for userland
As discussed on arch@, gcc 4.2.1 is on its way out. Turn off Werror on gcc
versions < 5.0 permantly. This will allow older platforms to continue to compile
w/o new errors once we take them out of universe by default. This will also free
developers from chasing down obsolete warnings that produce no beneficial
changes to the source.

Discussed on: arch@
Reviewed by: jhb@, emaste@, pfg@
Differential Revision: https://reviews.freebsd.org/D21378
2019-08-23 16:42:39 +00:00
Mark Johnston
5b699f1614 Add lockmgr(9) probes to the lockstat DTrace provider.
They follow the conventions set by rw and sx lock probes.  There is
an additional lockstat:::lockmgr-disown probe.

Update lockstat(1) to report on contention and hold events for
lockmgr locks.  Document the new probes in dtrace_lockstat.4, and
deduplicate some of the existing probe descriptions.

Reviewed by:	mjg
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21355
2019-08-21 23:43:58 +00:00
Warner Losh
51c5de5f25 Document RST support in nvme(4) and ahci(4). 2019-08-21 22:18:07 +00:00
John Baldwin
93a0379392 Trim a spurious blank line I added in r348969.
I did not bump .Dd since there is no content change.

MFC after:	3 days
2019-08-19 17:28:12 +00:00
Vincenzo Maffione
421c620837 link ptnet(4) man page to the build system
Reported by:	kevans
Reviewed by:	kevans, bcr
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21312
2019-08-19 17:26:04 +00:00
Konstantin Belousov
3a91d1062a i386: Implement atomic_load_64(9) and atomic_store_64(9).
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-18 15:58:44 +00:00
Ed Maste
2fe25d1638 arch.7: indicate expectation that arm (i.e., v5) will be removed before 13.0
MFC after:	1 week
2019-08-16 14:06:04 +00:00
Ed Maste
3a26da0b5c Regen src.conf.5 after r351027 (enable BSD_CRTBEGIN on powerpc) 2019-08-16 13:50:44 +00:00
Alexander Motin
105a032977 Limit memory window usage in ntb_transport to 256MB.
This adds safety net for the case of misconfigured NTB with too big
memory window, for which we may be unable to allocate a memory buffer,
which does not make much sense for the network interface.  While there,
fix the code to really work with asymmetric window sizes setup.

This makes driver just print warning message on boot instead of hanging
if too large memory window is configured.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-08-15 22:55:59 +00:00
Alexander Motin
63ac15aba4 Add NTB modules to i386 build.
There is no reason why NTB should not be usable on i386 if memory windows
are small enough.
2019-08-15 16:27:04 +00:00
Ed Maste
e5c6dece98 do not enable userland retpoline if not supported by compiler/linker
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21101
2019-08-15 12:48:17 +00:00
Simon J. Gerraty
5c5316098e Revert prior change till installworld sorted 2019-08-15 06:00:55 +00:00
Doug Moore
504f5e294e swap_pager.c reserves 2 blocks for a bsd label. Change that 2 to the
expression howmany(BBSIZE, PAGE_SIZE), where BBSIZE is the size of the
boot block area.  That can be less than 2 if PAGE_SIZE is big.

swapon(8) has an option to trim (delete) all the blocks of a device at
startup.  However, if the first of those blocks is a bsd label, then
trimming those blocks is destructive.  Change swapon to leave the
first BBSIZE bytes untrimmed.

Update manual pages to reflect changes in how swapon and how it may be
used, espeically in association with savecore.

Reviewed by: alc
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D21191
2019-08-15 02:30:44 +00:00
Ed Maste
c06ccdd2ba arch.7: be explicit about the future for unmigrated GCC 4.2.1 archs
Reported by:	imp
MFC after:	1 week
MFC with:	r351036
2019-08-15 02:14:44 +00:00
Simon J. Gerraty
f4ff569996 bsd.files.mk: fix targets to avoid directories
Reintroduce :T when file is used as part of a target name.

Reviewed by:	stevek
2019-08-14 22:33:46 +00:00
Ed Maste
4ef280656c arch.7: clarify that the table shows the default tool chain
Reported by:	jhb
MFC after:	1 week
MFC with:	r351035
Sponsored by:	The FreeBSD Foundation
2019-08-14 17:17:52 +00:00
Ed Maste
ff03c71275 arch.7: add deprecation notice for GCC 4.2.1
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-14 17:13:36 +00:00
Ed Maste
4715c31e3e arch.7: add tool chain table
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-14 17:04:28 +00:00
Andrew Turner
4ee71ec9eb Enable BSD_CRTBEGIN on powerpc
In r342974 jhibbits added support to build crtsavres.o. This was the
blocker for BSD_CRTBEGIN to be enabled there. As such enable this
option again.

Reviewed by:	jhibbits
Sponsored by:	DARPA, AFRL
2019-08-14 14:31:17 +00:00
Conrad Meyer
eefd8f96fb geom_uzip(4), mkuzip(8): Add Zstd image mode
The Zstd format bumps the CLOOP major number to 4 to avoid incompatibility
with older systems.  Support in geom_uzip(4) is conditional on the ZSTDIO
kernel option, which is enabled in amd64 GENERIC, but not all in-tree
configurations.

mkuzip(8) was modified slightly to always initialize the nblocks + 1'th
offset in the CLOOP file format.  Previously, it was only initialized in the
case where the final compressed block happened to be unaligned w.r.t.
DEV_BSIZE.  The "Fake" last+1 block change in r298619 means that the final
compressed block's 'blen' was never correct unless the compressed uzip image
happened to be BSIZE-aligned.  This happened in about 1 out of every 512
cases.  The zlib and lzma decompressors are probably tolerant of extra trash
following the frame they were told to decode, but Zstd complains that the
input size is incorrect.

Correspondingly, geom_uzip(4) was modified slightly to avoid trashing the
nblocks + 1'th offset when it is known to be initialized to a good value.
This corrects the calculated final real cluster compressed length to match
that printed by mkuzip(8).

mkuzip(8) was refactored somewhat to reduce code duplication and increase
ease of adding other compression formats.

  * Input block size validation was pulled out of individual compression
    init routines into main().

  * Init routines now validate a user-provided compression level or select
    an algorithm-specific default, if none was provided.

  * A new interface for calculating the maximal compressed size of an
    incompressible input block was added for each driver.  The generic code
    uses it to validate against MAXPHYS as well as to allocate compression
    result buffers in the generic code.

  * Algorithm selection is now driven by a table lookup, to increase ease of
    adding other formats in the future.

mkuzip(8) gained the ability to explicitly specify a compression level with
'-C'.  The prior defaults -- 9 for zlib and 6 for lzma -- are maintained.
The new zstd default is 9, to match zlib.

Rather than select lzma or zlib with '-L' or its absense, respectively, a
new argument '-A <algorithm>' is provided to select 'zlib', 'lzma', or
'zstd'.  '-L' is considered deprecated, but will probably never be removed.

All of the new features were documented in mkuzip.8; the page was also
cleaned up slightly.

Relnotes:	yes
2019-08-13 23:32:56 +00:00
Conrad Meyer
ac8e5d02cf Remove deprecated GEOM classes
Follow-up on r322318 and r322319 and remove the deprecated modules.

Shift some now-unused kernel files into userspace utilities that incorporate
them.  Remove references to removed GEOM classes in userspace utilities.

Reviewed by:	imp (earlier version)
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D21249
2019-08-13 20:06:55 +00:00
Ed Maste
76acea155a Remove some more leftover rlogin man page xrefs
rcmds were removed in r32435 and these three man pages can trivially
drop the references.

There's still a reference in pts.4 because it describes a mode
(TIOCPKT_NOSTOP), and only lists rlogin/rlogind as examples of programs
that use that mode.  To update later.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-13 15:41:36 +00:00
Ed Maste
85d553093f Regen src.conf.5 after r350980 (remove rsh/rlogin references)
Also pick up changes to LLVM_TARGET_RISCV, NAND, NVME, OPENM

Sponsored by:	The FreeBSD Foundation
2019-08-13 14:57:04 +00:00
Ed Maste
7e527d268e Remove rsh/rlogin references from security man page
More extensive changes to this page are certainly needed, but at least
remove references to binaries that no longer exist.

MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-13 14:47:24 +00:00
Ian Lepore
649e762762 Fix the driver name in ads111x.4, and hook the manpage up to the build.
The driver was originally written with the name ads1115, but at the last
minute it got renamed to ads111x to reflect its support for many related
chips, but I forgot to update the manpage to match the renaming before
committing it all.
2019-08-13 03:49:53 +00:00
Pedro F. Giffuni
98a43f9e22 Add deprecation notice to snd_maestro(4).
As suggested in:
https://wiki.freebsd.org/WhatsGoing/FreeBSD13

this old driver is buggy and no one is working on it so we should deprecate
it for the next release.

MFC after:	3 days
Diferential Revision:	https://reviews.freebsd.org/D21139
2019-08-13 01:25:07 +00:00
Pedro F. Giffuni
8068df3535 Add deprecation notice to snd_ds1(4).
As suggested in:
https://wiki.freebsd.org/WhatsGoing/FreeBSD13

We will be dropping the snd_ds1 driver. The driver is known to be buggy
and no one has been working on it for years now.
Users of old Yamaha cards may have luck with the OSS drivers instead.

MFC after:	3 days
Diferential Revision:	https://reviews.freebsd.org/D21138
2019-08-13 01:24:43 +00:00
Conrad Meyer
47fec6f3b5 r350739 try #2
For some inexplicable reason, C++ compilers reject the -Wno- flag, and also
(ab)use CWARNFLAGS.

Reported by:	imp
2019-08-08 04:29:46 +00:00
Conrad Meyer
be3ab1876e Disable useless -Wformat-zero-length
It is part of -Wformat, which is enabled by -Wall.  Empty format strings are
well defined and it is perfectly reasonable to expect them in a formatting
interface.
2019-08-08 03:27:46 +00:00
Li-Wen Hsu
26a6feda9c Follow r350693 to add a link for sbuf_nl_terminate(9)
Sponsored by:	The FreeBSD Foundation
2019-08-08 00:51:17 +00:00
Andriy Gapon
09b238e7c9 rc.8: fix a typo in r350701
I wronlgy expected that igor(1) checks spelling of all words.

MFC after:	1 week
2019-08-07 21:20:25 +00:00
Andriy Gapon
ce6c81b9d8 rc.8: add a reference to service(8)
While here:
- use Xr macro for a reference to sysrc(8)
- clarify that rc.shutdown invokes rc.d scripts with "faststop" argument

MFC after:	1 week
2019-08-07 21:07:59 +00:00