Commit Graph

19901 Commits

Author SHA1 Message Date
Simon J. Gerraty
f18ca7f311 Use file destdir for stage_as sets
We cannot use file (without :T) to name targets
but we can use the destination directory (with / replaced by _)
This has the benefit of minimizing the targets created.

Reviewed by:	bdrewery
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org//D21283
2019-09-06 19:05:01 +00:00
Mateusz Piotrowski
bb66a3e948 rc.8: Update documentation regarding old-style rc scripts
The EXAMPLES section does not contain any examples of output formats for
the old-style scripts. Remove the misleading bits stating otherwise.

Reviewed by:	bcr, imp
Approved by:	src (imp)
Differential Revision:	https://reviews.freebsd.org/D21552
2019-09-06 18:51:41 +00:00
Scott Long
e996e2bd6e Add myself as a mentor for Scott Phillip (scottph) 2019-09-06 17:06:48 +00:00
Emmanuel Vadot
725ee594b4 pkgbase: r351861 didn't solve everything, we need to default to the utilities package too 2019-09-06 12:26:45 +00:00
Takanori Watanabe
ca78bcd73b Add embedded Managed Object Format blob access to acpi_wmi(4).
This blob is can be converted to human readable form by bmfdec.
(http://github.com/pali/bmfdec)

Differential Revision:	https://reviews.freebsd.org/D21529
2019-09-06 10:12:05 +00:00
D Scott Phillips
ed4e4cff0e nvdimm(4): Add description of NVDIMM Namespace support
Reviewed by:	kib, bcr
Approved by:	emaste (mentor)
MFC after:	3 days
Sponsored by:	Intel Corporation
Differential Revision:	https://reviews.freebsd.org/D21536
2019-09-05 20:51:41 +00:00
Emmanuel Vadot
f6a3b357e9 pkgbase: Add the pkg trusted keys to the FreeBSD-utilities package 2019-09-05 19:17:17 +00:00
Mateusz Piotrowski
d15e810db9 rc: Honor ${name}_env when a custom *_cmd is defined (e.g., start_cmd)
A user may set ${name}_env variable in rc.conf(5) in order to set additional
environment variables for a service command.  Unfortunately, at the moment
this variable is only honored when the command is specified via the command
variable. Those additional environment variables coming from ${name}_env
are never set if the service is started via the ${rc_arg}_cmd variable (for
example start_cmd).

PR:		239692
Reviewed by:	bcr, jilles
Approved by:	src (jilles)
Differential Revision:	https://reviews.freebsd.org/D21228
2019-09-05 14:52:22 +00:00
Emmanuel Vadot
5112cfa07b pkgbase: Handle FILES when no FILESGROUP isn't used
bsd.files.mk only add the TAG when groups are used, fix this.

Reviewed by:	gjb
Differential Revision:	https://reviews.freebsd.org/D21520
2019-09-05 14:18:13 +00:00
Emmanuel Vadot
ce49128a13 pkgbase: Add tag for LIBSYMLINK case
Otherwised the files aren't packaged.

Reviewed by:	bapt, gjb
Differential Revision:	https://reviews.freebsd.org/D21508
2019-09-05 14:17:28 +00:00
Emmanuel Vadot
dba1420463 pkgbase: Add tags for includes in bsd.incs.mk
Otherwise the files aren't packaged.

Reviewed by:	bapt, gjb
Differential Revision:	https://reviews.freebsd.org/D21507
2019-09-05 14:16:39 +00:00
Emmanuel Vadot
4c1a82cea5 pkgbase: Create a FreeBSD-utilities package and make it the default one
The default package use to be FreeBSD-runtime but it should only contain
binaries and libs enough to boot to single user and repair the system, it
is also very handy to have a package that can be tranform to a small mfsroot.
So create a new package named FreeBSD-utilities and make it the default one.
Also move a few binaries and lib into this package when it make sense.
Reviewed by:	bapt, gjb
Differential Revision:	https://reviews.freebsd.org/D21506
2019-09-05 14:15:47 +00:00
Emmanuel Vadot
a7b5a3d486 pkgbase: Put a lot of binaries and lib in FreeBSD-runtime
All of them are needed to be able to boot to single user and be able
to repair a existing FreeBSD installation so put them directly into
FreeBSD-runtime.

Reviewed by:    bapt, gjb
Differential Revision:  https://reviews.freebsd.org/D21503
2019-09-05 14:13:08 +00:00
Bryan Drewery
5d0866bcea PROGS: Build common sources before recursed PROGS_TARGETS as well when building.
MFC after:	2 weeks
Sponsored by:	DellEMC
2019-09-04 18:32:11 +00:00
Jason Helfman
f4d7986ce1 - correct HISTORY section
- while here clarify wording

PR:		240260 (based on)
Submitted by:	gbergling@gmail.com
MFC after:	after 1 week
2019-09-04 17:52:30 +00:00
Kris Moore
59e50df3cd - Retire pc-sysinstall(8)
https://reviews.freebsd.org/D21094

Submitted by: kmoore@FreeBSD.org
Approved by: imp@FreeBSD.org
2019-09-03 19:42:04 +00:00
Mark Johnston
e46cfc2542 Revert a portion of r351628 that I did not mean to commit.
Reported by:	mjg
MFC with:	r351628
2019-09-03 14:39:36 +00:00
Ian Lepore
385a0efed3 Fix the name of the devicetree bindings document file cited in the manpage.
Reported by:	thj@
2019-09-02 18:32:08 +00:00
Mark Johnston
08cfa56ea3 Extend uma_reclaim() to permit different reclamation targets.
The page daemon periodically invokes uma_reclaim() to reclaim cached
items from each zone when the system is under memory pressure.  This
is important since the size of these caches is unbounded by default.
However it also results in bursts of high latency when allocating from
heavily used zones as threads miss in the per-CPU caches and must
access the keg in order to allocate new items.

With r340405 we maintain an estimate of each zone's usage of its
(per-NUMA domain) cache of full buckets.  Start making use of this
estimate to avoid reclaiming the entire cache when under memory
pressure.  In particular, introduce TRIM, DRAIN and DRAIN_CPU
verbs for uma_reclaim() and uma_zone_reclaim().  When trimming, only
items in excess of the estimate are reclaimed.  Draining a zone
reclaims all of the cached full buckets (the previous behaviour of
uma_reclaim()), and may further drain the per-CPU caches in extreme
cases.

Now, when under memory pressure, the page daemon will trim zones
rather than draining them.  As a result, heavily used zones do not incur
bursts of bucket cache misses following reclamation, but large, unused
caches will be reclaimed as before.

Reviewed by:	jeff
Tested by:	pho (an earlier version)
MFC after:	2 months
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D16667
2019-09-01 22:22:43 +00:00
Ed Maste
6c30aa54c3 Remove CLANG_NO_IAS definition
CLANG_NO_IAS is not used anywhere in the tree.

Sponsored by:	The FreeBSD Foundation
2019-09-01 16:47:48 +00:00
Santhosh Raju
47b9005348 Add myself (fox) and update the mentor information.
Approved by:	philip (mentor)
2019-09-01 15:39:28 +00:00
Jung-uk Kim
1c9c1f5903 Add support for TP-Link Archer T2U Nano.
MFC after:	2 weeks
2019-09-01 06:40:58 +00:00
Sevan Janiyan
e12102f114 Earliest reference to /dev/null I can find is in v4 sh(1) and nulldev in
nsys/ken/subr.c
via TUHS archive
https://minnie.tuhs.org/cgi-bin/utree.pl?file=V4

MFC after:	5 days
2019-08-30 21:49:00 +00:00
Mark Johnston
d794b3a3c2 Update and clean up the UMA man page.
- Fix warnings from igor and mandoc.
- Provide a brief description of the separation between zones and their
  backend slab allocators.
- Document cache zones and secondary zones.
- Document the kernel config options added in r350659.
- Document the uma_zalloc_pcpu() and uma_zfree_pcpu() wrappers.
- Document uma_zone_reserve(), uma_zone_reserve_kva() and
  uma_zone_prealloc().
- Document uma_zone_alloc() and uma_zone_freef().
- Add some missing MLINKs and Xrefs.

MFC after:	2 weeks
2019-08-30 19:35:44 +00:00
Alexander Motin
f4330e6155 Man page for AMD Non-Transparent Bridge (ntb_hw_amd) driver.
Submitted by:	Rajesh Kumar <rajesh1.kumar@amd.com>
Reviewed by:	bcr
MFC after:	3 days
Differential Revision:	https://reviews.freebsd.org/D21462
2019-08-29 13:46:54 +00:00
Edward Tomasz Napierala
5a38af1fb8 Introduce <sys/qmath.h>, a fixed-point math library from Netflix.
This makes it possible to perform mathematical operations
on
fractional values without using floating point. It operates on Q
numbers, which are integer-sized, opaque structures initialized
to hold a chosen number of integer and fractional
bits.


For a general description of the Q number system, see the "Fixed Point
Representation & Fractional Math" whitepaper[1]; for the actual
API see the qmath(3) man page.

This is one of dependencies for the upcoming stats(3) framework[2]
that will be applied to the TCP stack in a later commit.

1. https://www.superkits.net/whitepapers/Fixed%20Point%20Representation%20&%20Fractional%20Math.pdf
2. https://reviews.freebsd.org/D20477

Reviewed by:	bcr (man pages, earlier version), sef (earlier version)
Discussed with:	cem, dteske, imp, lstewart
Sponsored By:	Klara Inc, Netflix
Obtained from:	Netflix
Differential Revision:	https://reviews.freebsd.org/D20116
2019-08-27 11:46:22 +00:00
John Baldwin
b2e60773c6 Add kernel-side support for in-kernel TLS.
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets.  KTLS only supports
offload of TLS for transmitted data.  Key negotation must still be
performed in userland.  Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option.  All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.

Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type.  Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.

At present, rekeying is not supported though the in-kernel framework
should support rekeying.

KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer.  Each TLS frame is described by a single
ext_pgs mbuf.  The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.

KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.

Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame().  ktls_enqueue() is then
called to schedule TLS frames for encryption.  In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed.  For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().

A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue().  Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.

(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)

KTLS supports pluggable software encryption backends.  Internally,
Netflix uses proprietary pure-software backends.  This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames.  As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.

Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready().  At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.

ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation.  In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session.  TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted.  The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface.  If so, the packet is tagged
with the TLS send tag and sent to the interface.  The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation.  If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped.  In addition, a task is scheduled to refresh the TLS send
tag for the TLS session.  If a new TLS send tag cannot be allocated,
the connection is dropped.  If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag.  (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another.  As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)

ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8).  ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.

Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option.  They can also use this socket
option to toggle between software and ifnet TLS modes.

In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax.  However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.

Various sysctls and counters are available under the kern.ipc.tls
sysctl node.  The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default).  The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.

KTLS is enabled via the KERN_TLS kernel option.

This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.

Reviewed by:	gallatin, hselasky, rrs
Obtained from:	Netflix
Sponsored by:	Netflix, Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D21277
2019-08-27 00:01:56 +00:00
Edward Tomasz Napierala
375409b48b .Xr ports(7) and development(7).
MFC after:	2 weeks
2019-08-26 10:16:46 +00:00
Hiroki Sato
0e31d5241d Document soft updates journaling and GEOM-based UFS journaling.
Reviewed by:	mckusick
2019-08-25 18:52:10 +00:00
Konstantin Belousov
d2757137b8 Add nvdimm(4) man page.
Reviewed by:	emaste
Discussed with:	scottl, scottph
Sponsored by:	The FreeBSD Foundation
MFC after:	3 days
Differential revision:	https://reviews.freebsd.org/D21386
2019-08-24 15:38:07 +00:00
Warner Losh
7574e8572d Turn off -Werror for gcc 4.2.1 for userland
As discussed on arch@, gcc 4.2.1 is on its way out. Turn off Werror on gcc
versions < 5.0 permantly. This will allow older platforms to continue to compile
w/o new errors once we take them out of universe by default. This will also free
developers from chasing down obsolete warnings that produce no beneficial
changes to the source.

Discussed on: arch@
Reviewed by: jhb@, emaste@, pfg@
Differential Revision: https://reviews.freebsd.org/D21378
2019-08-23 16:42:39 +00:00
Mark Johnston
5b699f1614 Add lockmgr(9) probes to the lockstat DTrace provider.
They follow the conventions set by rw and sx lock probes.  There is
an additional lockstat:::lockmgr-disown probe.

Update lockstat(1) to report on contention and hold events for
lockmgr locks.  Document the new probes in dtrace_lockstat.4, and
deduplicate some of the existing probe descriptions.

Reviewed by:	mjg
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21355
2019-08-21 23:43:58 +00:00
Warner Losh
51c5de5f25 Document RST support in nvme(4) and ahci(4). 2019-08-21 22:18:07 +00:00
John Baldwin
93a0379392 Trim a spurious blank line I added in r348969.
I did not bump .Dd since there is no content change.

MFC after:	3 days
2019-08-19 17:28:12 +00:00
Vincenzo Maffione
421c620837 link ptnet(4) man page to the build system
Reported by:	kevans
Reviewed by:	kevans, bcr
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D21312
2019-08-19 17:26:04 +00:00
Konstantin Belousov
3a91d1062a i386: Implement atomic_load_64(9) and atomic_store_64(9).
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2019-08-18 15:58:44 +00:00
Ed Maste
2fe25d1638 arch.7: indicate expectation that arm (i.e., v5) will be removed before 13.0
MFC after:	1 week
2019-08-16 14:06:04 +00:00
Ed Maste
3a26da0b5c Regen src.conf.5 after r351027 (enable BSD_CRTBEGIN on powerpc) 2019-08-16 13:50:44 +00:00
Alexander Motin
105a032977 Limit memory window usage in ntb_transport to 256MB.
This adds safety net for the case of misconfigured NTB with too big
memory window, for which we may be unable to allocate a memory buffer,
which does not make much sense for the network interface.  While there,
fix the code to really work with asymmetric window sizes setup.

This makes driver just print warning message on boot instead of hanging
if too large memory window is configured.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-08-15 22:55:59 +00:00
Alexander Motin
63ac15aba4 Add NTB modules to i386 build.
There is no reason why NTB should not be usable on i386 if memory windows
are small enough.
2019-08-15 16:27:04 +00:00
Ed Maste
e5c6dece98 do not enable userland retpoline if not supported by compiler/linker
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D21101
2019-08-15 12:48:17 +00:00
Simon J. Gerraty
5c5316098e Revert prior change till installworld sorted 2019-08-15 06:00:55 +00:00
Doug Moore
504f5e294e swap_pager.c reserves 2 blocks for a bsd label. Change that 2 to the
expression howmany(BBSIZE, PAGE_SIZE), where BBSIZE is the size of the
boot block area.  That can be less than 2 if PAGE_SIZE is big.

swapon(8) has an option to trim (delete) all the blocks of a device at
startup.  However, if the first of those blocks is a bsd label, then
trimming those blocks is destructive.  Change swapon to leave the
first BBSIZE bytes untrimmed.

Update manual pages to reflect changes in how swapon and how it may be
used, espeically in association with savecore.

Reviewed by: alc
Approved by: markj (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D21191
2019-08-15 02:30:44 +00:00
Ed Maste
c06ccdd2ba arch.7: be explicit about the future for unmigrated GCC 4.2.1 archs
Reported by:	imp
MFC after:	1 week
MFC with:	r351036
2019-08-15 02:14:44 +00:00
Simon J. Gerraty
f4ff569996 bsd.files.mk: fix targets to avoid directories
Reintroduce :T when file is used as part of a target name.

Reviewed by:	stevek
2019-08-14 22:33:46 +00:00
Ed Maste
4ef280656c arch.7: clarify that the table shows the default tool chain
Reported by:	jhb
MFC after:	1 week
MFC with:	r351035
Sponsored by:	The FreeBSD Foundation
2019-08-14 17:17:52 +00:00
Ed Maste
ff03c71275 arch.7: add deprecation notice for GCC 4.2.1
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-14 17:13:36 +00:00
Ed Maste
4715c31e3e arch.7: add tool chain table
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
2019-08-14 17:04:28 +00:00
Andrew Turner
4ee71ec9eb Enable BSD_CRTBEGIN on powerpc
In r342974 jhibbits added support to build crtsavres.o. This was the
blocker for BSD_CRTBEGIN to be enabled there. As such enable this
option again.

Reviewed by:	jhibbits
Sponsored by:	DARPA, AFRL
2019-08-14 14:31:17 +00:00
Conrad Meyer
eefd8f96fb geom_uzip(4), mkuzip(8): Add Zstd image mode
The Zstd format bumps the CLOOP major number to 4 to avoid incompatibility
with older systems.  Support in geom_uzip(4) is conditional on the ZSTDIO
kernel option, which is enabled in amd64 GENERIC, but not all in-tree
configurations.

mkuzip(8) was modified slightly to always initialize the nblocks + 1'th
offset in the CLOOP file format.  Previously, it was only initialized in the
case where the final compressed block happened to be unaligned w.r.t.
DEV_BSIZE.  The "Fake" last+1 block change in r298619 means that the final
compressed block's 'blen' was never correct unless the compressed uzip image
happened to be BSIZE-aligned.  This happened in about 1 out of every 512
cases.  The zlib and lzma decompressors are probably tolerant of extra trash
following the frame they were told to decode, but Zstd complains that the
input size is incorrect.

Correspondingly, geom_uzip(4) was modified slightly to avoid trashing the
nblocks + 1'th offset when it is known to be initialized to a good value.
This corrects the calculated final real cluster compressed length to match
that printed by mkuzip(8).

mkuzip(8) was refactored somewhat to reduce code duplication and increase
ease of adding other compression formats.

  * Input block size validation was pulled out of individual compression
    init routines into main().

  * Init routines now validate a user-provided compression level or select
    an algorithm-specific default, if none was provided.

  * A new interface for calculating the maximal compressed size of an
    incompressible input block was added for each driver.  The generic code
    uses it to validate against MAXPHYS as well as to allocate compression
    result buffers in the generic code.

  * Algorithm selection is now driven by a table lookup, to increase ease of
    adding other formats in the future.

mkuzip(8) gained the ability to explicitly specify a compression level with
'-C'.  The prior defaults -- 9 for zlib and 6 for lzma -- are maintained.
The new zstd default is 9, to match zlib.

Rather than select lzma or zlib with '-L' or its absense, respectively, a
new argument '-A <algorithm>' is provided to select 'zlib', 'lzma', or
'zstd'.  '-L' is considered deprecated, but will probably never be removed.

All of the new features were documented in mkuzip.8; the page was also
cleaned up slightly.

Relnotes:	yes
2019-08-13 23:32:56 +00:00