Multiple periodic scripts sleep for a random amount of time in order to
mitigate the thundering herd problem. This is bad, because the sum of
multiple uniformly distributed random variables approaches a normal
distribution, so the problem isn't mitigated as effectively as it would be
with a single sleep.
This change creates a single configurable anticongestion sleep. periodic
will only sleep if at least one script requires it, and it will never sleep
more than once per invocation. It also won't sleep if periodic was run
interactively, fixing an unrelated longstanding bug.
PR: 217055
PR: 210188
Reviewed by: cy
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D10211
Changes include modifications in kernel crash dump routines, dumpon(8) and
savecore(8). A new tool called decryptcore(8) was added.
A new DIOCSKERNELDUMP I/O control was added to send a kernel crash dump
configuration in the diocskerneldump_arg structure to the kernel.
The old DIOCSKERNELDUMP I/O control was renamed to DIOCSKERNELDUMP_FREEBSD11 for
backward ABI compatibility.
dumpon(8) generates an one-time random symmetric key and encrypts it using
an RSA public key in capability mode. Currently only AES-256-CBC is supported
but EKCD was designed to implement support for other algorithms in the future.
The public key is chosen using the -k flag. The dumpon rc(8) script can do this
automatically during startup using the dumppubkey rc.conf(5) variable. Once the
keys are calculated dumpon sends them to the kernel via DIOCSKERNELDUMP I/O
control.
When the kernel receives the DIOCSKERNELDUMP I/O control it generates a random
IV and sets up the key schedule for the specified algorithm. Each time the
kernel tries to write a crash dump to the dump device, the IV is replaced by
a SHA-256 hash of the previous value. This is intended to make a possible
differential cryptanalysis harder since it is possible to write multiple crash
dumps without reboot by repeating the following commands:
# sysctl debug.kdb.enter=1
db> call doadump(0)
db> continue
# savecore
A kernel dump key consists of an algorithm identifier, an IV and an encrypted
symmetric key. The kernel dump key size is included in a kernel dump header.
The size is an unsigned 32-bit integer and it is aligned to a block size.
The header structure has 512 bytes to match the block size so it was required to
make a panic string 4 bytes shorter to add a new field to the header structure.
If the kernel dump key size in the header is nonzero it is assumed that the
kernel dump key is placed after the first header on the dump device and the core
dump is encrypted.
Separate functions were implemented to write the kernel dump header and the
kernel dump key as they need to be unencrypted. The dump_write function encrypts
data if the kernel was compiled with the EKCD option. Encrypted kernel textdumps
are not supported due to the way they are constructed which makes it impossible
to use the CBC mode for encryption. It should be also noted that textdumps don't
contain sensitive data by design as a user decides what information should be
dumped.
savecore(8) writes the kernel dump key to a key.# file if its size in the header
is nonzero. # is the number of the current core dump.
decryptcore(8) decrypts the core dump using a private RSA key and the kernel
dump key. This is performed by a child process in capability mode.
If the decryption was not successful the parent process removes a partially
decrypted core dump.
Description on how to encrypt crash dumps was added to the decryptcore(8),
dumpon(8), rc.conf(5) and savecore(8) manual pages.
EKCD was tested on amd64 using bhyve and i386, mipsel and sparc64 using QEMU.
The feature still has to be tested on arm and arm64 as it wasn't possible to run
FreeBSD due to the problems with QEMU emulation and lack of hardware.
Designed by: def, pjd
Reviewed by: cem, oshogbo, pjd
Partial review: delphij, emaste, jhb, kib
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4712
USB in places, as well as having the potential for reducing
performance. Since this is used even when powerd isn't enabled, these
two problems can cause on servers. Supermicro X9 motherboards, for
example, have problems with the virtual IPMI USB keyboards and mice
attaching and detaching repeatedly. Since there are issues on some
CPUs with C2, fail safe by defaulting to not altering it.
MFC After: 3 days
because they can use sysrc in conjunction with ssh and xargs to perform
en-masse changes in a large distribution with lots of jails spread over
many hosts on a LAN/WAN.
Provide a mechanism for disabling the warning eschewed by /etc/rc.d/jail
in said situation. If jail_confwarn="NO" is in rc.conf(5) (default "YES")
skip the warning that per-jail configurations are obsolete and that the
user should migrate to jail.conf(5).
Reviewed by: jelischer
MFC after: 3 days
Sponsored by: FIS Global, Inc.
Differential Revision: https://reviews.freebsd.org/D7465
leapfile will be ignored and ntpd will behave as if it has no
leapfile.
While here, remove an extraneous blank line.
Suggested by: ache
MFC after: 1 week
Add zfsd, which deals with hard drive faults in ZFS pools. It manages
hotspares and replements in drive slots that publish physical paths.
cddl/usr.sbin/zfsd
Add zfsd(8) and its unit tests
cddl/usr.sbin/Makefile
Add zfsd to the build
lib/libdevdctl
A C++ library that helps devd clients process events
lib/Makefile
share/mk/bsd.libnames.mk
share/mk/src.libnames.mk
Add libdevdctl to the build. It's a private library, unusable by
out-of-tree software.
etc/defaults/rc.conf
By default, set zfsd_enable to NO
etc/mtree/BSD.include.dist
Add a directory for libdevdctl's include files
etc/mtree/BSD.tests.dist
Add a directory for zfsd's unit tests
etc/mtree/BSD.var.dist
Add /var/db/zfsd/cases, where zfsd stores case files while it's shut
down.
etc/rc.d/Makefile
etc/rc.d/zfsd
Add zfsd's rc script
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c
Fix the resource.fs.zfs.statechange message. It had a number of
problems:
It was only being emitted on a transition to the HEALTHY state.
That made it impossible for zfsd to take actions based on drives
getting sicker.
It compared the new state to vdev_prevstate, which is the state that
the vdev had the last time it was opened. That doesn't make sense,
because a vdev can change state multiple times without being
reopened.
vdev_set_state contains logic that will change the device's new
state based on various conditions. However, the statechange event
was being posted _before_ that logic took effect. Now it's being
posted after.
Submitted by: gibbs, asomers, mav, allanjude
Reviewed by: mav, delphij
Relnotes: yes
Sponsored by: Spectra Logic Corp, iX Systems
Differential Revision: https://reviews.freebsd.org/D6564
periodic(8) already handles the security_show_{success,info,badconfig}
variables correctly. However, those variables aren't explicitly set in
/etc/defaults/periodic.conf or anywhere else, which suggests to the user
that they shouldn't be used.
etc/defaults/periodic.conf
Explicitly set defaults for security_show_{success,info,badconfig}
usr.sbin/periodic/periodic.sh
Update usage string
usr.sbin/periodic/periodic.8
Minor man page updates
One thing I'm _not_ doing is recommending setting security_output to
/var/log/security.log or adding that file to /etc/newsyslog.conf, because
periodic(8) would create it with default permissions, usually 644, and
that's probably a bad idea.
Reviewed by: brd
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6477
After a discussion on freebsd-fs@ there seemed to be a consensus that
the "-S" option for mountd should become the default.
Since the only known issue w.r.t. using "-S" was fixed by r299201,
this commit adds "-S" to the default mountd_flags.
Discussed on: freebsd-fs
PR: 9619, 131342, 206855
MFC after: 2 weeks
Relnotes: yes
By default set to 'YES' so it does not change the current behaviour for users,
this variable allows to decide to not extract crach dumps from the dump
device at boot time by setting it to "NO" in rc.conf.
Sponsored by: Gandi.net
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.
Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277
is exhausted.
How to use:
Basically we need to add on rc.conf an another option like:
If we want to protect only the main processes.
syslogd_oomprotect="YES"
If we want to protect all future children of the specified processes.
syslogd_oomprotect="ALL"
PR: 204741 (based on)
Submitted by: eugen@grosbein.net
Reviewed by: jhb, allanjude, rpokala and bapt
MFC after: 4 weeks
Relnotes: Yes
Sponsored by: gandi.net
Differential Revision: https://reviews.freebsd.org/D5176
When a user defines "jail_list" in rc.conf the jails are started in the
order defined. Currently the jails are not are stopped in reverse order
which may break dependencies between jails/services and prevent a clean
shutdown. The new parameter "jail_reverse_stop" will shutdown jails in
"jail_list" in reverse order when set to "YES".
Please note that this does not affect manual invocation of the jail rc
script. If a user runs the command
# service jail stop jail1 jail2 jail3
the jails will be stopped in exactly the order specified regardless of
jail_reverse_stop being defined in rc.conf.
PR: 196152
Approved by: jamie
MFC after: 1 week
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D5233
With this change, it's possible to redefine rc_conf_files (e.g.,
sysrc rc_conf_files+=/etc/rc.conf.other) and have the boot process
pick up settings in extra files. The sysrc(8) tool can be used to
query/enumerate/find/manage extra files configured in this manner.
Relnotes: yes
The working copy of leapfile resides in /var/dbntpd.leap-seconds.list.
/etc/ntp/leap-seconds (periodically updated from ftp://time.nist.gov/pub/
or ftp://tycho.usno.navy.mil/pub/ntp/) contains the master copy should
automatic leapfile updates be disabled (default).
Automatic leapfile updates are fetched from $ntp_leapfile_sources,
defaulting to https://www.ietf.org/timezones/data/leap-seconds.list,
within $ntp_leapfile_expiry_days (default 30 days) from leap-seconds
file expiry. Automatic updates can be enabled by setting
$daily_ntpd_leapfile_enable="YES" in periodic.conf. To avoid congesting
the ntp leapfile source the automatic update randomized by default but
can be disabled through daily_ntpd_avoid_congestion="NO" in
periodic.conf.
Suggested by: des
Reviewed by: des, roberto, dwmalone, ian, cperciva, glebius, gjb
MFC after: 1 week
X-MFC with: r289421, r293037
USB NICs.
USB network hardware may not be enumerated and available when the rc.d
networking scripts run. Eventually the USB attachment completes and devd
events cause the network initialization to happen, but by then other rc.d
scripts have already failed, because services which depend on NETWORKING
(such as mountcritremote) may end up running before the network is actually
ready.
There is an existing netwait script, but because it is dependent on
NETWORKING it runs too late to prevent failure of some other rc
scripts. This change flips the order so that NETWORKING depends on netwait,
and netwait now depends on devd and routing (the former is needed to make
interfaces appear, and the latter is needed to run the ping tests in
netwait).
The netwait script used to be oriented primarily towards "as soon as any
host is reachable the network is fully functional", so you gave it a list of
IPs to try and you could optionally name an interface and it would wait for
carrier on that interface. That functionality still works the same, but now
you can provide a list of interfaces to wait for and it waits until each one
of them is available. The ping logic still completes as soon as the first IP
on the list responds.
These changes were submitted by Brenden Molloy <brendan+freebsd@bbqsrc.net>
in PR 205186, and lightly modified by me to allow a list of interfaces
instead of just one.
PR: 205186
Differential Revision: https://reviews.freebsd.org/D4608 (timeout w/o review)
to the rc scripts. With these changes, setting nfs_server_managegids="YES"
in /etc/rc.conf will enable this capability.
Suggested by: jpaetzel
Tested by: jpaetzel
Reviewed by: rc (pending)
MFC after: 2 weeks
Most daily_status_security_* variables in periodic.conf were changed to
security_status_* in SVN r254974. The compatibility code for the old names
did not work.
PR: 204331
Submitted by: martin at lispworks.com
MFC after: 1 week
The command was checking local/remote system uptime, so rename the script to
match its function and to avoid confusion
The controlling variable in /etc/periodic.conf has been renamed from
daily_status_rwho_enable to daily_status_uptime_enable.
MFC after: 3 days
Reported by: Peter Jeremy <peter@rulingia.com>
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
if they are not required for mounting rootfs. However, it's possible
that some setups try to mount them in mountcritlocal (ie from fstab).
Export the list of current root mount holds using a new sysctl,
vfs.root_mount_hold, and make mountcritlocal retry if "mount -a" fails
and the list is not empty.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3709
setups that worked before, flip the default to "YES". Most people don't
have /etc/rctl.conf, so they won't be affected in any way.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
ACPI C3 ends up doing a lot more work before entering sleep, some of which
requires grabbing a global ACPI hardware serialising mutex.
Because of this, the more CPU cores you have, the more that lock contends
under load, reaching close to the #1 lock contention (after VM, which is being
worked on.)
Tested:
* Sandy bridge Xeon, 2 socket * 8 core
* Ivy bridge Xeon v2, 2 socket * 8 core
* Westmere-EX, 4 socket * 10 core
* Ivybridge desktop
* Sandybridge mobile
* Ivybridge mobile
MFC after: 2 weeks
In particular, this allows an administrator to specify "-h" for human
readable output if that is preferred.
The default setting passes "-d", so that can be excluded by using a custom
setting.
Differential Revision: https://reviews.freebsd.org/D2034
Submitted by: Lystopad Aleksandr <laa@laa.zp.ua>
(patch to add option for -h)
Reviewed by: bz
MFC after: 1 week
periodic(8) run, taken from uname(1) '-U' and '-K'
flags.
Reviewed by: allanjude, dvl
Differential Revision: https://reviews.freebsd.org/D1541
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
mrouted has been available in ports for the last 8 years as net/mrouted . An
equivalent rc.d script has been present in the port.
Remove all corresponding variables from etc/defaults/rc.conf
Relnotes: yes
have chosen different (and more traditional) stateless/statuful
NAT64 as translation mechanism. Last non-trivial commits to both
faith(4) and faithd(8) happened more than 12 years ago, so I assume
it is time to drop RFC3142 in FreeBSD.
No objections from: net@
This code has had an extensive rewrite and a good series of reviews, both by the author and other parties. This means a lot of code has been simplified. Pluggable structures for high-rate entropy generators are available, and it is most definitely not the case that /dev/random can be driven by only a hardware souce any more. This has been designed out of the device. Hardware sources are stirred into the CSPRNG (Yarrow, Fortuna) like any other entropy source. Pluggable modules may be written by third parties for additional sources.
The harvesting structures and consequently the locking have been simplified. Entropy harvesting is done in a more general way (the documentation for this will follow). There is some GREAT entropy to be had in the UMA allocator, but it is disabled for now as messing with that is likely to annoy many people.
The venerable (but effective) Yarrow algorithm, which is no longer supported by its authors now has an alternative, Fortuna. For now, Yarrow is retained as the default algorithm, but this may be changed using a kernel option. It is intended to make Fortuna the default algorithm for 11.0. Interested parties are encouraged to read ISBN 978-0-470-47424-2 "Cryptography Engineering" By Ferguson, Schneier and Kohno for Fortuna's gory details. Heck, read it anyway.
Many thanks to Arthur Mesh who did early grunt work, and who got caught in the crossfire rather more than he deserved to.
My thanks also to folks who helped me thresh this out on whiteboards and in the odd "Hallway track", or otherwise.
My Nomex pants are on. Let the feedback commence!
Reviewed by: trasz,des(partial),imp(partial?),rwatson(partial?)
Approved by: so(des)
