Commit Graph

9359 Commits

Author SHA1 Message Date
John Baldwin
16aabb761c Remove support for IPsec algorithms deprecated in r348205 and r360202.
Examples of depecrated algorithms in manual pages and sample configs
are updated where relevant.  I removed the one example of combining
ESP and AH (vs using a cipher and auth in ESP) as RFC 8221 says this
combination is NOT RECOMMENDED.

Specifically, this removes support for the following ciphers:
- des-cbc
- 3des-cbc
- blowfish-cbc
- cast128-cbc
- des-deriv
- des-32iv
- camellia-cbc

This also removes support for the following authentication algorithms:
- hmac-md5
- keyed-md5
- keyed-sha1
- hmac-ripemd160

Reviewed by:	cem, gnn (older verisons)
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24342
2020-05-02 00:06:58 +00:00
Warner Losh
e2d6a6e938 Document the passthru commands.
Reviewed by: bcr@ (earlier version)
Differential Revision: https://reviews.freebsd.org/D24639
2020-05-01 20:29:51 +00:00
Warner Losh
a41b48ea12 Various improvements to this man page:
o Be consistent about device-id and namespace-id
o Use consistent arg markup for these
o document you can use disk names too
o document nsid command better
o document the idenntify command
o add a couple of examples.

Differential Revision: https://reviews.freebsd.org/D24638
2020-05-01 20:29:46 +00:00
Olivier Cochard
b33e1edd26 Skip bectl jail test if jail not installed (WITHOUT_JAIL).
Approved by:	kevans
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24613
2020-04-29 14:15:21 +00:00
Xin LI
85f4f80c42 Do not overflow when calculating file system size.
Reported by:		Hyeongseok Kim <hyeongseok kim lge com>
Reviewed by:		cem, Hyeongseok Kim
MFC after:		3 days
Differential Revision:	https://reviews.freebsd.org/D24603
2020-04-28 05:10:34 +00:00
Xin LI
401475f50c Fix a bug with dirty file system handling.
r356313 broke handling of dirty file system because we have restricted
the correction of "odd" byte sequences to checkfat(), and as a result
the dirty bit is never cleared.  The old fsck_msdosfs code would write
FAT twice to fix the dirty bit, which is also not ideal.

Fix this by introducing a new rountine, cleardirty() which will perform
the set of clean bit only, and use it in checkfilesys() if we thought
the file system was dirty.

Reviewed by:		cem, emaste
MFC after:		3 day
Differential Revision:	https://reviews.freebsd.org/D24581
2020-04-27 02:01:48 +00:00
Conrad Meyer
7575e5c472 dumpon.8: Update minimum key size language
dumpon(8) has not accepted 1024-bit RSA keys since prior to r339784 (2018-10).
The manual page language was not updated at that time (oops).  The minimum
accepted is 2048 bits, which is also a good default choice.

Sponsored by:	Dell EMC Isilon
2020-04-23 18:04:52 +00:00
Conrad Meyer
4647ce4fb6 EKCD: Preload error strings, PRNG seed; use OAEP padding
Preload OpenSSL ERR string data so that the formatted error messages are
vaguely meaningful. Add OpenSSL error information to the RSA_public_encrypt()
operation failure case in one-time key generation.

For obsolescent OpenSSL versions (*cough* FIPS *cough*), pre-seed the PRNG
before entering Cap mode, as old versions of OpenSSL are unaware of kernel
RNG interfaces aside from /dev/random (such as the long-supported kern.arnd, or
the slightly more recent getentropy(3) or getrandom(2)). (RSA_public_encrypt()
wants a seeded PRNG to randomize the "PS" portion of PKCS 1.5 padding or the
"MGF" pseudo-random function in OAEP padding.)

Switch dumpon to encrypt the one-time key with OAEP padding (recommended since
1998; RFC2437) rather than the obsolescent PKCS 1.5 padding (1993; RFC2313).

Switch decryptcore to attempt OAEP decryption first, and try PKCS 1.5
decryption on failure. This is intended only for transition convenience, and
we should obsolete support for non-OAEP padding in a release or two.

Reviewed by:	markj
MFC After:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D24534
2020-04-23 17:56:48 +00:00
Alexander Motin
5458a1c840 Allow namespace-id specification where it makes sense.
It makes tool more convenient to not require user to explicitly convert
namespace device name into controller device name.  There should be no
changes to already existing syntaxes.

MFC after:	1 week
2020-04-20 14:54:41 +00:00
Alexander Motin
1f15d49eea Open device with O_RDONLY when command is non-invasive.
This allows to use some of the subcommands against mounted nvd devices.

MFC after:	1 week
Sponsored by:	iXystems, Inc.
2020-04-20 13:47:07 +00:00
Kristof Provost
8fd675d8a0 pfctl: Call ifa_load() before ifa_grouplookup()
ifa_grouplookup() uses the data loaded in ifa_load() (through is_a_group()), so
we must call ifa_load() before we can rely on any of the data it populates.

Submitted by:	Nick Rogers
MFC after:	1 week
Sponsored by:	RG Nets
2020-04-19 15:37:13 +00:00
Kristof Provost
681652667a pfctl: Remove unused variable
Submitted by:	Nick Rogers
MFC after:	1 week
Sponsored by:	RG Nets
2020-04-19 15:32:14 +00:00
Alan Somers
cb67aaf20c bectl.8: correctly sort SEE ALSO section after 360078
Reported by:	yuripv
MFC after:	2 weeks
MFC with:	360078
2020-04-19 01:43:59 +00:00
Alan Somers
27586155ac bectl.8, beinstall.sh.8: fix man page section of beinstall.sh
Reported by:	phk
MFC after:	2 weeks
2020-04-18 19:53:47 +00:00
Mateusz Piotrowski
789e4d2f20 Improve manual page formatting
-	Use appropriate macros for command arguments.
-	Increase option list indentation for better readability.

MFC after:	3 days
2020-04-14 10:10:31 +00:00
Li-Wen Hsu
862ac3b74a Ensure kyua list working when there is no /dev/nvme*
Sponsored by:	The FreeBSD Foundation
2020-04-13 21:08:54 +00:00
Mariusz Zaborski
eec362ef58 decryptcore: load the nls data
Load the nls data before the openssl will try to do it in the
capability mode.
On my machine the sa_ossl_private_decrypt is trying to do that.

MFC after:	2 weeks
2020-04-11 17:30:33 +00:00
Kirk McKusick
0c08ecdff3 Inode check-hash errors were being reported after system crashes.
Trace the cause down to journalled soft updates recovery code in
fsck failing to recompute the check-hash after updating an inode.

As inode check-hash was first introduced to UFS in FreeBSD 13,
there is no need to MFC this commit.

Reported by:  Chuck Silvers
Sponsored by: Netflix
2020-04-10 23:58:07 +00:00
Kirk McKusick
2a18059670 Add an inode check-hash verification when running the journalled
soft update recovery code with the debugging (-d) option.

As inode check-hash was first introduced to UFS in FreeBSD 13,
there is no need to MFC this commit.

Reported by:  Chuck Silvers
Sponsored by: Netflix
2020-04-10 23:49:34 +00:00
David Bright
0e3e53d746 Add a basic test for nvmecontrol
I recently made some bug fixes in nvmecontrol. It occurred to me that
since nvmecontrol lacks any kyua tests, I should convert the informal
testing I did into a more formal automated test. The test in this
change should be considered just a starting point; it is neither
complete nor thorough. While converting the test to ATF/kyua, I
discovered a small bug in nvmecontrol; the nvmecontrol devlist command
would always exit with an unsuccessful status. So I included the fix
for that, too, so that the test won't fail.

Reviewed by:	imp@
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D24269
2020-04-07 20:26:42 +00:00
Kirk McKusick
33ceb489d3 Clean up global variable declarations in the dump and restore
utilities so that they will compile with -fno-common.

Started by:  Kyle Evans (kevans)
Reviewed by: Kyle Evans (kevans)
MFC after:   1 week
Differential Revision: https://reviews.freebsd.org/D24210
2020-04-04 00:56:56 +00:00
Mateusz Piotrowski
b75ab02f57 Bump date after recent changes and lint 2020-04-03 13:05:26 +00:00
Poul-Henning Kamp
9546e08e7b Add two options to recoverdisk(1) inspired by a recent
data-archæology project:

-u pattern

    Fill unread parts of the output with pattern.
    Default pattern is "_UNREAD_", empty pattern disables.

-v

    Fancy status reporting using ANSI escapes and UTF-8
2020-04-02 15:17:53 +00:00
David Bright
6995fb5eb7 Fix various Coverity-detected errors in nvmecontrol
This fixes several Coverity-detected errors in nvmecontrol. While in
here, a couple additional errors with shift/mask confusion that were
not diagnosed by Coverity are also fixed.

CIDs addressed: 1040299, 1040300, 1403972, 1403973, 1403985, 1403988,
1403990, 1404374, 1404427, 1404469, 1404510, 1404534, 1418118

CID 1403657 (resource leak of shared library handle) was marked
"intentional" in the Coverity scan database.

Reviewed by:	vangyzen, robert.herndon_dell.com
Reviewed by:	daniel.william.ryan_gmail.com (earlier version)
Reviewed by:	rramsden_isilon.com (earlier version), imp
MFC after:	5 days
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D24203
2020-04-02 13:52:54 +00:00
Kyle Evans
c3e9752ea1 fsck_ffs/fsdb: fix -fno-common build
This one is also a small list:

- 3x duplicate definition (ufs2_zino, returntosingle, nflag)
- 5x 'needs extern', 3/5 of which are referenced in fsdb

-fno-common will become the default in GCC10/LLVM11.

MFC after:	1 week
2020-03-29 20:03:46 +00:00
Kyle Evans
b4cd5daf75 iscontrol: move definition of vflag/iscsidev to iscontrol.c
Mark the declaration extern as these are used elsewhere; this fixes the
build with -fno-common.

MFC after:	3 days
2020-03-29 02:28:15 +00:00
Andrey V. Elsukov
db1102f213 Use IP_FW_NAT44_DESTROY opcode for IP_FW3 socket option to destroy
NAT instance.

The NAT44 group of opcodes for IP_FW3 socket option is modern way
to control NAT instances and this method can be used in future to
switch from numeric to named NAT instances, like was done for ipfw
tables.
The IP_FW_NAT_DEL opcode is the last remnant of old ipfw_ctl control
plane that doesn't support versioned operations. This interface will
be retired soon.

Reviewed by:	melifaro
MFC after:	10 days
Sponsored by:	Yandex LLC
2020-03-24 12:27:02 +00:00
Baptiste Daroussin
1b6c9c3f43 Remove the link to libl which only contains a stub function
on yywrap, if the flex is told yywrap is not in use, then this linkage becomes
unnecessary
2020-03-23 14:37:00 +00:00
Baptiste Daroussin
e2328be509 Revove useless linking to yacc 2020-03-23 14:11:49 +00:00
Kristof Provost
d2568b024d pfctl: improve rule load times with thousands of interfaces
r343287 / D18759 introduced ifa_add_groups_to_map() which is now run by
ifa_load/ifa_lookup/host_if. When loading an anchor or ruleset via pfctl that
does NOT contain ifnames as hosts, host() still ends up iterating all
interfaces twice, grabbing SIOCGIFGROUP ioctl twice for each. This adds an
unnecessary amount of time on systems with thousands or tens of thousands of
interfaces.

Prioritize the IPv4/6 check over the interface name lookup, which skips loading
the iftab and iterating all interfaces when the configuration does not contain
interface names.

Submitted by:	Nick Rogers
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D24100
2020-03-19 12:54:43 +00:00
Ed Maste
2ee11faacf ldconfig: add aout deprecation notice
aout support in ldconfig hasn't been required since FreeBSD 2.x.
Anyone still using FreeBSD 2 shared libraries can also use a FreeBSD 2
ldconfig to generate aout ldconfig hints.

Sponsored by:	The FreeBSD Foundation
2020-03-17 15:57:25 +00:00
Alexander V. Chernikov
e3bf606c5d Revert r358858 as it breaks some ipfw(8) setups.
Reported by:	O. Hartmann <o.hartmann@walstatt.org>
2020-03-11 08:08:53 +00:00
Alexander V. Chernikov
8149b12da1 Don't assume !IPv6 is IPv4 in ipfw(8) add_src() and add_dst().
Submitted by:	Neel Chauhan <neel AT neelc DOT org>
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D21812
2020-03-10 20:30:21 +00:00
Gleb Smirnoff
3f16af55b2 Align the buffer to the alignment of the structure we expect.
Submitted by:	Slawa Olhovchenkov <slw zxy.spb.ru>
2020-03-05 21:01:47 +00:00
Gleb Smirnoff
8cd9718f0b When a machine boots the NFS mounting script is executed after
interfaces are configured, but for many interfaces (e.g. all Intel)
ifconfig causes link renegotiation, so the first attempt to mount
NFS always fails. After that mount_nfs sleeps for 30 seconds, while
only a couple seconds are actually required for interface to get up.

Instead of sleeping, do select(2) on routing socket and check if
some interface became UP and in this case retry immediately.

Reviewed by:	rmacklem
Differential Revision:	https://reviews.freebsd.org/D23934
2020-03-04 22:27:16 +00:00
Eric van Gyzen
7bef70737b dumpon: skip size check if using zstd
As with gzip, let the dump device be smaller than physical memory
when using zstd and full dumps.

Also print the error message if the size check fails, even if -v
is not specified.  Failing silently is not friendly.

Reviewed by:	cem markj
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D23923
2020-03-03 22:14:23 +00:00
Li-Wen Hsu
fe44c67716 Fix test cases after r358448
sbin.pfctl.pfctl_test.pf0087
sbin.pfctl.pfctl_test.selfpf0087

MFC with:	r358448
Sponsored by:	The FreeBSD Foundation
2020-02-29 13:24:41 +00:00
Alexander V. Chernikov
af14123601 Make ping6(1) return code consistent with the man page.
When every sendto() call originated by ping6(1) fails, current code always
 returns 2 ("transmission was successful but no responses were received")
 which is incorrect. Return EX_OSERR instead as in many cases it indicates
 some kernel-level problems.

MFC after:	3 weeks
2020-02-15 15:39:53 +00:00
Dimitry Andric
dacf624d5f Fix the following -Werror warning from clang 10.0.0 in sconfig:
sbin/sconfig/sconfig.c:909:5: error: misleading indentation; statement is not part of the previous 'else' [-Werror,-Wmisleading-indentation]
                                exit (-1);
                                ^
sbin/sconfig/sconfig.c:907:6: note: previous statement is here
                        } else
                          ^

The intent was to group the exit() call with the previous fprintf()
call.

MFC after:	3 days
2020-02-13 18:37:17 +00:00
Eugene Grosbein
04777538e7 ipfw nat: add missing bits after r357092 (RFC 6598/Carrier Grade NAT)
Submitted by:	Neel Chauhan <neel AT neelc DOT org>
Reviewed by:	Lutz Donnerhacke
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D23448
2020-02-12 00:46:33 +00:00
Xin LI
9e4029ff49 Use humanize_number to format available and bad space sizes.
Reviewed by:		mckusick (earlier version)
MFC after:		2 weeks
Differential Revision:	https://reviews.freebsd.org/D23050
2020-02-10 04:16:41 +00:00
Warner Losh
ab9848aa9b No need to make sunlabel anymore
It was only built on sparc64. Since it wasn't a general tool on other
architectures, no need to keep it around for another release.

Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D23524
2020-02-06 17:52:02 +00:00
Kyle Evans
21c1a93c04 init(8): set environment variables from the "daemon" class as well
Specifically, when running /etc/rc. This allows one to specify via
login.conf(5) an environment that should be used when running services to
ease, e.g., setting up env vars for an HTTP proxy consistently across cron
and services alike.

Future changes will extend cron(8)/service(8) to use environment vars
pecified in login.conf(5) as well to promote a more cohesive experience.

This is a part of D21481.

Submitted by:	Andrew Gierth <andrew_tao173.riddles.org.uk>
2020-02-05 04:27:44 +00:00
Xin LI
43f1940992 Diff reduction against NetBSD, no functional change.
MFC after:	1 week
2020-02-02 20:53:31 +00:00
Dimitry Andric
e54f2950a6 Fix the following -Werror warning from clang 10.0.0 in newfs_msdos:
sbin/newfs_msdos/newfs_msdos.c:181:2: error: misleading indentation; statement is not part of the previous 'if' [-Werror,-Wmisleading-indentation]
        if (o.align) {
        ^
sbin/newfs_msdos/newfs_msdos.c:179:5: note: previous statement is here
    if (argc < 1 || argc > 2)
    ^

MFC after:	3 days
2020-01-28 20:10:26 +00:00
Alex Richardson
162ae9c834 Allow bootstrapping makefs on older FreeBSD hosts and Linux/macOS
In order to do so we need to install the msdosfs headers to the bootstrap
sysroot and avoid includes of kernel headers that may not exist on every
host (e.g. sys/lockmgr.h). This change should allow bootstrapping of makefs
on FreeBSD 11+ as well as Linux and macOS.

We also have to avoid using the IO_SYNC macro since that may not be
available. In makefs it is only used to switch between calling
bwrite() and bdwrite() which both call the same function. Therefore we
can simply always call bwrite().

For our CheriBSD builds we always bootstrap makefs by setting
LOCAL_XTOOL_DIRS='lib/libnetbsd usr.sbin/makefs' and use the makefs binary
from the build tree to create a bootable disk image.

Reviewed By:	brooks
Differential Revision: https://reviews.freebsd.org/D23201
2020-01-27 12:02:41 +00:00
Alexander V. Chernikov
75b893375f Add support for RFC 6598/Carrier Grade NAT subnets. to libalias and ipfw.
In libalias, a new flag PKT_ALIAS_UNREGISTERED_RFC6598 is added.
 This is like PKT_ALIAS_UNREGISTERED_ONLY, but also is RFC 6598 aware.
Also, we add a new NAT option to ipfw called unreg_cgn, which is like
 unreg_only, but also is RFC 6598-aware.  The reason for the new
 flags/options is to avoid breaking existing networks, especially those
 which rely on RFC 6598 as an external address.

Submitted by:	Neel Chauhan <neel AT neelc DOT org>
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22877
2020-01-24 20:35:41 +00:00
Kyle Evans
6966ac055c Drop "All Rights Reserved" from all libbe/bectl files
I sent out an e-mail on 2020/01/21 with a plan to do this to Kyle, Rob, and
Wes; all parties have responded in the affirmative that it's OK to drop it
from these files.
2020-01-24 02:18:09 +00:00
Warner Losh
73c6e0c358 Small tweak to the default behavior of shutdown -c
'shutdown -c' is supposed to power cycle the system rather than doing a normal
reboot. However, when that fails, it halts the system. This is not quite right
since the intent isn't to halt the system but to restart. Make the default init
behavior be to restart the system. The halt(8) interface can be used if you'd
like to powercycle or halt.

MFC After: 1 week
Differential Revision: https://reviews.freebsd.org/D23129
2020-01-17 01:20:48 +00:00
Eric van Gyzen
a532f299c9 savecore: include time zone in info.N file
This helps with event correlation when machines are distributed
across multiple time zones.

Format the time with relaxed ISO 8601 for all the usual reasons.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2020-01-13 22:01:37 +00:00
Xin LI
d14a599d69 Tighten FAT checks and fix off-by-one error in corner case.
sbin/fsck_msdosfs/fat.c:
 - readfat:
    * Only truncate out-of-range cluster pointers (1, or greater than
      NumClusters but smaller than CLUST_RSRVD), as the current cluster
      may contain some data. We can't fix reserved cluster pointers at
      this pass, because we do no know the potential cluster preceding
      it.
    * Accept valid cluster for head bitmap. This is a no-op, and mainly
      to improve code readability, because the 1 is already handled in
      the previous else if block.
 - truncate_at: absorbed into checkchain.
 - checkchain: save the previous node we have traversed in case that we
   have a chain that ends with a special (>= CLUST_RSRVD) cluster, or is
   free. In these cases, we need to truncate at the cluster preceding the
   current cluster, as the current cluster contains a marker instead of
   a next pointer and can not be changed to CLUST_EOF (the else case can
   happen if the user answered "no" at some point in readfat()).
 - clearchain: correct the iterator for next cluster so that we don't
   stop after clearing the first cluster.
 - checklost: If checkchain() thinks the chain have no cluster, it
   doesn't make sense to reconnect it, so don't bother asking.

Reviewed by:	kevlo
MFC after:	24 days
X-MFC-With:	r356313
Differential Revision:	https://reviews.freebsd.org/D23065
2020-01-12 06:13:52 +00:00
Xin LI
d3dd66792b Correct off-by-two issue when determining FAT type.
In the code we used NumClusters as the upper (non-inclusive) boundary
of valid cluster number, so the actual value was 2 (CLUST_FIRST) more
than the real number of clusters. This causes a FAT16 media with
65524 clusters be treated as FAT32 and might affect FAT12 media with
4084 clusters as well.

To fix this, we increment NumClusters by CLUST_FIRST after the type
determination.

PR:		243179
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D23082
2020-01-11 17:41:20 +00:00
Xin LI
727d995c7d Apply typo fix from NetBSD, we have already applied all NetBSD changes so
update the NetBSD tag while I'm there.

MFC after:	2 weeks
2020-01-11 04:02:40 +00:00
Xin LI
ed0879d944 Require FAT to occupy at least one sector.
Obtained from:	Android https://r.android.com/1205830
MFC after:	3 days
2020-01-11 03:59:06 +00:00
Mark Johnston
c23df8eafa lagg: Further cleanup of the rr_limit option.
Add an option flag so that arbitrary updates to a lagg's configuration
do not clear sc_stride.  Preseve compatibility for old ifconfig
binaries.  Update ifconfig to use the new flag and improve the casting
used when parsing the option parameter.

Modify the RR transmit function to avoid locklessly reading sc_stride
twice.  Ensure that sc_stride is always 1 or greater.

Reviewed by:	hselasky
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D23092
2020-01-09 14:58:41 +00:00
Xin LI
9a67c570a9 fsck_msdosfs.8: document -M.
Reported by:	mckusick
Reviewed by:	mckusick, emaste, bcr
MFC after:	28 days
X-MFC-with:	r356313
Differential Revision:	https://reviews.freebsd.org/D23049
2020-01-07 04:33:14 +00:00
Alexander Motin
83018b7987 Fix host memory buffer sizes reporting.
Hardware reports values in 4KB units, not in bytes.

MFC after:	3 days
2020-01-06 01:51:23 +00:00
Xin LI
9708ba9f29 Reduce memory footprint of fsck_msdosfs.
This is a re-apply r356249 with changes to make GCC happy.

This utility was initially written for FAT12/16, which were inherently
small. When FAT32 support was added, the old data structure and
algorithms remain used with minimal changes.

With growing size of FAT32 media, the current data structure that
requires 4 32-bit variables per each FAT32 table entry would consume up
to 4 GiB of RAM, which can be too big for systems with limited RAM
available.

Address this by taking a different approach of validating the FAT.

The FAT is essentially a set of linked lists of chains that was
referenced by directory entries, and the checker needs to make sure that
the linked chains of clusters do not have cross-linked chains, and every
chain were referenced by one and only one directory entry.  Instead of
keeping track of the chain's 'head' cluster number, the size of the
chain, the used status of the chain and the "next" pointer which is
content of the FAT table, we create accessors for the FAT table data
for the "next" pointer, and keep only one bit to indicate if the
current cluster is a 'head' node of a cluster chain, in a bitmap.

We further overhaul the FAT checker to find out the possible head nodes
by excluding ones that are not (in other words, nodes that have some
other nodes claiming them as the next node) instead of marking the head
nodes for each node on the chain.  This approach greatly reduced the
complexiety of computation from O(N^2) worst case, to an O(N) scan for
worst case.  The file (cluster chain) length is not useful for the FAT
checker, so don't bother to calculate them in the FAT checker and
instead leave the task to the directory structure check, at which point
we would have non-crossed cluster chains, and we are guaranteed that
each cluster will be visited for at most one time.

When checking the directory structures, we use the head node indicator
to as the visited (used) flag: every cluster chain can only be
referenced by one directory entry, so we clear them when calculating
the length of the chain, and we can immediately tell if there are
anomalies in the directory entry.

As a result, the required RAM size is now 1 bit per each entry of
the FAT table, plus memory needed to hold the FAT table in memory,
instead of 16 bytes (=128 bits) per each entry.  For FAT12 and FAT16,
we will load the whole FAT table into memory as they are smaller than
128KiB, and for FAT32, we first attempt to mmap() it into memory, and
when that fails, we would fall back to a simple LRU cache of 4 MiB of
RAM.

sbin/fsck_msdosfs/boot.c:

 - Added additional sanity checks for valid FAT32/FAT16/FAT12 cluster
   number.
 - FAT32: check if root directory starts with a valid cluster number,
   moved from dir.c.  There is no point to proceed if the filesystem
   is already damaged beyond repair.

sbin/fsck_msdosfs/check.c:

 - Combine phase 1 and phase 2, now that the readfat() is able to
   detect cross chains.

sbin/fsck_msdosfs/dir.c:

 - Refactor code to use FAT accessor instead of accessing the internal
   representation of FAT table.
 - Make use of the cluster chain head bitmap.
 - Clarify and simplify directory entry check, remove unnecessary
   checks that are would be done at a later time (for example, whether
   the directory's second cluster is a valid one, which is examined
   more throughly in a later checkchain() and does not prevent us
   from proceeding further).

sbin/fsck_msdosfs/dosfs.h:

 - Remove internal representation of FAT table, which is replaced by
   the head bitmap that is opaque to other code.
 - Added a special CLUST_DEAD cluster type to indicate errors.

sbin/fsck_msdosfs/ext.h:

 - Added a flag that overrides mmap(2) setting.  The corresponding
   command line option, -M is intentionally undocumented as we do not
   expect users to need it.
 - Added accessors for FAT table and convert existing interface to use
   it.

sbin/fsck_msdosfs/fat.c:

 - Added head bitmap to represent whether a cluster is a head cluster.
 - Converted FAT internal representation to accessors.
 - Implemented a LRU cache for FAT32 when mmap(2) should not or can not
   be used.
 - _readfat: Attempt a mmap(2) and fall back to regular read for
   non-FAT32 file systems; use the LRU cache for FAT32 and prepopulate
   the cache with the first 4MiB of the entries.
 - readfat: Added support of head bitmap and use the population scan to
   detect bogus chains.
 - clusterdiff: removed, FATs are copied from the checked copy via
   writefat()/copyfat().
 - checkchain: calculates the length of a cluster chain and make sure
   that it ends with a valid EOF marker.
 - clearchain: follow and clear a chain and maintain the free cluster
   count.
 - checklost: convert to use head bitmap. At the end of all other scans,
   the remaining 'head' nodes are leaders of lost cluster chains.

sbin/fsck_msdosfs/fat.c:

 - Added a new -M option which is intentionally undocumented, to disable
   the use of mmap().

Reviewed by:	kevlo
MFC after:	1 month
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D22965
2020-01-03 00:31:48 +00:00
Kyle Evans
8f5c6c31ae libbe(3): promote dependent clones when destroying an environment
When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.

This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.

PR:		242592
Submitted by:	Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D22953
2020-01-02 18:46:33 +00:00
Xin LI
73db93b889 Revert r356249 for now as it broke GCC builds. 2020-01-01 09:22:06 +00:00
Xin LI
b06cf1e44f Reduce memory footprint of fsck_msdosfs.
This utility was initially written for FAT12/16, which were inherently
small. When FAT32 support was added, the old data structure and
algorithms remain used with minimal changes.

With growing size of FAT32 media, the current data structure that
requires 4 32-bit variables per each FAT32 table entry would consume up
to 4 GiB of RAM, which can be too big for systems with limited RAM
available.

Address this by taking a different approach of validating the FAT.

The FAT is essentially a set of linked lists of chains that was
referenced by directory entries, and the checker needs to make sure that
the linked chains of clusters do not have cross-linked chains, and every
chain were referenced by one and only one directory entry.  Instead of
keeping track of the chain's 'head' cluster number, the size of the
chain, the used status of the chain and the "next" pointer which is
content of the FAT table, we create accessors for the FAT table data
for the "next" pointer, and keep only one bit to indicate if the
current cluster is a 'head' node of a cluster chain, in a bitmap.

We further overhaul the FAT checker to find out the possible head nodes
by excluding ones that are not (in other words, nodes that have some
other nodes claiming them as the next node) instead of marking the head
nodes for each node on the chain.  This approach greatly reduced the
complexiety of computation from O(N^2) worst case, to an O(N) scan for
worst case.  The file (cluster chain) length is not useful for the FAT
checker, so don't bother to calculate them in the FAT checker and
instead leave the task to the directory structure check, at which point
we would have non-crossed cluster chains, and we are guaranteed that
each cluster will be visited for at most one time.

When checking the directory structures, we use the head node indicator
to as the visited (used) flag: every cluster chain can only be
referenced by one directory entry, so we clear them when calculating
the length of the chain, and we can immediately tell if there are
anomalies in the directory entry.

As a result, the required RAM size is now 1 bit per each entry of
the FAT table, plus memory needed to hold the FAT table in memory,
instead of 16 bytes (=128 bits) per each entry.  For FAT12 and FAT16,
we will load the whole FAT table into memory as they are smaller than
128KiB, and for FAT32, we first attempt to mmap() it into memory, and
when that fails, we would fall back to a simple LRU cache of 4 MiB of
RAM.

sbin/fsck_msdosfs/boot.c:

 - Added additional sanity checks for valid FAT32/FAT16/FAT12 cluster
   number.
 - FAT32: check if root directory starts with a valid cluster number,
   moved from dir.c.  There is no point to proceed if the filesystem
   is already damaged beyond repair.

sbin/fsck_msdosfs/check.c:

 - Combine phase 1 and phase 2, now that the readfat() is able to
   detect cross chains.

sbin/fsck_msdosfs/dir.c:

 - Refactor code to use FAT accessor instead of accessing the internal
   representation of FAT table.
 - Make use of the cluster chain head bitmap.
 - Clarify and simplify directory entry check, remove unnecessary
   checks that are would be done at a later time (for example, whether
   the directory's second cluster is a valid one, which is examined
   more throughly in a later checkchain() and does not prevent us
   from proceeding further).

sbin/fsck_msdosfs/dosfs.h:

 - Remove internal representation of FAT table, which is replaced by
   the head bitmap that is opaque to other code.
 - Added a special CLUST_DEAD cluster type to indicate errors.

sbin/fsck_msdosfs/ext.h:

 - Added a flag that overrides mmap(2) setting.  The corresponding
   command line option, -M is intentionally undocumented as we do not
   expect users to need it.
 - Added accessors for FAT table and convert existing interface to use
   it.

sbin/fsck_msdosfs/fat.c:

 - Added head bitmap to represent whether a cluster is a head cluster.
 - Converted FAT internal representation to accessors.
 - Implemented a LRU cache for FAT32 when mmap(2) should not or can not
   be used.
 - _readfat: Attempt a mmap(2) and fall back to regular read for
   non-FAT32 file systems; use the LRU cache for FAT32 and prepopulate
   the cache with the first 4MiB of the entries.
 - readfat: Added support of head bitmap and use the population scan to
   detect bogus chains.
 - clusterdiff: removed, FATs are copied from the checked copy via
   writefat()/copyfat().
 - checkchain: calculates the length of a cluster chain and make sure
   that it ends with a valid EOF marker.
 - clearchain: follow and clear a chain and maintain the free cluster
   count.
 - checklost: convert to use head bitmap. At the end of all other scans,
   the remaining 'head' nodes are leaders of lost cluster chains.

sbin/fsck_msdosfs/fat.c:

 - Added a new -M option which is intentionally undocumented, to disable
   the use of mmap().

Reviewed by:	kevlo
MFC after:	1 month
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D22965
2020-01-01 07:43:08 +00:00
Rick Macklem
56c049a003 Fix mount_nfs to recognize the NFSv4 specific errors returned by nmount(2).
When mount_nfs calls nmount(2), certain NFSv4 specific errors such as
NFSERR_MINORVERMISMATCH can be returned.
Without this patch, 10021 is reported as an unknown error.
This is not particulcarily serious, but make it difficult for sysadmins
to figure out why the mount attempt is failing.
This patch uses nfsv4_errstr.h to convert 10021 and similar to error strings
that can be printed out.
A positive side effect of this patch is the removal of a reference to
sys/nfsclient/nfs.h, which should no longer be used, since it is
part of the old NFS client.

This patch should only affect reporting of failed mount attempts and not the
semantics of NFS mount attempts.
2019-12-26 22:33:20 +00:00
Kirk McKusick
c094263a24 When running fsck_ffs manually, do not ask:
USE JOURNAL? [yn]

when the journal timestamp does not match the filesystem mount time
as we are just going to print an error and fall through to a full fsck.
Instead, just run a full fsck.

Requested by: Bjoern A. Zeeb (bz)
MFC after:    7 days
2019-12-24 23:03:12 +00:00
Mark Johnston
c104c2990d lagg: Clean up handling of the rr_limit option.
- Don't allow an unprivileged user to set the stride. [1]
- Only set the stride under the softc lock.
- Rename the internal fields to accurately reflect their use.  Keep
  ro_bkt to avoid changing the user API.
- Simplify the implementation.  The port index is just sc_seq / stride.
- Document rr_limit in ifconfig.8.

Reported by:	Ilja Van Sprundel <ivansprundel@ioactive.com> [1]
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D22857
2019-12-22 21:56:47 +00:00
Xin LI
e87e283ce3 Remove unused includes.
MFC after:	2 weeks
2019-12-22 05:44:29 +00:00
Kirk McKusick
b947472d42 Fix typo in hastd.8 manual page.
Reported by: Steve Kargl <sgk at troutmask dot apl dot washington dot edu>
MFC after:   3 days
2019-12-22 01:22:51 +00:00
Ryan Libby
36947e1f4d Declare packed struct ata_params as 2-byte-aligned
This avoids gcc9 warning about unaligned access to the structure when
casting to uint16_t pointer type.

Submitted by:	imp
Reviewed by:	imp
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D22888
2019-12-21 02:44:00 +00:00
Alexander Motin
3ab798a100 Add missing "ereport." prefixes of ZFS events.
I was unable to find time when those were working.  I think they have been
broken for at least 5 years or even longer.

Discussed with:	avg@
MFC after:	1 month
2019-12-18 18:43:44 +00:00
Warner Losh
6ee8575cd7 Remove all the RELEASE_CRUNCH defines, they are useless.
RELEASE_CRUNCH has been broken for a very long time. Remove the
last remants from the tree.
2019-12-16 21:06:24 +00:00
Alexander Motin
5233eb9dc6 Properly detect ATA sanitize errors.
It seems I read specifications not careful enough.  There are devices not
setting successful completion bit, causing previous code report false error.

MFC after:	1 week
2019-12-15 23:28:53 +00:00
Rick Macklem
547a4dba9f Update the mount_nfs.8 man page to include NFSv4.2.
r355677 added NFSv4.2 support to the NFS client. This patch updates the
mount_nfs.8 man page to reflect that.
It also clarifies that the "nolockd" option does not apply to NFSv4 mounts.

This is a content change.
2019-12-14 21:49:47 +00:00
Kristof Provost
3c7fbb06a0 pfctl: Warn users when they run into kernel limits
Warn users when they try to add/delete/modify more items than the kernel will
allow.

Reviewed by:	allanjude (previous version), Lutz Donnerhacke
Differential Revision:	https://reviews.freebsd.org/D22733
2019-12-14 02:03:47 +00:00
Alexander Motin
de57976691 Fix $() handling, broken since the beginning at r108014.
Due to off-by-one error in brackets counting it consumed the rest of the
string, preventing later variables expansions.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-12-13 17:52:09 +00:00
Conrad Meyer
d82e4d759d fsirand(8): Just use arc4random(3)
Remove single use of dubious srandomdev(3) + random(3) and replace with
arc4random(3), as is used already in this program.

Follow-up question: Do we really need this program anymore?  In base?
2019-12-13 04:12:13 +00:00
Simon J. Gerraty
2c9a9dfc18 Update Makefile.depend files
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22494
2019-12-11 17:37:53 +00:00
Simon J. Gerraty
5ab1c5846f Add Makefile.depend.options
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend

DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options

See share/mk/dirdeps-options.mk

Reviewed by:	 bdrewery
MFC after:	1 week
Sponsored by:   Juniper Networks
Differential Revision:  https://reviews.freebsd.org/D22469
2019-12-11 17:37:37 +00:00
Eric van Gyzen
15da40b0af fsck_ffs: fix some memory leaks found by Coverity.
Reported by:	Coverity
CID:		1380549 1380550 1380551
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2019-12-10 20:04:08 +00:00
Don Lewis
1bf6738f80 Fix a logic bug in error handling code. It is an error if p == NULL.
The linelen tests are only meaningful when p != NULL.

Reported by:	Coverity
Coverity CID:	1368655
MFC after:	1 month
2019-12-09 07:18:40 +00:00
Xin LI
2780a26b6a Fix a couple of minor issues with newfs_msdos:
- Do not unnecessarily strdup().
 - Check return value of getdiskinfo(), if it failed, bail out.

Reviewed by:	imp
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D22729
2019-12-08 01:20:37 +00:00
Jens Schweikhardt
e7114e1e11 Correct a handful of typos/grammos. 2019-12-07 15:17:00 +00:00
Warner Losh
f86e60008b Regularize my copyright notice
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
  All Rights Reserved on same line as other copyright holders (but not
  me). Other such holders are also listed last where it's clear.
2019-12-04 16:56:11 +00:00
Xin LI
7b9934a1c6 Explicitly exit() instead of return in main().
MFC after:	2 weeks
2019-12-03 07:03:25 +00:00
Xin LI
e453f01668 newfs_msdos: -A is incompatible with -r, not -o.
PR:		242314
Submitted by:	Guy Yur <guyyur gmail com>
MFC after:	2 weeks
2019-12-03 07:01:28 +00:00
Benedict Reuschling
23614c2b39 Capitalize some user-visible output messages in
the bectl utility.

No functional changes.

Approved by:	    imp@
MFC after:	    7 days
Differential Revision:	https://reviews.freebsd.org/D22330
2019-11-30 14:17:45 +00:00
Xin LI
f00c55e25a Use strlcat().
MFC after:	2 weeks
2019-11-30 05:57:54 +00:00
Conrad Meyer
74aed808a1 Fix braino in previous bugfix r300174
The previous revision missed the exact same error in a copy paste block
of the same code in another function.  Fix the identical case, too.

A DHCP client identifier is simply the hardware type (one byte)
concatenated with the hardware address (some variable number of bytes,
but at most 16).  Limit the size of the temporary buffer to match and
the rest of the calculations shake out correctly.

PR:		238022
Reported by:	Young <yangx92 AT hotmail.com>
Submitted by:	Young <yangx92 AT hotmail.com>
MFC after:	I don't plan to but you should feel free
Security:	yes
2019-11-29 03:31:47 +00:00
Alan Somers
2eb6acc277 ping, ping6: Use setitimer(2) instead of obsolete alarm(3)
Submitted by:	Ján Sučan <sucanjan@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D22103
2019-11-26 05:06:25 +00:00
Dave Cottlehuber
130cfcf3fc dhclient: support option 114, default-url ascii
This will enable further automation of HTTP UEFI boot loader support by
providing a specific option for providing the boot URL to FreeBSD.

Documented in:

https://www.iana.org/assignments/bootp-dhcp-parameters/bootp-dhcp-parameters.xhtml
https://kb.isc.org/docs/isc-dhcp-44-manual-pages-dhcp-options
https://tools.ietf.org/html/rfc3679

Approved by:	emaste
MFC after:	2 weeks
Sponsored by:	SkunkWerks, GmbH
Differential Revision:	https://reviews.freebsd.org/D22475
2019-11-22 20:22:16 +00:00
Alexander Motin
f97cf1a168 Fix off-by-one error in HPA/AMA maximum reporting.
Before my refactoring the code reported value as maximum number of sectors,
adding one to the maximum sector number returned by respective command.
While this difference is somewhat confusing, restore previous behavior.

MFC after:	3 days
2019-11-22 15:41:47 +00:00
Gleb Smirnoff
71f0077631 Remove sio(4).
It had been disconnected from build in r181233 in 2008.

Reviewed by:	imp
2019-11-21 01:24:49 +00:00
Ed Maste
d9aee13f6f makefs: avoid warning when creating FAT filesystem on existing file
Previously the mkfs_msdos function (from newfs_msdos) emitted warnings
in the case that an image size is specified and the target is not a
file, or no size is specified and the target is not a character device.
The latter warning (not a character device) doesn't make sense when this
code is used in makefs, regardless of whether an image size is specified
or not.

Sponsored by:	The FreeBSD Foundation
2019-11-08 14:11:25 +00:00
Kyle Evans
f852618602 mdmfs(8): add -k skel option to populate fs from a skeleton
mdmfs(8) lacks the ability to populate throwaway memory filesystems from an
existing directory.

This features permits an interesting setup where /var for instance lives on
a device where wear-leveling is something you want to avoid as much as
possible and nonetheless you don't want to lose your logs, ports metadata,
etc. Here are the steps:

1. Copy /var to /var.bak;
2. Mount an mfs into /var using -k /var.bak at startup;
3. Synchronize /var to /var.bak weekly and on shutdown.

Note that this more or less mimics OpenBSD's mount_mfs(8) -P flag.

PR:		146254
Submitted by:	jlh (many moons ago)
MFC after:	1 week
2019-11-01 03:10:53 +00:00
Kirk McKusick
e39c92986d Replace an uninitialized variable with the correct element from the
superblock when doing recovery with journalled soft updates.

Reported by:  Chuck Silvers
MFC after:    3 days
Sponsored by: Netflix
2019-10-22 22:23:59 +00:00
Alan Somers
e9dfc15a83 Fix option names in the Examples section of the manual page
This corrects an oversight from r351423.

Submitted by:	Ján Sučan <sucanjan@gmail.com>
MFC after:	Never
Differential Revision:	https://reviews.freebsd.org/D22093
2019-10-20 20:29:17 +00:00
Andriy Gapon
47adb0e0e0 ddb: use 'textdump dump' instead of 'call doadump'
The change is for the example in textdump.4 and the default ddb.conf.

First of all, doadump now requires an argument and it won't do a
textdump if the argument is not 'true'.
And 'textdump dump' is more idiomatic anyway.

For what it's worth, ddb 'dump' command seems to always request a vmcore
dump even if a textdump was requested earlier, e.g., by 'textdump set'.
Finally, ddb 'call' command is not documented.

MFC after:	2 weeks
2019-10-18 12:32:01 +00:00
Kyle Evans
8338f584dc bectl(8): destroy: use BE_DESTROY_AUTOORIGIN if -o is not specified
-o will force the origin to be destroyed unconditionally.
BE_DESTROY_AUTOORIGIN, on the other hand, will only destroy the origin if it
matches the format used by be_snapshot. This lets us clean up the snapshots
that are clearly not user-managed (because we're creating them) while
leaving user-created snapshots in place and warning that they're still
around when the BE created goes away.
2019-10-16 14:55:56 +00:00
Andrey V. Elsukov
51b1593065 Explicitly initialize the memory buffer to store O_ICMP6TYPE opcode.
By default next_cmd() initializes only first u32 of opcode. O_ICMP6TYPE
opcode has array of bit masks to store corresponding ICMPv6 types.
An opcode that precedes O_ICMP6TYPE, e.g. O_IP6_DST, can have variable
length and during opcode filling it can modify memory that will be used
by O_ICMP6TYPE opcode. Without explicit initialization this leads to
creation of wrong opcode.

Reported by:	Boris N. Lytochkin
Obtained from:	Yandex LLC
MFC after:	3 days
2019-10-15 09:50:02 +00:00
Andriy Gapon
5fda0d60c1 add ability to set watchdog timeout for a shutdown
This change allows to specify a watchdog(9) timeout for a system
shutdown.  The timeout is activated when the watchdogd daemon is
stopped.  The idea is to a prevent any indefinite hang during late
stages of the shutdown.  The feature is implemented in rc.d/watchdogd,
it builds upon watchdogd -x option.

Note that the shutdown timeout is not actiavted when the watchdogd
service is individually stopped by an operator.  It is also not
activated for the 'shutdown' to the single-user mode.  In those cases it
is assumed that the operator knows what they are doing and they have
means to recover the system should it hang.

Significant subchanges and implementation details:
- the argument to rc.shutdown, completely unused before, is assigned to
  rc_shutdown variable that can be inspected by rc scripts
- init(8) passes "single" or "reboot" as the argument, this is not
  changed
- the argument is not mandatory and if it is not set then rc_shutdown is
  set to "unspecified"
- however, the default jail management scripts and jail configuration
  examples have been updated to pass "jail" to rc.shutdown, just in case
- the new timeout can be set via watchdogd_shutdown_timeout rc option
- for consistency, the regular timeout can now be set via
  watchdogd_timeout rc option
- watchdogd_shutdown_timeout and watchdogd_timeout override timeout
  specifications in watchdogd_flags
- existing configurations, where the new rc options are not set, should
  keep working as before

I am not particularly wed to any of the implementation specifics.
I am open to changing or removing any of them as long as the provided
functionality is the same (or very close) to the proposed one.
For example, I think it can be implemented without using watchdogd -x,
by means of watchdog(1) alone.  In that case there would be a small
window between stopping watchdogd and running watchdog, but I think that
that is acceptable.

Reviewed by:	bcr (man page changes)
MFC after:	5 weeks
Relnotes:	yes
Differential Revision: https://reviews.freebsd.org/D21221
2019-10-03 11:23:10 +00:00
Warner Losh
a35a97ae12 Size is unsigned, so remove the test entirely.
The kernel won't crash if you have a bad value and I'd rather not have
nvmecontrol know the internal details about how the nvme driver limits
the transfer size.
2019-09-25 07:51:30 +00:00
Warner Losh
087d0e0b5e After my comnd changes, the number of threads and size weren't set. In
addition, the flags are optional, but were made to be mandatory. Set
these things, as well as santiy check the specified size.

Submitted by: Stefan Rink
PR: 240798
2019-09-25 00:24:57 +00:00