This helps with event correlation when machines are distributed
across multiple time zones.
Format the time with relaxed ISO 8601 for all the usual reasons.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
sbin/fsck_msdosfs/fat.c:
- readfat:
* Only truncate out-of-range cluster pointers (1, or greater than
NumClusters but smaller than CLUST_RSRVD), as the current cluster
may contain some data. We can't fix reserved cluster pointers at
this pass, because we do no know the potential cluster preceding
it.
* Accept valid cluster for head bitmap. This is a no-op, and mainly
to improve code readability, because the 1 is already handled in
the previous else if block.
- truncate_at: absorbed into checkchain.
- checkchain: save the previous node we have traversed in case that we
have a chain that ends with a special (>= CLUST_RSRVD) cluster, or is
free. In these cases, we need to truncate at the cluster preceding the
current cluster, as the current cluster contains a marker instead of
a next pointer and can not be changed to CLUST_EOF (the else case can
happen if the user answered "no" at some point in readfat()).
- clearchain: correct the iterator for next cluster so that we don't
stop after clearing the first cluster.
- checklost: If checkchain() thinks the chain have no cluster, it
doesn't make sense to reconnect it, so don't bother asking.
Reviewed by: kevlo
MFC after: 24 days
X-MFC-With: r356313
Differential Revision: https://reviews.freebsd.org/D23065
In the code we used NumClusters as the upper (non-inclusive) boundary
of valid cluster number, so the actual value was 2 (CLUST_FIRST) more
than the real number of clusters. This causes a FAT16 media with
65524 clusters be treated as FAT32 and might affect FAT12 media with
4084 clusters as well.
To fix this, we increment NumClusters by CLUST_FIRST after the type
determination.
PR: 243179
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D23082
Add an option flag so that arbitrary updates to a lagg's configuration
do not clear sc_stride. Preseve compatibility for old ifconfig
binaries. Update ifconfig to use the new flag and improve the casting
used when parsing the option parameter.
Modify the RR transmit function to avoid locklessly reading sc_stride
twice. Ensure that sc_stride is always 1 or greater.
Reviewed by: hselasky
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23092
This is a re-apply r356249 with changes to make GCC happy.
This utility was initially written for FAT12/16, which were inherently
small. When FAT32 support was added, the old data structure and
algorithms remain used with minimal changes.
With growing size of FAT32 media, the current data structure that
requires 4 32-bit variables per each FAT32 table entry would consume up
to 4 GiB of RAM, which can be too big for systems with limited RAM
available.
Address this by taking a different approach of validating the FAT.
The FAT is essentially a set of linked lists of chains that was
referenced by directory entries, and the checker needs to make sure that
the linked chains of clusters do not have cross-linked chains, and every
chain were referenced by one and only one directory entry. Instead of
keeping track of the chain's 'head' cluster number, the size of the
chain, the used status of the chain and the "next" pointer which is
content of the FAT table, we create accessors for the FAT table data
for the "next" pointer, and keep only one bit to indicate if the
current cluster is a 'head' node of a cluster chain, in a bitmap.
We further overhaul the FAT checker to find out the possible head nodes
by excluding ones that are not (in other words, nodes that have some
other nodes claiming them as the next node) instead of marking the head
nodes for each node on the chain. This approach greatly reduced the
complexiety of computation from O(N^2) worst case, to an O(N) scan for
worst case. The file (cluster chain) length is not useful for the FAT
checker, so don't bother to calculate them in the FAT checker and
instead leave the task to the directory structure check, at which point
we would have non-crossed cluster chains, and we are guaranteed that
each cluster will be visited for at most one time.
When checking the directory structures, we use the head node indicator
to as the visited (used) flag: every cluster chain can only be
referenced by one directory entry, so we clear them when calculating
the length of the chain, and we can immediately tell if there are
anomalies in the directory entry.
As a result, the required RAM size is now 1 bit per each entry of
the FAT table, plus memory needed to hold the FAT table in memory,
instead of 16 bytes (=128 bits) per each entry. For FAT12 and FAT16,
we will load the whole FAT table into memory as they are smaller than
128KiB, and for FAT32, we first attempt to mmap() it into memory, and
when that fails, we would fall back to a simple LRU cache of 4 MiB of
RAM.
sbin/fsck_msdosfs/boot.c:
- Added additional sanity checks for valid FAT32/FAT16/FAT12 cluster
number.
- FAT32: check if root directory starts with a valid cluster number,
moved from dir.c. There is no point to proceed if the filesystem
is already damaged beyond repair.
sbin/fsck_msdosfs/check.c:
- Combine phase 1 and phase 2, now that the readfat() is able to
detect cross chains.
sbin/fsck_msdosfs/dir.c:
- Refactor code to use FAT accessor instead of accessing the internal
representation of FAT table.
- Make use of the cluster chain head bitmap.
- Clarify and simplify directory entry check, remove unnecessary
checks that are would be done at a later time (for example, whether
the directory's second cluster is a valid one, which is examined
more throughly in a later checkchain() and does not prevent us
from proceeding further).
sbin/fsck_msdosfs/dosfs.h:
- Remove internal representation of FAT table, which is replaced by
the head bitmap that is opaque to other code.
- Added a special CLUST_DEAD cluster type to indicate errors.
sbin/fsck_msdosfs/ext.h:
- Added a flag that overrides mmap(2) setting. The corresponding
command line option, -M is intentionally undocumented as we do not
expect users to need it.
- Added accessors for FAT table and convert existing interface to use
it.
sbin/fsck_msdosfs/fat.c:
- Added head bitmap to represent whether a cluster is a head cluster.
- Converted FAT internal representation to accessors.
- Implemented a LRU cache for FAT32 when mmap(2) should not or can not
be used.
- _readfat: Attempt a mmap(2) and fall back to regular read for
non-FAT32 file systems; use the LRU cache for FAT32 and prepopulate
the cache with the first 4MiB of the entries.
- readfat: Added support of head bitmap and use the population scan to
detect bogus chains.
- clusterdiff: removed, FATs are copied from the checked copy via
writefat()/copyfat().
- checkchain: calculates the length of a cluster chain and make sure
that it ends with a valid EOF marker.
- clearchain: follow and clear a chain and maintain the free cluster
count.
- checklost: convert to use head bitmap. At the end of all other scans,
the remaining 'head' nodes are leaders of lost cluster chains.
sbin/fsck_msdosfs/fat.c:
- Added a new -M option which is intentionally undocumented, to disable
the use of mmap().
Reviewed by: kevlo
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D22965
When removing a boot environment iterate over the dependents and process the
snapshots by grabbing any clones. Promote the clones we found and then
remove the target environment.
This fixes the ability to destroy a boot environment when it has been used
to spawn one or more other boot environments.
PR: 242592
Submitted by: Wes Maag <jwmaag gmail com> (with changes by myself)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22953
This utility was initially written for FAT12/16, which were inherently
small. When FAT32 support was added, the old data structure and
algorithms remain used with minimal changes.
With growing size of FAT32 media, the current data structure that
requires 4 32-bit variables per each FAT32 table entry would consume up
to 4 GiB of RAM, which can be too big for systems with limited RAM
available.
Address this by taking a different approach of validating the FAT.
The FAT is essentially a set of linked lists of chains that was
referenced by directory entries, and the checker needs to make sure that
the linked chains of clusters do not have cross-linked chains, and every
chain were referenced by one and only one directory entry. Instead of
keeping track of the chain's 'head' cluster number, the size of the
chain, the used status of the chain and the "next" pointer which is
content of the FAT table, we create accessors for the FAT table data
for the "next" pointer, and keep only one bit to indicate if the
current cluster is a 'head' node of a cluster chain, in a bitmap.
We further overhaul the FAT checker to find out the possible head nodes
by excluding ones that are not (in other words, nodes that have some
other nodes claiming them as the next node) instead of marking the head
nodes for each node on the chain. This approach greatly reduced the
complexiety of computation from O(N^2) worst case, to an O(N) scan for
worst case. The file (cluster chain) length is not useful for the FAT
checker, so don't bother to calculate them in the FAT checker and
instead leave the task to the directory structure check, at which point
we would have non-crossed cluster chains, and we are guaranteed that
each cluster will be visited for at most one time.
When checking the directory structures, we use the head node indicator
to as the visited (used) flag: every cluster chain can only be
referenced by one directory entry, so we clear them when calculating
the length of the chain, and we can immediately tell if there are
anomalies in the directory entry.
As a result, the required RAM size is now 1 bit per each entry of
the FAT table, plus memory needed to hold the FAT table in memory,
instead of 16 bytes (=128 bits) per each entry. For FAT12 and FAT16,
we will load the whole FAT table into memory as they are smaller than
128KiB, and for FAT32, we first attempt to mmap() it into memory, and
when that fails, we would fall back to a simple LRU cache of 4 MiB of
RAM.
sbin/fsck_msdosfs/boot.c:
- Added additional sanity checks for valid FAT32/FAT16/FAT12 cluster
number.
- FAT32: check if root directory starts with a valid cluster number,
moved from dir.c. There is no point to proceed if the filesystem
is already damaged beyond repair.
sbin/fsck_msdosfs/check.c:
- Combine phase 1 and phase 2, now that the readfat() is able to
detect cross chains.
sbin/fsck_msdosfs/dir.c:
- Refactor code to use FAT accessor instead of accessing the internal
representation of FAT table.
- Make use of the cluster chain head bitmap.
- Clarify and simplify directory entry check, remove unnecessary
checks that are would be done at a later time (for example, whether
the directory's second cluster is a valid one, which is examined
more throughly in a later checkchain() and does not prevent us
from proceeding further).
sbin/fsck_msdosfs/dosfs.h:
- Remove internal representation of FAT table, which is replaced by
the head bitmap that is opaque to other code.
- Added a special CLUST_DEAD cluster type to indicate errors.
sbin/fsck_msdosfs/ext.h:
- Added a flag that overrides mmap(2) setting. The corresponding
command line option, -M is intentionally undocumented as we do not
expect users to need it.
- Added accessors for FAT table and convert existing interface to use
it.
sbin/fsck_msdosfs/fat.c:
- Added head bitmap to represent whether a cluster is a head cluster.
- Converted FAT internal representation to accessors.
- Implemented a LRU cache for FAT32 when mmap(2) should not or can not
be used.
- _readfat: Attempt a mmap(2) and fall back to regular read for
non-FAT32 file systems; use the LRU cache for FAT32 and prepopulate
the cache with the first 4MiB of the entries.
- readfat: Added support of head bitmap and use the population scan to
detect bogus chains.
- clusterdiff: removed, FATs are copied from the checked copy via
writefat()/copyfat().
- checkchain: calculates the length of a cluster chain and make sure
that it ends with a valid EOF marker.
- clearchain: follow and clear a chain and maintain the free cluster
count.
- checklost: convert to use head bitmap. At the end of all other scans,
the remaining 'head' nodes are leaders of lost cluster chains.
sbin/fsck_msdosfs/fat.c:
- Added a new -M option which is intentionally undocumented, to disable
the use of mmap().
Reviewed by: kevlo
MFC after: 1 month
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D22965
When mount_nfs calls nmount(2), certain NFSv4 specific errors such as
NFSERR_MINORVERMISMATCH can be returned.
Without this patch, 10021 is reported as an unknown error.
This is not particulcarily serious, but make it difficult for sysadmins
to figure out why the mount attempt is failing.
This patch uses nfsv4_errstr.h to convert 10021 and similar to error strings
that can be printed out.
A positive side effect of this patch is the removal of a reference to
sys/nfsclient/nfs.h, which should no longer be used, since it is
part of the old NFS client.
This patch should only affect reporting of failed mount attempts and not the
semantics of NFS mount attempts.
USE JOURNAL? [yn]
when the journal timestamp does not match the filesystem mount time
as we are just going to print an error and fall through to a full fsck.
Instead, just run a full fsck.
Requested by: Bjoern A. Zeeb (bz)
MFC after: 7 days
- Don't allow an unprivileged user to set the stride. [1]
- Only set the stride under the softc lock.
- Rename the internal fields to accurately reflect their use. Keep
ro_bkt to avoid changing the user API.
- Simplify the implementation. The port index is just sc_seq / stride.
- Document rr_limit in ifconfig.8.
Reported by: Ilja Van Sprundel <ivansprundel@ioactive.com> [1]
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22857
This avoids gcc9 warning about unaligned access to the structure when
casting to uint16_t pointer type.
Submitted by: imp
Reviewed by: imp
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D22888
I was unable to find time when those were working. I think they have been
broken for at least 5 years or even longer.
Discussed with: avg@
MFC after: 1 month
It seems I read specifications not careful enough. There are devices not
setting successful completion bit, causing previous code report false error.
MFC after: 1 week
r355677 added NFSv4.2 support to the NFS client. This patch updates the
mount_nfs.8 man page to reflect that.
It also clarifies that the "nolockd" option does not apply to NFSv4 mounts.
This is a content change.
Warn users when they try to add/delete/modify more items than the kernel will
allow.
Reviewed by: allanjude (previous version), Lutz Donnerhacke
Differential Revision: https://reviews.freebsd.org/D22733
Due to off-by-one error in brackets counting it consumed the rest of the
string, preventing later variables expansions.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Remove single use of dubious srandomdev(3) + random(3) and replace with
arc4random(3), as is used already in this program.
Follow-up question: Do we really need this program anymore? In base?
Update a bunch of Makefile.depend files as
a result of adding Makefile.depend.options files
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22494
Leaf directories that have dependencies impacted
by options need a Makefile.depend.options file
to avoid churn in Makefile.depend
DIRDEPS for cases such as OPENSSL, TCP_WRAPPERS etc
can be set in local.dirdeps-options.mk
which can add to those set in Makefile.depend.options
See share/mk/dirdeps-options.mk
Reviewed by: bdrewery
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D22469
- Do not unnecessarily strdup().
- Check return value of getdiskinfo(), if it failed, bail out.
Reviewed by: imp
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22729
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.
The previous revision missed the exact same error in a copy paste block
of the same code in another function. Fix the identical case, too.
A DHCP client identifier is simply the hardware type (one byte)
concatenated with the hardware address (some variable number of bytes,
but at most 16). Limit the size of the temporary buffer to match and
the rest of the calculations shake out correctly.
PR: 238022
Reported by: Young <yangx92 AT hotmail.com>
Submitted by: Young <yangx92 AT hotmail.com>
MFC after: I don't plan to but you should feel free
Security: yes
Before my refactoring the code reported value as maximum number of sectors,
adding one to the maximum sector number returned by respective command.
While this difference is somewhat confusing, restore previous behavior.
MFC after: 3 days
Previously the mkfs_msdos function (from newfs_msdos) emitted warnings
in the case that an image size is specified and the target is not a
file, or no size is specified and the target is not a character device.
The latter warning (not a character device) doesn't make sense when this
code is used in makefs, regardless of whether an image size is specified
or not.
Sponsored by: The FreeBSD Foundation
mdmfs(8) lacks the ability to populate throwaway memory filesystems from an
existing directory.
This features permits an interesting setup where /var for instance lives on
a device where wear-leveling is something you want to avoid as much as
possible and nonetheless you don't want to lose your logs, ports metadata,
etc. Here are the steps:
1. Copy /var to /var.bak;
2. Mount an mfs into /var using -k /var.bak at startup;
3. Synchronize /var to /var.bak weekly and on shutdown.
Note that this more or less mimics OpenBSD's mount_mfs(8) -P flag.
PR: 146254
Submitted by: jlh (many moons ago)
MFC after: 1 week
This corrects an oversight from r351423.
Submitted by: Ján Sučan <sucanjan@gmail.com>
MFC after: Never
Differential Revision: https://reviews.freebsd.org/D22093
The change is for the example in textdump.4 and the default ddb.conf.
First of all, doadump now requires an argument and it won't do a
textdump if the argument is not 'true'.
And 'textdump dump' is more idiomatic anyway.
For what it's worth, ddb 'dump' command seems to always request a vmcore
dump even if a textdump was requested earlier, e.g., by 'textdump set'.
Finally, ddb 'call' command is not documented.
MFC after: 2 weeks
-o will force the origin to be destroyed unconditionally.
BE_DESTROY_AUTOORIGIN, on the other hand, will only destroy the origin if it
matches the format used by be_snapshot. This lets us clean up the snapshots
that are clearly not user-managed (because we're creating them) while
leaving user-created snapshots in place and warning that they're still
around when the BE created goes away.
By default next_cmd() initializes only first u32 of opcode. O_ICMP6TYPE
opcode has array of bit masks to store corresponding ICMPv6 types.
An opcode that precedes O_ICMP6TYPE, e.g. O_IP6_DST, can have variable
length and during opcode filling it can modify memory that will be used
by O_ICMP6TYPE opcode. Without explicit initialization this leads to
creation of wrong opcode.
Reported by: Boris N. Lytochkin
Obtained from: Yandex LLC
MFC after: 3 days
This change allows to specify a watchdog(9) timeout for a system
shutdown. The timeout is activated when the watchdogd daemon is
stopped. The idea is to a prevent any indefinite hang during late
stages of the shutdown. The feature is implemented in rc.d/watchdogd,
it builds upon watchdogd -x option.
Note that the shutdown timeout is not actiavted when the watchdogd
service is individually stopped by an operator. It is also not
activated for the 'shutdown' to the single-user mode. In those cases it
is assumed that the operator knows what they are doing and they have
means to recover the system should it hang.
Significant subchanges and implementation details:
- the argument to rc.shutdown, completely unused before, is assigned to
rc_shutdown variable that can be inspected by rc scripts
- init(8) passes "single" or "reboot" as the argument, this is not
changed
- the argument is not mandatory and if it is not set then rc_shutdown is
set to "unspecified"
- however, the default jail management scripts and jail configuration
examples have been updated to pass "jail" to rc.shutdown, just in case
- the new timeout can be set via watchdogd_shutdown_timeout rc option
- for consistency, the regular timeout can now be set via
watchdogd_timeout rc option
- watchdogd_shutdown_timeout and watchdogd_timeout override timeout
specifications in watchdogd_flags
- existing configurations, where the new rc options are not set, should
keep working as before
I am not particularly wed to any of the implementation specifics.
I am open to changing or removing any of them as long as the provided
functionality is the same (or very close) to the proposed one.
For example, I think it can be implemented without using watchdogd -x,
by means of watchdog(1) alone. In that case there would be a small
window between stopping watchdogd and running watchdog, but I think that
that is acceptable.
Reviewed by: bcr (man page changes)
MFC after: 5 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D21221
The kernel won't crash if you have a bad value and I'd rather not have
nvmecontrol know the internal details about how the nvme driver limits
the transfer size.
