Traditionally the GEOM's primary channel of information from kernel to
user-space was confxml, fetched by libgeom through kern.geom.confxml
sysctl. It is convenient and informative, representing full state of
GEOM in a single XML document. But problems start to arise on systems
with hundreds of disks, where the full confxml size reaches many
megabytes, taking significant time to first write it and then parse.
This patch introduces alternative solution, allowing to fetch much
smaller XML document, subset of the full confxml, limited to 64KB and
representing only one specified geom and optionally its parents. It
uses existing GEOM control interface, extended with new "getxml" verb.
In case of any error, such as the buffer overflow, it just transparently
falls back to traditional full confxml. This patch uses the new API in
user-space GEOM tools where it is possible.
Reviewed by: imp
MFC after: 2 month
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D34529
Create g_part_getattr to allow gpart geoms to have their attributes queried.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D32782
The gunion(8) utility is used to track changes to a read-only disk on
a writable disk. Logically, a writable disk is placed over a read-only
disk. Write requests are intercepted and stored on the writable
disk. Read requests are first checked to see if they have been
written on the top (writable disk) and if found are returned. If
they have not been written on the top disk, then they are read from
the lower disk.
The gunion(8) utility can be especially useful if you have a large
disk with a corrupted filesystem that you are unsure of how to
repair. You can use gunion(8) to place another disk over the corrupted
disk and then attempt to repair the filesystem. If the repair fails,
you can revert all the changes in the upper disk and be back to the
unchanged state of the lower disk thus allowing you to try another
approach to repairing it. If the repair is successful you can commit
all the writes recorded on the top disk to the lower disk.
Another use of the gunion(8) utility is to try out upgrades to your
system. Place the upper disk over the disk holding your filesystem
that is to be upgraded and then run the upgrade on it. If it works,
commit it; if it fails, revert the upgrade.
Further details can be found in the gunion(8) manual page.
Reviewed by: Chuck Silvers, kib (earlier version)
tested by: Peter Holm
Differential Revision: https://reviews.freebsd.org/D32697
The gctl_error() function provides GEOM modules with the ability
to report only a single message. When running with the verbose
flag, commands that handle multiple devices may want to report a
message for each of the devices on which it operates. This commit
adds the gctl_msg() function that can be called multiple times
to post messages. When finished issuing messages, the application
must either call gctl_post_messages() or call gctl_error() to cause
the messages to be reported to the calling process.
Tested by: Peter Holm
When using bio's created by g_clone_bio() or g_duplicate_bio()
their consumer device (the device to which their I/O requests
are sent) is listed by the geom debugging facility as [unknown].
If available, this update lists the consumer associated with
the bio's parent.
MFC after: 2 weeks
Sponsored by: Netflix
All I/O requests through the taste consumers are synchronous, done
with g_read_data() and without any locks held. It makes no sense
to delegate the I/O to g_down/g_up threads.
This removes many of context switches during disk retaste.
MFC after: 2 weeks
The geom_gate API provides 2 distinct paths for exchanging error
details between the kernel and the userland client: Including an error
code in the g_gate_ctl_io structure passed in the ioctl(2) call or
having the ioctl(2) call return -1 with an error code in errno. The
latter reflects errors in the ioctl(2) call itself whilst the former
reflects errors within the geom_gate instance.
The G_GATE_CMD_START ioctl blocks waiting for an I/O request to be
directed to the geom_gate instance and the wait can fail
(necessitating an error return) if the geom_gate instance is destroyed
or if the msleep(9) fails. The code previously treated both error
cases indentically: Returning ECANCELED as a geom_gate instance error
(which the ggatec treats as a fatal error). Whilst this is the correct
behaviour if the geom_gate instance is destroyed, a msleep(9) failure
is unrelated to the geom_gate instance itself and should be reported
as an ioctl(2) "failure". The distinction is important because
msleep(9) can return ERESTART, which means the system call should be
retried (and this will occur automatically as part of the generic
syscall return processing).
This change alters the msleep(9) handling to directly return the error
code from msleep(9), which ensures ERESTART is correctly handled,
rather than being treated as a fatal error.
Reviewed by: Johannes Totz <jo@bruelltuete.com>
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D33996
All I/O requests through the taste consumer are synchronous, done
with g_read_data() and without any locks held. It makes no sense
to delegate the I/O to g_down/g_up threads.
This removes many of context switches during disk retaste.
MFC after: 2 weeks
The only cases when direct dispatch does not make sense is for I/O
submission from down thread and for completion from up thread. In
all other cases, if both consumer and producer are OK about it, we
can save on context switches.
MFC after: 2 weeks
Unlike normal consumers all taste consumer I/O is synchronous, done
with g_read_data() and without any locks held. It makes no sense to
delegate I/O submission to g_down thread.
This should remove number of context switches during disk retaste.
MFC after: 2 weeks
Like BIO_FLUSH, there is no reason for consumers to pass a BIO_SPEEDUP
request with non-NULL bio_data, so assert this.
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
In particular, there is no need to allocate a data block when passing
BIO_FLUSH requests to child providers, and g_io_request() asserts that
bp->bio_data == NULL for such requests.
PR: 255131
Reported and tested by: nvass@gmx.com
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
I observed a situation where some read requests failed when a 2-way geom
mirror lost one disk. The problem appears to be in the logic that skips
retrying a failed request when a mirror has only one active disk.
Generally, that makes sense. But during a transition from two disks to
one it is possible that the request failed on the failing disk before it
was inactivated and, so, the remaining active disk is the disk that
should be tried.
This change adds an additional check to ensure that it was the (only)
active disk that was already tried.
Reviewed by: mav
MFC after: 3 weeks
As documented in the HiFive Unmatched Software Reference Manual.
Reviewed by: imp, mhorne
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D34010
These routines are used internally by GEOM to dispatch I/O requests to a
provider, typically for tasting or for updating GEOM class metadata
blocks.
These routines assumed that partial I/O did not occur without setting
BIO_ERROR, but this is possible in at least two cases:
- Some or all of the I/O range is beyond the provider's mediasize.
In this scenario g_io_check() truncates the bounds of the request
before it is handed to the target provider.
- A read from vnode-backed md(4) device returns EOF (the backing vnode
is allowed to be smaller than the device itself) or partial vnode I/O
occurs.
In these scenarios g_read_data() could return a partially uninitialized
buffer. Many consumers are not affected by the first case, since the
offsets used for provider metadata or tasting are relative to the
provider's mediasize, but in some cases metadata is read at fixed
offsets, such as when searching for a UFS superblock using the offsets
defined by SBLOCKSEARCH.
Thus, modify the routines to explicitly check for a non-zero residual
and return EIO in that case. Remove a related check from the
DIOCGDELETE ioctl handler, it is handled within g_delete_data() now.
Reviewed by: mav, imp, kib
Reported by: KMSAN
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31293
The only event hooked up is NOTE_ATTRIB, which is triggered when the
device is resized. Support for other NOTE_* events to follow.
Reviewed by: kib, jhb
Differential Revision: https://reviews.freebsd.org/D33402
IVs are not the size of keys as a general case. Most often they are
the size of a single block.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33885
It must be greater than zero, and be multiple of the device block size.
In collaboration with: pho
Reviewed by: markj, mckusick
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33721
With crafted input to the G_GATE_CMD_CREATE ioctl, geom_gate can be made
to print kernel memory to the system console, potentially revealing
sensitive data from whatever was previously in that memory page.
But but but: this is a case of the sys admin misconfiguring, and you'd
need root privileges to do this.
Submitted By: Johannes Totz <jo@bruelltuete.com>
MFC after: 2 weeks
Reviewed By: asomers
Differential Revision: https://reviews.freebsd.org/D31727
- Remove timeouts from msleep()'s. Those should always be woken up.
- Move wakeup() under the lock to not call on possibly freed pointer.
- Remove some dead code.
MFC after: 2 weeks
The commit at hand happens to break userspace build as the header ins
included by sbin/gbde/gbde.c and the __diagused macro is not provided to
userspace.
Revert until this gets sorted out.
This reverts commit 26e837e2d4.
This definition enables callers to estimate remaining space on the
kstack, and take action on it. Notably, it enables optimizations in the
GEOM and netgraph subsystems to directly dispatch work items when there
is sufficient stack space, rather than queuing them for a worker thread.
Implement it for riscv, arm, and mips. Remove the #ifdefs, so it will
not go unimplemented elsewhere.
PR: 259157
Reviewed by: mav, kib, markj (previous version)
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32580
Single provider may have multiple consumers, and locking one of consumers
is not sufficient to protect the provider. Though the only part of the
provider this locking protects now is its statistics.
Reported by: Arka Sharma <arka.sw1988@gmail.com>
MFC after: 2 weeks
The mntfs vnode lock should be before topology, as established in
ffs_mountfs(). Extend the locked region in ffs_unmount().
Reported and reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D33013
disk labels.
When the geom label subsystem is checking labels to discover if they
are UFS/FFS filesystems, do not print a kernel error message if a
superblock is found with a check-hash error. That issue is best
handled later if an attempt is made to actually use the filesystem.
Sponsored by: Netflix
It is needed for g_vfs_close() invalidating the buffers. We rely on the
vnode lock for correctness.
Reported and tested by: pho
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D32761
Move the efimedia reporting to g_part_mbr_efimedia and use that from
g_part_mbr_dumpconf to report it.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D32781
Move the efimedia reporting to g_part_gpt_efimedia and use that from
g_part_gpt_dumpconf to report it.
Sponsored by: Netflix
Reviewed by: mav
Differential Revision: https://reviews.freebsd.org/D32780
I dropped the + 1 from the other two instances in each file but failed
to do so for this one, resulting in a more egregious buffer overread
than the one I was fixing (since the read character ended up in the
output if there was space).
Reported by: Jenkins
Fixes: 34fb1c133c ("Fix intra-object buffer overread for labeled msdosfs volumes")
