We periodically record synchronization progress in the metadata
block of the disk being synchronized; this allows an interrupted
synchronization to be resumed. However, the frequency of these
updates heavily pessimized synchronization time on some media. This
change modifies gmirror to update metadata based on a time period,
and adds a sysctl to control that period. The default value results
in a much lower update frequency and increases the completion time
for an interrupted rebuild only marginally.
Reported by: Andre Albsmeier <andre@fbsd.e4m.org>
MFC after: 3 weeks
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
A negative value can be used to suppress all prints from the gmirror
kernel code, which can be useful when attempting to trigger race
conditions using stress tests.
MFC after: 1 week
produces hex numbers for the dsn. Since that come is from EDK2, change
this for symmetry, by generating the dsn as a hex number.
Noticed by: gpart list | grep efimedia | awk -F: '{print $2;}' | \
sed -e 's/^ *//g;s/,,/,/' | grep MBR | efidp -p | efidp -f
Sponsored by: Netflix
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Initially, only tag files that use BSD 4-Clause "Original" license.
RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133
This geom does not immediately detach its consumer relying on the
wither-washer to do that. Since that happens asynchronously we may get
additional spoiling events. So, we need to account for that.
There are multiple options for fixing this issue like detaching
immediately or checking for G_CF_ORPHAN in g_slice_spoiled().
The most reliable and least intrusive fix seems to be setting
geom->softc to NULL on the first call and checking for NULL on
subsequent calls. This is something that the code did before r325227.
Reported by: David Wolfskill <david@catwhisker.org>,
O. Hartmann <o.hartmann@walstatt.org>
Tested by: David Wolfskill <david@catwhisker.org> (earlier version)
Discussed with: mav
MFC after: 1 week
X-MFC with: r325227
At present, g_slice_orphan and g_slice_spoiled destroy the softc
(struct g_slicer) even before calling g_wither_geom, so there can
be active and incoming io requests at that time and g_slice_start
can access the softc.
This commit changes the code to destroy the softc only after all
providers are closed.
While there, a couple of small cleanups.
Reported by: Ben RUBSON <ben.rubson@gmail.com>
Tested by: Ben RUBSON <ben.rubson@gmail.com>
Reviewed by: mav, smh (earlier version)
MFC after: 2 weeks
Sponsored by: Panzura
Differential Revision: https://reviews.freebsd.org/D12809
g_mirror_destroy() is supposed to unlock the softc before indicating
success, but it wasn't doing so if the caller raced with another
thread destroying the mirror.
MFC after: 1 week
Sponsored by: Dell EMC Isilon
When using a kernel built with the GZIO config option, dumpon -z can be
used to configure gzip compression using the in-kernel copy of zlib.
This is useful on systems with large amounts of RAM, which require a
correspondingly large dump device. Recovery of compressed dumps is also
faster since fewer bytes need to be copied from the dump device.
Because we have no way of knowing the final size of a compressed dump
until it is written, the kernel will always attempt to dump when
compression is configured, regardless of the dump device size. If the
dump is aborted because we run out of space, an error is reported on
the console.
savecore(8) is modified to handle compressed dumps and save them to
vmcore.<index>.gz, as it does when given the -z option.
A new rc.conf variable, dumpon_flags, is added. Its value is added to
the boot-time dumpon(8) invocation that occurs when a dump device is
configured in rc.conf.
Reviewed by: cem (earlier version)
Discussed with: def, rgrimes
Relnotes: yes
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D11723
GEOM consumer can be orphaned, and then reattach to another provider.
From a user point of view, this makes gmountver(4) work again.
Reviewed by: avg, mav
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12228
Like r266444, g_resize_provider_event can attempt to orphan an already
orphaned geom_dev consumer. This will cause a panic in g_dev_orphan. Apply
the same fix as was applied to g_orphan_register.
Reviewed by: ae
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12469
In theory, all data access errors mean that a member is out of sync
at most. But they were treated as more serious errors to avoid the
situation where a flaky disk gets repeatedly disconnected, re-synchronized,
reconnected and then disconnected again.
ENXIO is a special error that means that the member disk disappeared,
so it should get the same handling as the GEOM orphaning event.
There is a better chance that when the disk is reconnected, it will be
a good member again.
When ENXIO happens on a read we use the exisiting G_MIRROR_BUMP_SYNCID
mechanism which means that the mirror's syncid is increased as soon
as there is a write to the mirror. That's because no data has got out
of sync yet, but the problematic memeber is disconnected, so the future
write will make it stale.
When ENXIO happens on a write we use a new G_MIRROR_BUMP_SYNCID_NOW
mechanism which means that we update the mirror metadata as soon as
possible because the problematic memeber is already behind.
Reviewed by: markj, imp
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D9463
In integrity mode, a larger logical sector (e.g., 4096 bytes) spans several
physical sectors (e.g., 512 bytes) on the backing device. Due to hash
overhead, a 4096 byte logical sector takes 8.5625 512-byte physical sectors.
This means that only 288 bytes (256 data + 32 hash) of the last 512 byte
sector are used.
The memory allocation used to store the encrypted data to be written to the
physical sectors comes from malloc(9) and does not use M_ZERO.
Previously, nothing initialized the final physical sector backing each
logical sector, aside from the hash + encrypted data portion. So 224 bytes
of kernel heap memory was leaked to every block :-(.
This patch addresses the issue by initializing the trailing portion of the
physical sector in every logical sector to zeros before use. A much simpler
but higher overhead fix would be to tag the entire allocation M_ZERO.
PR: 222077
Reported by: Maxim Khitrov <max AT mxcrypt.com>
Reviewed by: emaste
Security: yes
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12272
the g_journal level needs to check whether it is holding a newer
copy of the block than that which exists on the disk. If so, it
needs to return its copy. If not, it should pass the request down
to the disk to fulfill. It currently considers six queues:
0) delayed queue,
1) unsent (current queue),
2) in-flight to the journal (flush queue),
3) active journal (active queue),
4) inactive journal (inactive queue), and
5) inflight to the disk (copy queue).
Checking on two of these queues is unnecessary:
0) The delayed requests should not be used for reads because they
have not yet been entered into the journal, so their value should
reflect the disk contents, not the future contents that are not
yet committed.
2) Because all the bio's in the flush queue are also found on the
active queue, there is no need to inspect the flush queue for
reads since they will be found when searching the active queue.
Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Discussed with: kib
MFC after: 1 week
geom_bsd, geom_mbr and geom_sunlabel have been obsolete since Marcel
Moolenaar's geom_part was in FreeBSD 7. They haven't been in GENERIC
since FreeBSD 8. Add warning when used.
geom_vol_ffs has been obsolete since ufs support to geom_label was
committed in FreeBSD 5. It hasn't been in GENERIC since FreeBSD 5.
Add warning when used.
geom_fox has been obsolete since gmultipath was committed in FreeBSD 7.
(no warning added, since this is a very obscure class).
These will all be removed in FreeBSD 12.
MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D11935
Note: Classes will be removed after MFC
No need to set any fields in the cloned device. devfs uses symlinks,
so the adev entries returned won't be presented to the drivers. Since
we don't save copies, nothing else will see them. This code came from
the old compat code, and it appears to be obsolete or never needed.
Submitted by: kib@
Differential Review: https://reviews.freebsd.org/D11919
Implement disk_add_alias to allow aliases to be added to disks. All
disk have a primary name (say "foo") can also have secondary names
(say "bar") such that all instances of "foo" also have a "bar"
alias. So if you have foo0, foo0p1, foo1, foo1s1 and foo1s1a nodes
created by the foo driver and gpart, device nodes bar0, bar0p1, bar1,
bar1s1 and bar1s1a will appear as symlinks back to the original nodes.
This generalizes to multiple aliases. However, since the unit number
follows the primary name, multiple device drivers can't create the
same aliases unless those drives coorinate the unit number space (eg
you couldn't add an alias 'disk' to both 'da' and 'ada' because it's
possible to have da0 and ada0, because 'disk0' is ambiguous).
Differential Revision: https://reviews.freebsd.org/D11873
When we're creating new providers for each of the partitions, add
aliases to the geom before we create the provider so when geom_dev
tastes the provider, the aliases are in place so the proper /dev
entries are created. So foo5p6 gets created as an alias for bar5p6
when foo is an alias for bar in the geom we're partitioning with
g_part. This also copies aliases from the container geom (eg disk) to
the label geom (the disk with GPT partitioning) so that aliases nest
properly.
Differential Revision: https://reviews.freebsd.org/D11873
Add an alias name list to geoms. Use them in geom_dev to create
aliases. Previously, geom_dev would create an device node for the name
of the geom. Now, additional nodes are created pointing back to the
primary node with make_dev_alias_p. Aliases must be in place on the
geom before any tasting occurs.
Differential Revision: https://reviews.freebsd.org/D11873
in the flush_queue:
1 2 3 4 5 6 7 8 9 10
and another 10 bio's go into the flush queue after only the first five
bio's are removed from the flush queue, the queue should look like:
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20,
but because of the bug we end up with
6 11 12 13 14 15 16 17 18 19 20 7 8 9 10.
So the sequence of the bio's is damaged in the flush queue (and
therefore in the journal on disk !). This error can be triggered by
ffs_snapshot() when a block is read with readblock() and gjournal finds
this block in the broken flush queue before it goes to the correct
active queue.
The fix is to place all new blocks at the end of the queue.
Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Discussed with: kib
MFC after: 1 week
system having over 4GB RAM. That's due to:
1) the limit being u_int instead of u_long like vm.kmem_size (the limit is
half of vm.kmem_size by default for amd64);
2) sysctl handler g_journal_cache_limit_sysctl() using u_int instead of u_long.
The fix is to replace u_int with u_long for the kern.geom.journal.cache.limit
sysctl variable.
PR: 198500
Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Reported by: Eugene Grosbein
Discussed with: kib
MFC after: 1 week
RPI1-B, Alix and APU2 boards as well as NanoBSD with the following message:
vnode_pager_generic_getpages_done: I/O read error 5
Seems the breakage was because it was missed to include acr in glabel update.
Reported by: Peter Blok <pblok@bsd4all.org>,
madpilot, imp and trasz.
Reviewed by: trasz
Tested by: Peter Blok and madpilot.
MFC after: 3 days.
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D11365
Add -o [no]verify option to mdconfig (and document in man page.)
Implement GEOM attribute MNT::verified to ask md if the backing vnode is
verified.
Check for MNT::verified in cd9660 mount to flag the mount as MNT_VERIFIED if
the underlying device has been verified.
Reviewed by: rwatson
Approved by: sjg (mentor)
Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D2902
During gmirror startup, if component mirrors are found to be dirty as is
typical after a system crash, the mirrors are synchronized to the mirror
with highest priority. However if a gmirror starts without all of its
mirrors present, for example because of some transient delays during
tasting, the remaining mirrors must be synchronized before they may become
active.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Before this change it was impossible to set number of PKCS#5v2 iterations,
required to set passphrase, if it has two keys and never had any passphrase.
Due to present metadata format limitations there are still cases when number
of iterations can not be changed, but now it works in cases when it can.
PR: 218512
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D10338
At this point we have not rendezvous'ed with the mirror worker thread, and
I/O may still be in flight. Various I/O completion paths expect to be able
to obtain a reference to the mirror softc from the GEOM, so setting it to
NULL may result in various NULL pointer dereferences if the mirror is
stopped with -f or the kernel is shut down while a mirror is
synchronizing. The worker thread will clear the softc pointer before
exiting.
Tested by: pho
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
We are otherwise susceptible to a race with a concurrent teardown of the
mirror provider, causing the I/O to be left uncompleted after the mirror
started withering.
Tested by: pho
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Regular I/O requests may be blocked by concurrent synchronization requests
targeted to the same LBAs, in which case they are moved to a holding queue
until the conflicting I/O completes. We therefore want to stop
synchronization before completing pending I/O in g_mirror_destroy_provider()
since this ensures that blocked I/O requests are completed as well.
Tested by: pho
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Entries may be removed and freed if an I/O error occurs during mirror
synchronization, so we cannot assume that all entries of ds_bios are
valid.
Also ensure that a synchronization BIO's array index is preserved after
a successful write.
Reported and tested by: pho
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
This patch adds a general mechanism for providing encryption keys to the
kernel from the boot loader. This is intended to enable GELI support at
boot time, providing a better mechanism for passing keys to the kernel
than environment variables. It is designed to be extensible to other
applications, and can easily handle multiple encrypted volumes with
different keys.
This mechanism is currently used by the pending GELI EFI work.
Additionally, this mechanism can potentially be used to interface with
GRUB, opening up options for coreboot+GRUB configurations with completely
encrypted disks.
Another benefit over the existing system is that it does not require
re-deriving the user key from the password at each boot stage.
Most of this patch was written by Eric McCorkle. It was extended by
Allan Jude with a number of minor enhancements and extending the keybuf
feature into boot2.
GELI user keys are now derived once, in boot2, then passed to the loader,
which reuses the key, then passes it to the kernel, where the GELI module
destroys the keybuf after decrypting the volumes.
Submitted by: Eric McCorkle <eric@metricspace.net> (Original Version)
Reviewed by: oshogbo (earlier version), cem (earlier version)
MFC after: 3 weeks
Relnotes: yes
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D9575
In GELI, anywhere we are zeroing out possibly sensitive data, like
the metadata struct, the metadata sector (both contain the encrypted
master key), the user key, or the master key, use explicit_bzero.
Didn't touch the bzero() used to initialize structs.
Reviewed by: delphij, oshogbo
Sponsored by: ScaleEngine Inc.
Differential Revision: https://reviews.freebsd.org/D9809
A request may be queued while the queue lock is dropped when the mirror is
being destroyed. The corresponding wakeup would be lost, possibly resulting
in an apparent hang of the mirror worker thread.
Tested by: pho (part of a larger patch)
MFC after: 1 week
Sponsored by: Dell EMC Isilon
The worker thread will destroy the mirror provider as part of its teardown
sequence. The call made sense in the initial revision of gmirror, but
became unnecessary in r137248.
Tested by: pho (part of a larger diff)
MFC afteR: 2 weeks
Sponsored by: Dell EMC Isilon
- Don't execute any of g_mirror_shutdown_post_sync() when panicking. We
cannot safely idle the mirror or stop synchronization in that state, and
the current attempts to do so complicate debugging of gmirror itself.
- Check for a non-NULL panicstr instead of using SCHEDULER_STOPPED(). The
latter was added for use in the locking primitives.
Reviewed by: mav, pjd
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
gpart(8) has functionality to change the label of an GPT partition.
This functionality works like it should, however, after a label change
the /dev/gpt/ entries remain unchanged. glabel(8) status output remains
unchanged. The change only takes effect after a reboot.
PR: 162690
Submitted by: sub.mesa@gmail, Ben RUBSON <ben.rubson@gmail.com>, ae
Reviewed by: allanjude, bapt, bcr
MFC after: 6 weeks.
Differential Revision: https://reviews.freebsd.org/D9935
with geom_flashmap(4) and teach it about MMC for slicing enhanced
user data area partitions. The FDT slicer still is the default for
CFI, NAND and SPI flash on FDT-enabled platforms.
- In addition to a device_t, also pass the name of the GEOM provider
in question to the slicers as a single device may provide more than
provider.
- Build a geom_flashmap.ko.
- Use MODULE_VERSION() so other modules can depend on geom_flashmap(4).
- Remove redundant/superfluous GEOM routines that either do nothing
or provide/just call default GEOM (slice) functionality.
- Trim/adjust includes
Submitted by: jhibbits (RouterBoard bits)
Reviewed by: jhibbits
The PBKDF2 in sys/geom/eli/pkcs5v2.c is around half the speed it could be
GELI's PBKDF2 uses a simple benchmark to determine a number of iterations
that will takes approximately 2 seconds. The security provided is actually
half what is expected, because an attacker could use the optimized
algorithm to brute force the key in half the expected time.
With this change, all newly generated GELI keys will be approximately 2x
as strong. Previously generated keys will talk half as long to calculate,
resulting in faster mounting of encrypted volumes. Users may choose to
rekey, to generate a new key with the larger default number of iterations
using the geli(8) setkey command.
Security of existing data is not compromised, as ~1 second per brute force
attempt is still a very high threshold.
PR: 202365
Original Research: https://jbp.io/2015/08/11/pbkdf2-performance-matters/
Submitted by: Joe Pixton <jpixton@gmail.com> (Original Version), jmg (Later Version)
Reviewed by: ed, pjd, delphij
Approved by: secteam, pjd (maintainer)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D8236
Don't start switcher kproc until the first GEOM is created.
Reviewed by: pjd
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D8576
Some g_raid tasters attempt metadata reads in multiples of the provider
sectorsize. Reads larger than MAXPHYS are invalid, so detect and abort
in such situations.
Spiritually similar to r217305 / PR 147851.
PR: 214721
Sponsored by: Dell EMC Isilon
With clang 4.0.0, I'm getting the following warnings:
sys/geom/vinum/geom_vinum_state.c:186:7: error: logical not is only
applied to the left hand side of this bitwise operator
[-Werror,-Wlogical-not-parentheses]
if (!flags & GV_SETSTATE_FORCE)
^ ~
The logical not operator should obiously be called after masking.
Reviewed by: mav, pfg
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D9093
Changes include modifications in kernel crash dump routines, dumpon(8) and
savecore(8). A new tool called decryptcore(8) was added.
A new DIOCSKERNELDUMP I/O control was added to send a kernel crash dump
configuration in the diocskerneldump_arg structure to the kernel.
The old DIOCSKERNELDUMP I/O control was renamed to DIOCSKERNELDUMP_FREEBSD11 for
backward ABI compatibility.
dumpon(8) generates an one-time random symmetric key and encrypts it using
an RSA public key in capability mode. Currently only AES-256-CBC is supported
but EKCD was designed to implement support for other algorithms in the future.
The public key is chosen using the -k flag. The dumpon rc(8) script can do this
automatically during startup using the dumppubkey rc.conf(5) variable. Once the
keys are calculated dumpon sends them to the kernel via DIOCSKERNELDUMP I/O
control.
When the kernel receives the DIOCSKERNELDUMP I/O control it generates a random
IV and sets up the key schedule for the specified algorithm. Each time the
kernel tries to write a crash dump to the dump device, the IV is replaced by
a SHA-256 hash of the previous value. This is intended to make a possible
differential cryptanalysis harder since it is possible to write multiple crash
dumps without reboot by repeating the following commands:
# sysctl debug.kdb.enter=1
db> call doadump(0)
db> continue
# savecore
A kernel dump key consists of an algorithm identifier, an IV and an encrypted
symmetric key. The kernel dump key size is included in a kernel dump header.
The size is an unsigned 32-bit integer and it is aligned to a block size.
The header structure has 512 bytes to match the block size so it was required to
make a panic string 4 bytes shorter to add a new field to the header structure.
If the kernel dump key size in the header is nonzero it is assumed that the
kernel dump key is placed after the first header on the dump device and the core
dump is encrypted.
Separate functions were implemented to write the kernel dump header and the
kernel dump key as they need to be unencrypted. The dump_write function encrypts
data if the kernel was compiled with the EKCD option. Encrypted kernel textdumps
are not supported due to the way they are constructed which makes it impossible
to use the CBC mode for encryption. It should be also noted that textdumps don't
contain sensitive data by design as a user decides what information should be
dumped.
savecore(8) writes the kernel dump key to a key.# file if its size in the header
is nonzero. # is the number of the current core dump.
decryptcore(8) decrypts the core dump using a private RSA key and the kernel
dump key. This is performed by a child process in capability mode.
If the decryption was not successful the parent process removes a partially
decrypted core dump.
Description on how to encrypt crash dumps was added to the decryptcore(8),
dumpon(8), rc.conf(5) and savecore(8) manual pages.
EKCD was tested on amd64 using bhyve and i386, mipsel and sparc64 using QEMU.
The feature still has to be tested on arm and arm64 as it wasn't possible to run
FreeBSD due to the problems with QEMU emulation and lack of hardware.
Designed by: def, pjd
Reviewed by: cem, oshogbo, pjd
Partial review: delphij, emaste, jhb, kib
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4712
It is quite specific mode of operation without storing on-disk metadata.
It can be useful in some cases in combination with some external control
tools handling mirror creation and disks hot-plug.
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Upstream the BUF_TRACKING and FULL_BUF_TRACKING buffer debugging code.
This can be handy in tracking down what code touched hung bios and bufs
last. The full history is especially useful, but adds enough bloat that
it shouldn't be enabled in release builds.
Function names (or arbitrary string constants) are tracked in a
fixed-size ring in bufs. Bios gain a pointer to the upper buf for
tracking. SCSI CCBs gain a pointer to the upper bio for tracking.
Reviewed by: markj
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D8366
Normally gmirror allows colliding requests to proceed whenever a
synchronization request completes and advances to the next offset. However
if an I/O request collides with one of the final g_mirror_syncreqs, nothing
releases it once synchronization completes, resulting in an apparent I/O
hang. The same problem can occur if synchronization is aborted by an
I/O error. Therefore, be sure to requeue pending requests when
mirror synchronization is stopped for any reason.
While here, remove some dead code from g_mirror_regular_release().
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Introduce internal counter to track opens. Using provider's counters is
not very successfull after calling g_wither_provider().
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
When a syncid bump is pending, any write to the mirror results in the
updated syncid being written to each component's metadata block. However,
the update was only being performed after the writes to the mirror
componenents were queued. Instead, synchronously update the metadata block
first.
MFC after: 3 weeks
Sponsored by: Dell EMC Isilon
Consider a mirror with two components, m1 and m2. Suppose a hardware error
results in the removal of m2, with m1's genid bumped. Suppose further that
a replacement mirror component m3 is created and synchronized, after which
the system is shut down uncleanly. During a subsequent bootup, if gmirror
tastes m1 and m2 first, m2 will be removed from the mirror because it is
broken, but the mirror will be started without bumping the syncid on m1
because all elements of the mirror are accounted for. Then m3 will be
added to the already-running mirror with the same syncid as m1, so the
components will not be synchronized despite the unclean shutdown.
Handle this scenario by bumping the syncid of healthy components if any
broken mirrors are discovered during mirror startup.
MFC after: 3 weeks
Sponsored by: Dell EMC Isilon
The g_io_schedule_up() gets its "if" condition swapped to make
it more similar to g_io_schedule_down().
Suggested by: mav@
Reviewed by: mav@
MFC after: 1 month
Such errors can occur as the result of a write error or because the disk
backing the mirror element was removed. They result in a generation ID bump
on all active elements of the mirror, so we can safely disconnect the mirror
component rather than destroy it.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D7750
These are useful for testing changes to I/O error handling, and for
reproducing existing bugs in a controlled manner. The fail points are
g_mirror_regular_request_read
g_mirror_regular_request_write
g_mirror_sync_request_read
g_mirror_sync_request_write
g_mirror_metadata_write
They all effectively allow one to inject an error value into the bio_error
field of a corresponding BIO request as it is being completed.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
report the size only after first opening. And due to the events are
asynchronous, some consumers can receive this event too late and
this confuses them. This partially restores previous behaviour, and
at the same time this should fix the problem, when already opened
provider loses resize event.
PR: 211028
MFC after: 3 weeks
when it is being opened. This should fix the possible loss of a resize
event when disk capacity changed.
PR: 211028
Reported by: Dexuan Cui <decui at microsoft dot com>
MFC after: 3 weeks
superblock, allowing provider to be bit bigger, i.e. have some
extra padding after the FS image. That in some cases might be
a side-effect of using CLOOP format which enforces certain block
size and trying to compress image that is not exactly the number
of those blocks in size. The UFS itself does not have any issues
mounting such padded file systems, so it's what GEOM_LABEL should
do.
Submitted by: @mizhka_gmail.com
Differential Revision: https://reviews.freebsd.org/D6208
necessary because CLOOP format lacks explicit EOF or length, so that
in the presence of padding or when the CLOOP is put onto a larger
partition upper level provider size may be larger. Bound amount
of extra data that we might touch to the max length of the compressed
block and detect zero-padding in the last cluster, which when
sector is all-zero might cause us to emit bogus I/O error after
decompression of that fails. To not make code any more complicated
that it needs to be deal with it in lazy-manner, i.e. when we
first access that specific cluster.
This change also fixes stupid mistake in the LZMA code, inherited
from geom_lzma, which does not share length of the output buffer
buffer with the decompression routine, so that in the presence
of corrupted or purposedly tailored data may easily cause heap
overflow and kernel memory corruption.
Beef up validation of the CLOOP TOC by checking that lengths of
all but the last compressed clusters match upper limit set by
the decompressor and improve some error diagnostic output while
I am here.
2.Add kern.geom.uzip.attach_to tunable to artifically limit
attaching uzip to certain devices in the dev tree only.
For example the following only makes us attaching to the
GPT labels:
kern.geom.uzip.attach_to="gpt/*"
3.Add kern.geom.uzip.noattach_to, which does opposite to the (2)
above, i.e. prevents geom_uzip from tasting / attaching to
providers matching some pattern. By default we don't attach
to our own kind, i.e. kern.geom.uzip.noattach_to="*.uzip".
It saves us quite some CPU cycles, esp on low-end embedded
systems.
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D7013
The GEOM disk d_mtx is only acquired on disk creation and destruction.
It is a good candidate for replacement with a pool mutex. This eliminates
the mutex initialization and teardown and the mutex and name variables
themselves from struct disk.
sys/geom/geom_disk.h:
Take d_mtx and d_mtx_name out of struct disk.
sys/geom/geom_disk.c:
Use mtx_pool_lock() and mtx_pool_unlock() to guard the disk
initialization state instead of a dedicated mutex.
This allows removing the initialization and destruction of
d_mtx.
sys/sys/param.h:
Bump __FreeBSD_version to 1100119 for the change to struct disk.
Suggested by: jhb
Sponsored by: Spectra Logic
Approved by: re (gjb)
This will result in lock recursion and is more generally incorrect since
the completion handlers will just reinsert the BIOs into the queue we're
trying to drain.
Reviewed by: imp, ngie
Approved by: re (gjb)
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D6908
after the underlying device went away.
The problem was that callers who queue the GEOM resize provider
event didn't check to make sure that the provider had not been
withered. For the other equivalent case, g_new_provider_event(),
the code checks to see whether the provider has been withered
before queueing a g_new_provider_event() to the event thread.
In some cases, a resize provider event would come through after
the provider had been withered and all of the existing consumers
had been orphaned. When the resize event triggered a taste of
the provider, that would attach a new consumer to the now
withered provider. The wither washer (g_wither_washer() would
never be able to completely tear down the GEOM because of the
consumers that were hanging around.
The solution was to check the G_PF_WITHER provider flag before
queueing the g_resize_provider_event(), and add an assert to
g_resize_provider_event() to insure that it isn't called on a
withered provider.
sys/geom/geom_subr.c:
In g_resize_provider(), don't try to continue if the
G_PF_WITHER flag is set.
In g_resize_provider_event(), add an assert that the
G_PF_WITHER flag is not set.
In g_access(), if a provider has an error, print out the
name of the provider with the error.
Sponsored by: Spectra Logic
Approved by: re (marius)
MFC after: 3 days
device is gone.
The problem was that when disk_gone() is called, if the GEOM disk
creation process has not yet happened, the withering process
couldn't start.
We didn't record any state in the GEOM disk code, and so the d_gone()
callback to the da(4) driver never happened.
The solution is to track the state of the creation process, and
initiate the withering process from g_disk_create() if the disk is
being created.
This change does add fields to struct disk, and so I have bumped
DISK_VERSION.
geom_disk.c: Track where we are in the disk creation process,
and check to see whether our underlying disk has
gone away or not.
In disk_gone(), set a new d_goneflag variable that
g_disk_create() can check to see if it needs to
clean up the disk instance.
geom_disk.h: Add a mutex to struct disk (for internal use) disk
init level, and a gone flag.
Bump DISK_VERSION because the size of struct disk has
changed and fields have been added at the beginning.
Sponsored by: Spectra Logic
Approved by: re (marius)
otherwise the system will hang.
This is a temporarily least intrusive crutch to get certain panicing systems
dumping. The proper fix should question is g_mirror_destroy() should be called
on a panicing system at all.
Discussed with: mav
