Commit Graph

2241 Commits

Author SHA1 Message Date
Jung-uk Kim
9c40dcbe5f Make geli(8) buildable. 2018-09-19 07:08:04 +00:00
Conrad Meyer
1b0909d51a OpenCrypto: Convert sessions to opaque handles instead of integers
Track session objects in the framework, and pass handles between the
framework (OCF), consumers, and drivers.  Avoid redundancy and complexity in
individual drivers by allocating session memory in the framework and
providing it to drivers in ::newsession().

Session handles are no longer integers with information encoded in various
high bits.  Use of the CRYPTO_SESID2FOO() macros should be replaced with the
appropriate crypto_ses2foo() function on the opaque session handle.

Convert OCF drivers (in particular, cryptosoft, as well as myriad others) to
the opaque handle interface.  Discard existing session tracking as much as
possible (quick pass).  There may be additional code ripe for deletion.

Convert OCF consumers (ipsec, geom_eli, krb5, cryptodev) to handle-style
interface.  The conversion is largely mechnical.

The change is documented in crypto.9.

Inspired by
https://lists.freebsd.org/pipermail/freebsd-arch/2018-January/018835.html .

No objection from:	ae (ipsec portion)
Reported by:	jhb
2018-07-18 00:56:25 +00:00
Conrad Meyer
1df7f41560 OCF: Convert consumers to the session id typedef
These were missed in the earlier r336269.

No functional change.

Sponsored by:	Dell EMC Isilon
2018-07-16 19:01:05 +00:00
Mariusz Zaborski
78f79a9a08 Let geli deal with lost devices without crashing.
PR:		162036
Submitted by:	Fabian Keil <fk@fabiankeil.de>
Obtained from:	ElectroBSD
Discussed with: pjd@
2018-07-15 18:03:19 +00:00
Warner Losh
4bae19e9b8 g_eli_key_cmp is used only in the kernel, so only define it in the
kernel.
2018-07-13 18:21:38 +00:00
Mikolaj Golub
874774c5d4 geom_gate: enable resize
Reviewed By:	pjd
Approved By:	pjd
Differential Revision:	https://reviews.freebsd.org/D11531
2018-07-13 07:08:06 +00:00
Ed Maste
76db6c8773 gpart: add EFI alias for MBR partition scheme
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15870
2018-06-17 20:10:48 +00:00
Ed Maste
a0a8412b2a Sort geom/part mbr/ebr/ldm alias table entries
Having the table entries in alpha order simplifies future additions.

Sponsored by:	The FreeBSD Foundation
2018-06-17 20:06:27 +00:00
Mariusz Zaborski
31f7586d73 Introduce the 'n' flag for the geli attach command.
If the 'n' flag is provided the provided key number will be used to
decrypt device. This can be used combined with dryrun to verify if the key
is set correctly. This can be also used to determine which key slot we want to
change on already attached device.

Reviewed by:	allanjude
Differential Revision:	https://reviews.freebsd.org/D15309
2018-05-09 20:53:38 +00:00
Mark Johnston
bd92e6b6f5 Refactor some of the MI kernel dump code in preparation for netdump.
- Add clear_dumper() to complement set_dumper().
- Drain netdump's preallocated mbuf pool when clearing the dumper.
- Don't do bounds checking for dumpers with mediasize 0.
- Add dumper callbacks for initialization for writing out headers.

Reviewed by:	sbruno
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D15252
2018-05-06 00:22:38 +00:00
Mark Johnston
681554d70b Remove a redundant assertion.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2018-05-06 00:05:03 +00:00
Mark Johnston
40e805221b Avoid dropping the topology lock in gmirror's dumpconf implementation.
Doing so introduces races which can lead to a use-after-free when
grabbing a snapshot of the GEOM mesh.

To ensure that a mirror's disk list remains stable, change its locking
protocol: both the softc lock and the topology lock are now required
to modify the list, so either lock is sufficient for traversal.

Tested by:	pho
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-05-06 00:03:24 +00:00
Ed Maste
b525a10ac0 gpart: add fat32lba MBR partition type
FAT32 partition with LBA addressing.

Reviewed by:	marcel
MFC after:	3 days
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D15266
2018-05-04 00:34:27 +00:00
Kyle Evans
74d6c131cb Annotate geom modules with MODULE_VERSION
GEOM ELI may double ask the password during boot. Once at loader time, and
once at init time.

This happens due a module loading bug. By default GEOM ELI caches the
password in the kernel, but without the MODULE_VERSION annotation, the
kernel loads over the kernel module, even if the GEOM ELI was compiled into
the kernel. In this case, the newly loaded module
purges/invalidates/overwrites the GEOM ELI's password cache, which causes
the double asking.

MFC Note: There's a pc98 component to the original submission that is
omitted here due to pc98 removal in head. This part will need to be revived
upon MFC.

Reviewed by:	imp
Submitted by:	op
Obtained from:	opBSD
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D14992
2018-04-10 19:18:16 +00:00
Mariusz Zaborski
8f1c45c20a Introduce dry run option for attaching the device.
This will allow us to verify if passphrase and key is valid without
decrypting whole device.

Reviewed by:	cem@, allanjude@
Differential Revision:	https://reviews.freebsd.org/D15000
2018-04-10 13:22:48 +00:00
Kyle Evans
2967ace894 Retire the geom_aes class
It's had a good life, but it's not really configurable and not really used.

Obtained from:	opBSD (with some changes)
Differential Revision:	https://reviews.freebsd.org/D14991
2018-04-09 17:30:30 +00:00
Brooks Davis
6469bdcdb6 Move most of the contents of opt_compat.h to opt_global.h.
opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c.  A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by:	kib, cem, jhb, jtl
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14941
2018-04-06 17:35:35 +00:00
Sean Bruno
2c385d51ce Squash error from geom by sizing ident strings to DISK_IDENT_SIZE.
Display attribute in future error strings and differentiate g_handleattr()
error messages for ease of debugging in the future.

"g_handleattr: md1 bio_length 24 strlen 31 -> EFAULT"

Reported by:	swills
Reviewed by:	imp cem avg
Sponsored by:	Limelight Networks
Differential Revision:	https://reviews.freebsd.org/D14962
2018-04-05 13:56:40 +00:00
Kirk McKusick
fb15890a8c When freeing a superblock returned by ffs_sbget, be sure to also
free the superblock summary information.

Reported by: Peter Holm (pho@)
Tested by: Peter Holm (pho@)
2018-03-24 15:36:25 +00:00
Mariusz Zaborski
9ea857cf0f Remove unneeded variable which was introduced in r328472.
Pointed out by:	pjd@
2018-03-18 15:09:55 +00:00
Andriy Gapon
aca41af247 g_access: deal with races created by geoms that drop the topology lock
The problem is that g_access() must be called with the GEOM topology
lock held.  And that gives a false impression that the lock is indeed
held across the call.  But this isn't always true because many classes,
ZVOL being one of the many, need to drop the lock.  It's either to
perform an I/O on the first open or to acquire a different lock (like in
g_mirror_access).

That, of course, can break many assumptions.  For example,
g_slice_access() adds an extra exclusive count on the first open. As
described above, an underlying geom may drop the topology lock and that
would open a race with another thread that would also request another
extra exclusive count.  In general, two consumers may be granted
incompatible accesses.

To avoid this problem the code is changed to mark a geom with special
flag before calling its access method and clear the flag afterwards.  If
another thread sees that flag, then it means that the topology lock has
been dropped (either by the geom in question or downstream from it), so
it is not safe to make another access call.  So, the second thread would
use g_topology_sleep() to wait until the flag is cleared and only then
would it proceed with the access.

Also see http://docs.freebsd.org/cgi/mid.cgi?809d9254-ee56-59d8-69a4-08838e985cea

PR:		225960
Reported by:	asomers
Reviewed by:	markj, mav
MFC after:	3 weeks
Differential Revision: https://reviews.freebsd.org/D14533
2018-03-15 09:16:10 +00:00
Conrad Meyer
ee4d316fe7 g_part_gpt: Fix memory leak in error path
If g_part_gpt_read() encountered a disk with bad primary and secondary
tables, it could leak memory.

Reported by:	Coverity
Sponsored by:	Dell EMC Isilon
2018-03-07 01:55:50 +00:00
Conrad Meyer
90575a0ec9 g_label_ufs: Fix typo from r330264
Reported by:	O. Hartmann <o.hartmann AT walstatt.org>
Sponsored by:	Dell EMC Isilon
2018-03-02 06:02:54 +00:00
Kirk McKusick
efbf396426 This change is some refactoring of Mark Johnston's changes in r329375
to fix the memory leak that I introduced in r328426. Instead of
trying to clear up the possible memory leak in all the clients, I
ensure that it gets cleaned up in the source (e.g., ffs_sbget ensures
that memory is always freed if it returns an error).

The original change in r328426 was a bit sparse in its description.
So I am expanding on its description here (thanks cem@ and rgrimes@
for your encouragement for my longer commit messages).

In preparation for adding check hashing to superblocks, r328426 is
a refactoring of the code to get the reading/writing of the superblock
into one place. Unlike the cylinder group reading/writing which
ends up in two places (ffs_getcg/ffs_geom_strategy in the kernel
and cgget/cgput in libufs), I have the core superblock functions
just in the kernel (ffs_sbfetch/ffs_sbput in ffs_subr.c which is
already imported into utilities like fsck_ffs as well as libufs to
implement sbget/sbput). The ffs_sbfetch and ffs_sbput functions
take a function pointer to do the actual I/O for which there are
four variants:

    ffs_use_bread / ffs_use_bwrite for the in-kernel filesystem

    g_use_g_read_data / g_use_g_write_data for kernel geom clients

    ufs_use_sa_read for the standalone code (stand/libsa/ufs.c
	but not stand/libsa/ufsread.c which is size constrained)

    use_pread / use_pwrite for libufs

Uses of these interfaces are in the UFS filesystem, geoms journal &
label, libsa changes, and libufs. They also permeate out into the
filesystem utilities fsck_ffs, newfs, growfs, clri, dump, quotacheck,
fsirand, fstyp, and quot. Some of these utilities should probably be
converted to directly use libufs (like dumpfs was for example), but
there does not seem to be much win in doing so.

Tested by: Peter Holm (pho@)
2018-03-02 04:34:53 +00:00
Mark Johnston
16759360d4 Fix a memory leak introduced in r328426.
ffs_sbget() may return a superblock buffer even if it fails, so the
caller must be prepared to free it in this case. Moreover, when tasting
alternate superblock locations in a loop, ffs_sbget()'s readfunc
callback must free the previously allocated buffer.

Reported and tested by:	pho
Reviewed by:		kib (previous version)
Differential Revision:	https://reviews.freebsd.org/D14390
2018-02-16 15:41:03 +00:00
Alan Somers
834063202a gpart: append partition name to the underlying provider's physical path
If the underlying provider's physical path is null, then the gpart device's
physical path will be, too. Otherwise, it will append the partition name,
such as "/p1" or "/s1/a". This will make gpart work better with zfsd(8).

PR:		224965
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D14010
2018-02-14 20:26:09 +00:00
Alan Somers
0bab7fa8a7 geli: append "/eli" to the underlying provider's physical path
If the underlying provider's physical path is null, then the geli device's
physical path will be, too. Otherwise, it will append "/eli".  This will make
geli work better with zfsd(8).

PR:		224962
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D13979
2018-02-14 20:15:32 +00:00
Justin Hibbits
d793587fe2 Fix a panic introduced in r329225
Some GEOM partition tables may be destroyed with incomplete partition
entries.  Guard against this with NULL checks.

Reported by:	pholm,others
Reviewed by:	markj
Tested by:	pholm
2018-02-14 15:12:09 +00:00
Justin Hibbits
08a3b42fdb Narrow a race, and fix a leak, in g_part_wither
A race in g_part_wither() can lead to I/O being performed with a freed GEOM
when the device disappears.  Close the race as best as we can for now,
following the code patterns from g_part_ctl_destroy() and g_part_ctl_undo().
This also fixes a leak, as g_wither_geom() does not wither providers, it
only orphans them, so the partition entries would never get destroyed in
g_wither_washer().

Note, this is not a complete fix, it can still race with g_part_start(), the
race has merely been narrowed.

Reviewed by:	markj
Sponsored by:	Dell EMC Isilon
2018-02-13 17:40:09 +00:00
Conrad Meyer
b42712a8b7 Add GUID and alias for Apple APFS partition
PR:		225813
Submitted by:	James Wright <james.wright AT jigsawdezign.com>
2018-02-11 06:57:20 +00:00
Mark Johnston
0d02f6c201 Simplify synchronization read error handling.
Since synchronization reads are performed by submitting a request to
the external mirror provider, we know that the request returns with an
error only when gmirror was unable to read a copy of the block from any
mirror. Thus, there is no need to retry the request from the
synchronization error handler.

Tested by:	pho
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-02-06 16:02:33 +00:00
Alan Somers
f5b4099e6b geom: don't write stack garbage in disk labels
Most consumers of g_metadata_store were passing in partially unallocated
memory, resulting in stack garbage being written to disk labels. Fix them by
zeroing the memory first.

gvirstor repeated the same mistake, but in the kernel.

Also, glabel's label contained a fixed-size string that wasn't
initialized to zero.

PR:		222077
Reported by:	Maxim Khitrov <max@mxcrypt.com>
Reviewed by:	cem
MFC after:	3 weeks
X-MFC-With:	323314
X-MFC-With:	323338
Differential Revision:	https://reviews.freebsd.org/D14164
2018-02-04 14:49:55 +00:00
Xin LI
90a48fba23 After r328426, g_label depends on UFS (option FFS) code to read UFS
superblock, and the kernel will fail to link when UFS is not built
in.  This commit makes it depend on a small portion of FFS bits and
thereby fixes build for this situation.

This is intended as an interim bandaid, and the actual superblock
reading code should probably be made independent of UFS, so we do
not need to depend on it (see kib@'s comment in the review for
details), and we will revisit this once the superblock check hashes
are all in place.

Differential Revision:	https://reviews.freebsd.org/D14092
2018-02-03 09:15:13 +00:00
Kirk McKusick
5d84ae8b49 Null out journal softc pointer earlier to avoid a segment fault
that can otherwise occur.

PR:           221804
Submitted by: Andreas Longwitz <longwitz at incore.de>
MFC after:    1 week
2018-01-31 23:30:49 +00:00
Mariusz Zaborski
0fc4adbe06 Don't truncate name of glabel.
If it's to long just report that.

Reviewed by:	trasz@
Differential Revision:	https://reviews.freebsd.org/D13746
2018-01-27 12:28:52 +00:00
Kirk McKusick
dffce2150e Refactoring of reading and writing of the UFS/FFS superblock.
Specifically reading is done if ffs_sbget() and writing is done
in ffs_sbput(). These functions are exported to libufs via the
sbget() and sbput() functions which then used in the various
filesystem utilities. This work is in preparation for adding
subperblock check hashes.

No functional change intended.

Reviewed by: kib
2018-01-26 00:58:32 +00:00
Pedro F. Giffuni
ac2fffa4b7 Revert r327828, r327949, r327953, r328016-r328026, r328041:
Uses of mallocarray(9).

The use of mallocarray(9) has rocketed the required swap to build FreeBSD.
This is likely caused by the allocation size attributes which put extra pressure
on the compiler.

Given that most of these checks are superfluous we have to choose better
where to use mallocarray(9). We still have more uses of mallocarray(9) but
hopefully this is enough to bring swap usage to a reasonable level.

Reported by:	wosch
PR:		225197
2018-01-21 15:42:36 +00:00
Alan Somers
6f7f85e0e1 gnop(8): add the ability to set a nop provider's physical path
While I'm here, expand the existing tests a bit.

MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D13579
2018-01-18 05:57:10 +00:00
Pedro F. Giffuni
0cee6dcdb0 misc geom and gnu: make some use of mallocarray(9).
Focus on code where we are doing multiplications within malloc(9). None of
these ire likely to overflow, however the change is still useful as some
static checkers can benefit from the allocation attributes we use for
mallocarray.

This initial sweep only covers malloc(9) calls with M_NOWAIT. No good
reason but I started doing the changes before r327796 and at that time it
was convenient to make sure the sorrounding code could handle NULL values.

Differential revision: https://reviews.freebsd.org/D13837
2018-01-15 21:23:16 +00:00
Andriy Gapon
6ce374aa94 geom_disk / scsi_da: deny opening write-protected disks for writing
Ths change consists of two parts.

geom_disk: deny opening a disk for writing if it's marked as
write-protected.  A new disk(9) flag is added to mark write protected
disks.  A possible alternative could be to add another parameter to d_open,
so that the open mode could be passed to it and the disk drivers could
make the decision internally, but the flag required less churn.

scsi_da: add a new phase of disk probing to query the all pages mode
sense page.  We can determine if the disk is write protected using bit 7
of the device specific field in the mode parameter header returned by
MODE SENSE.

PR:		224037
Reviewed by:	mav
MFC after:	4 weeks
Differential Revision: https://reviews.freebsd.org/D13360
2018-01-15 11:20:00 +00:00
Mark Johnston
762f440f15 Fix handling of read errors during mirror synchronization.
We would previously just free the request BIO, which would either cause
the disk to stay stuck in the SYNCHRONIZING state, or result in
synchronization completing without having copied the block which
returned an error.

With this change, if the disk which returned an error is the only active
disk in the mirror, the synchronizing disk is kicked out. Otherwise, the
read is retried.

Reported and tested by:	pho (previous version)
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2018-01-10 19:37:21 +00:00
Mark Johnston
792f0c3b09 Clarify the use of the gmirror flag mask constants.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2018-01-10 15:21:36 +00:00
Mark Johnston
aed882a9fb Avoid referencing a possibly freed consumer after r327496.
g_mirror_regular_request() may free the gmirror consumer for a disk
if that disk is being disconnected, after which we must not dereference
the consumer pointer.

CID:		1384280
X-MFC with:	r327496
2018-01-10 05:06:21 +00:00
Mark Johnston
8b0a00b745 Sort and remove unneeded includes.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2018-01-08 15:56:40 +00:00
Mark Johnston
7653e6d781 Release the queue lock before restarting the worker loop.
Reported and tested by:	pho
MFC after:	3 days
Sponsored by:	Dell EMC Isilon
2018-01-08 15:41:49 +00:00
Mark Johnston
1787c3feb4 Fix some I/O ordering issues in gmirror.
- BIO_FLUSH requests were dispatched to the disks directly from
  g_mirror_start() rather than going through the mirror's I/O request
  queue, so they could have been reordered with preceding writes.
  Address this by processing such requests from the queue, avoiding
  direct dispatch.
- Handling for collisions with synchronization requests was too
  fine-grained and could cause reordering of writes. In particular,
  BIO_ORDERED was not being honoured. Address this by effectively
  freezing the request queue any time a collision with a synchronization
  request occurs. The queue is unfrozen once the collision with the
  first frozen request is over.
- The above-mentioned collision handling allowed reads to jump ahead
  of writes to the same offset. Address this by freezing all request
  types when a collision occurs, not just BIO_WRITEs and BIO_DELETEs.

Also add some more fail points for use in testing error handling.

Reviewed by:	imp
MFC after:	3 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13559
2018-01-02 18:11:54 +00:00
Colin Percival
8b8a7c43a9 Instrument "boot holds" for the benefit of the TSLOG framework. These
are places where the "main thread" of the booting kernel (either the
thread which later becomes swapper or the thread which later becomes
init) has to stop and wait for action to take place in another thread
before continuing.

There are currently three such holds:
1. The intr_config_hooks SYSINIT waits for hooks registered via the
config_intrhook_establish function; this allows (typically) devices
which need interrupts enabled to complete their initialization to do
so before root is mounted.

2. The g_waitidle function waits for the GEOM event queue to be empty;
this ensures that all of the disks which have been attached have been
tasted before we attempt to mount root.

3. The vfs_mountroot_wait function (in addition to calling g_waitidle)
waits for holds registered via root_mount_hold; among other things, this
is used by the USB subsystem to ensure that we don't fail to mount root
if it's located on a USB disk which takes a while to probe.
2017-12-31 09:23:52 +00:00
Pedro F. Giffuni
2afb21f309 geom_ccd.c: Fix the licenses properly
The license merging in r109471 didn't take into account that licensing
could change. Just removing the 3rd clause obviates the copyright
assignment to the NetBSD Foundation.

We do have plenty of files that have two or more licensing as in this
case, so fix this properly by splitting back the licenses as they are
upstream.

Obtained from:	NetBSD
2017-12-30 02:07:18 +00:00
Pedro F. Giffuni
68689f580b geom_ccd.c: Update the license with changes from upstream.
Part of this file originated in NetBSD, with the original file
carrying two versions of 4-clause BSD licenses. r109471 attempted to
simplify the situation by putting both licenses together.

Meanwhile, NetBSD dropped Clauses 3 and 4 from their own license, and
eventually NetBSD got permission from the University of Utah to drop the
3rd clause.

Keep the license "simple" by dropping the third clause since both TNF,
Utah/Berkeley and phk agree in principle that it can be dropped.

Obtained from:	NetBSD (ccd.c CVS 1.128, 1.138)
2017-12-30 01:37:08 +00:00
Alexander Kabaev
151ba7933a Do pass removing some write-only variables from the kernel.
This reduces noise when kernel is compiled by newer GCC versions,
such as one used by external toolchain ports.

Reviewed by: kib, andrew(sys/arm and sys/arm64), emaste(partial), erj(partial)
Reviewed by: jhb (sys/dev/pci/* sys/kern/vfs_aio.c and sys/kern/kern_synch.c)
Differential Revision: https://reviews.freebsd.org/D10385
2017-12-25 04:48:39 +00:00
Mark Johnston
9abe2e7e98 Avoid using bioq_* in gmirror.
gmirror does not perform any sorting of I/O requests, so the bioq API
doesn't provide any advantages over plain TAILQs. The API also does not
provide operations needed by an upcoming change.

No functional change intended. The diff shrinks the geom_mirror.ko
text and the gmirror softc slightly.

Tested by:	pho (part of a larger patch)
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-12-19 17:13:04 +00:00
Mark Johnston
68eadcec0f Give a couple of predication functions a bool return type.
No functional change intended.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-12-15 19:14:21 +00:00
Mark Johnston
204d94f161 Typo.
MFC after:	1 week
2017-12-15 19:03:03 +00:00
Mark Johnston
8b93770503 Address a possible lost wakeup for gmirror events.
g_mirror_event_send() acquires the I/O queue lock to deliver a wakeup
to the worker thread, and this is done after enqueuing the event.
So it's sufficient to check the event queue before atomically releasing
the queue lock and going to sleep.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-12-12 17:29:34 +00:00
Mark Johnston
b634781eac Give g_mirror_event_get() a more accurate name.
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-12-12 17:25:25 +00:00
Mark Johnston
a3584ee355 Decrement sc_writes when BIO_DELETE requests complete.
Otherwise a gmirror that has received a BIO_DELETE request will never be
marked clean (unless sc_writes overflows).

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-12-12 17:24:30 +00:00
Eugene Grosbein
1b8ea9beff geom_raid (RAID5): do not lose bp->bio_error, keep it in pbp->bio_error
and return it by passing to g_raid_iodone()

Approved by:	mav (mentor)
MFC after:	3 days
2017-12-07 20:09:17 +00:00
Eugene Grosbein
01a51aba38 Fix use-after-free that sometimes results in a garbage returned
instead of right error code after requests to SINGLE/CONCAT volumes, f.e:

# dd if=/dev/raid/r0 bs=512 of=/dev/null
dd: /dev/raid/r0: Unknown error: -559038242

Reviewed by:	avg (mentor), mav (mentor)
MFC after:	3 days
2017-12-07 05:55:18 +00:00
Warner Losh
700b6f8e23 When building standalone, include stand.h rather than the kernel
includes or the userland includes.

Sponsored by: Netflix
2017-12-05 21:37:32 +00:00
Warner Losh
3a7d67e741 We don't need both _STAND and _STANDALONE. There's more places that
use _STANDALONE, so change the former to the latter.

Sponsored by: Netflix
2017-12-02 00:07:09 +00:00
Mark Johnston
2ceafb776e Update gmirror metadata less frequently when synchronizing.
We periodically record synchronization progress in the metadata
block of the disk being synchronized; this allows an interrupted
synchronization to be resumed. However, the frequency of these
updates heavily pessimized synchronization time on some media. This
change modifies gmirror to update metadata based on a time period,
and adds a sysctl to control that period. The default value results
in a much lower update frequency and increases the completion time
for an interrupted rebuild only marginally.

Reported by:	Andre Albsmeier <andre@fbsd.e4m.org>
MFC after:	3 weeks
2017-11-30 20:36:29 +00:00
Pedro F. Giffuni
3728855a0f sys/geom: adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:17:37 +00:00
Mark Johnston
0349817103 Allow kern.geom.mirror.debug to be negative.
A negative value can be used to suppress all prints from the gmirror
kernel code, which can be useful when attempting to trigger race
conditions using stress tests.

MFC after:	1 week
2017-11-23 14:07:52 +00:00
Warner Losh
fdcf0c7477 While the EFI spec allows numbers to be in many forms, libefivar
produces hex numbers for the dsn. Since that come is from EDK2, change
this for symmetry, by generating the dsn as a hex number.

Noticed by: gpart list | grep efimedia | awk -F: '{print $2;}' | \
	sed -e 's/^ *//g;s/,,/,/' | grep MBR | efidp -p | efidp -f
Sponsored by: Netflix
2017-11-21 06:12:21 +00:00
Warner Losh
2ab9683565 Remove trailing whitespace (one I just introduced and a bunch of
others in the same directory).

Sponsored by: Netflix
2017-11-21 05:42:13 +00:00
Warner Losh
d65b6588d6 Implement efi media tagging for MBR partitioning types.
Sponsored by: Netflix
2017-11-21 05:35:21 +00:00
Pedro F. Giffuni
df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Andriy Gapon
f0fa2af656 geom_slice: fix r325227, protect against multiple calls to g_slice_free
This geom does not immediately detach its consumer relying on the
wither-washer to do that.  Since that happens asynchronously we may get
additional spoiling events.  So, we need to account for that.

There are multiple options for fixing this issue like detaching
immediately or checking for G_CF_ORPHAN in g_slice_spoiled().
The most reliable and least intrusive fix seems to be setting
geom->softc to NULL on the first call and checking for NULL on
subsequent calls.  This is something that the code did before r325227.

Reported by:	David Wolfskill <david@catwhisker.org>,
		O. Hartmann <o.hartmann@walstatt.org>
Tested by:	David Wolfskill <david@catwhisker.org> (earlier version)
Discussed with:	mav
MFC after:	1 week
X-MFC with:	r325227
2017-11-01 10:53:10 +00:00
Andriy Gapon
9662d80af5 geom_slice: do not destroy softc until providers are gone
At present, g_slice_orphan and g_slice_spoiled destroy the softc
(struct g_slicer) even before calling g_wither_geom, so there can
be active and incoming io requests at that time and g_slice_start
can access the softc.

This commit changes the code to destroy the softc only after all
providers are closed.

While there, a couple of small cleanups.

Reported by:	Ben RUBSON <ben.rubson@gmail.com>
Tested by:	Ben RUBSON <ben.rubson@gmail.com>
Reviewed by:	mav, smh (earlier version)
MFC after:	2 weeks
Sponsored by:	Panzura
Differential Revision: https://reviews.freebsd.org/D12809
2017-10-31 10:10:13 +00:00
Edward Tomasz Napierala
338ed98ad2 Add back missing MTX_DEF, it still needs to be there.
(Although it's defined to be 0, so there's no functional change.)

Reported by:	glebius
MFC after:	2 weeks
2017-10-29 12:03:06 +00:00
Mark Johnston
cef5abd140 Fix a lock leak in g_mirror_destroy().
g_mirror_destroy() is supposed to unlock the softc before indicating
success, but it wasn't doing so if the caller raced with another
thread destroying the mirror.

MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-10-27 17:05:14 +00:00
Edward Tomasz Napierala
0d73fface2 Make gmountver(8) use direct dispatch.
MFC after:	2 weeks
2017-10-26 10:18:31 +00:00
Edward Tomasz Napierala
0a8cfed8cf Make gmountver(8) use G_PF_ACCEPT_UNMAPPED.
MFC after:	2 weeks
2017-10-26 09:29:35 +00:00
Mark Johnston
64a16434d8 Add support for compressed kernel dumps.
When using a kernel built with the GZIO config option, dumpon -z can be
used to configure gzip compression using the in-kernel copy of zlib.
This is useful on systems with large amounts of RAM, which require a
correspondingly large dump device. Recovery of compressed dumps is also
faster since fewer bytes need to be copied from the dump device.

Because we have no way of knowing the final size of a compressed dump
until it is written, the kernel will always attempt to dump when
compression is configured, regardless of the dump device size. If the
dump is aborted because we run out of space, an error is reported on
the console.

savecore(8) is modified to handle compressed dumps and save them to
vmcore.<index>.gz, as it does when given the -z option.

A new rc.conf variable, dumpon_flags, is added. Its value is added to
the boot-time dumpon(8) invocation that occurs when a dump device is
configured in rc.conf.

Reviewed by:	cem (earlier version)
Discussed with:	def, rgrimes
Relnotes:	yes
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11723
2017-10-25 00:51:00 +00:00
Alan Somers
27f0f2ec5f Display rotation rate and TRIM/UNMAP support in diskinfo(8)
Bump __FreeBSD_version due to the expansion of struct diocgattr_arg.

Reviewed by:	mav, allanjude, imp
MFC after:	3 weeks
Relnotes:	yes
Sponsored by:	Spectra Logic Corp
Differential Revision:	https://reviews.freebsd.org/D12578
2017-10-04 15:09:49 +00:00
Edward Tomasz Napierala
b73e1f746a Don't destroy gmountver(8) devices on shutdown, unless they are orphaned.
Otherwise we would fail to sync the filesystem on reboot.

MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
2017-10-04 12:25:39 +00:00
Edward Tomasz Napierala
2b4490a5d3 Clear G_CF_ORPHAN when attaching. This fixes cases where the same
GEOM consumer can be orphaned, and then reattach to another provider.

From a user point of view, this makes gmountver(4) work again.

Reviewed by:	avg, mav
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D12228
2017-10-02 11:57:00 +00:00
Conrad Meyer
a523de2365 g_resize_provider_event: Do not invoke orphan method twice
Like r266444, g_resize_provider_event can attempt to orphan an already
orphaned geom_dev consumer.  This will cause a panic in g_dev_orphan.  Apply
the same fix as was applied to g_orphan_register.

Reviewed by:	ae
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12469
2017-09-24 19:59:26 +00:00
Andriy Gapon
7103ac8ad6 gmirror: treat ENXIO as disk disconnect, not media error
In theory, all data access errors mean that a member is out of sync
at most.  But they were treated as more serious errors to avoid the
situation where a flaky disk gets repeatedly disconnected, re-synchronized,
reconnected and then disconnected again.

ENXIO is a special error that means that the member disk disappeared,
so it should get the same handling as the GEOM orphaning event.
There is a better chance that when the disk is reconnected, it will be
a good member again.

When ENXIO happens on a read we use the exisiting G_MIRROR_BUMP_SYNCID
mechanism which means that the mirror's syncid is increased as soon
as there is a write to the mirror.  That's because no data has got out
of sync yet, but the problematic memeber is disconnected, so the future
write will make it stale.

When ENXIO happens on a write we use a new G_MIRROR_BUMP_SYNCID_NOW
mechanism which means that we update the mirror metadata as soon as
possible because the problematic memeber is already behind.

Reviewed by:	markj, imp
MFC after:	3 weeks
Differential Revision: https://reviews.freebsd.org/D9463
2017-09-15 13:57:08 +00:00
Conrad Meyer
ea5eee641e Fix information leak in geli(8) integrity mode
In integrity mode, a larger logical sector (e.g., 4096 bytes) spans several
physical sectors (e.g., 512 bytes) on the backing device.  Due to hash
overhead, a 4096 byte logical sector takes 8.5625 512-byte physical sectors.
This means that only 288 bytes (256 data + 32 hash) of the last 512 byte
sector are used.

The memory allocation used to store the encrypted data to be written to the
physical sectors comes from malloc(9) and does not use M_ZERO.

Previously, nothing initialized the final physical sector backing each
logical sector, aside from the hash + encrypted data portion.  So 224 bytes
of kernel heap memory was leaked to every block :-(.

This patch addresses the issue by initializing the trailing portion of the
physical sector in every logical sector to zeros before use.  A much simpler
but higher overhead fix would be to tag the entire allocation M_ZERO.

PR:		222077
Reported by:	Maxim Khitrov <max AT mxcrypt.com>
Reviewed by:	emaste
Security:	yes
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12272
2017-09-09 01:41:01 +00:00
Warner Losh
a905e3962c The hard drive media device path contains the size of the partition,
not its end. This makes the GEOM efimedia attribute match the
FreeBSD:Boot1Device environment variable now.

Sponsored by: Netflix
2017-09-02 07:04:06 +00:00
Warner Losh
ab4effdc68 Add efimedia attribute for all GPT partitions.
Sposnored by: Netflix
Differential Revision: https://reviews.freebsd.org/D12206
2017-09-01 17:55:25 +00:00
Konstantin Belousov
58d8f357c7 Let g_access() log the actual error number.
Submitted by:	 Fabian Keil <fk@fabiankeil.de>
PR:	221855
MFC after:	1 week
2017-08-27 12:24:25 +00:00
Mariusz Zaborski
3453dc72ad Hide length of geli passphrase during boot.
Introduce additional flag to the geli which allows to restore previous
behavior.

Reviewed by:	AllanJude@, cem@ (previous version)
MFC:		1 month
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D11751
2017-08-26 14:07:24 +00:00
Kirk McKusick
037331ddbd When read requests are sent from a filesystem running above g_journal,
the g_journal level needs to check whether it is holding a newer
copy of the block than that which exists on the disk. If so, it
needs to return its copy. If not, it should pass the request down
to the disk to fulfill. It currently considers six queues:

0) delayed queue,
1) unsent (current queue),
2) in-flight to the journal (flush queue),
3) active journal (active queue),
4) inactive journal (inactive queue), and
5) inflight to the disk (copy queue).

Checking on two of these queues is unnecessary:

0) The delayed requests should not be used for reads because they
   have not yet been entered into the journal, so their value should
   reflect the disk contents, not the future contents that are not
   yet committed.

2) Because all the bio's in the flush queue are also found on the
   active queue, there is no need to inspect the flush queue for
   reads since they will be found when searching the active queue.

Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Discussed with: kib
MFC after: 1 week
2017-08-13 18:09:22 +00:00
Kirk McKusick
8fccf8ffd7 Eliminate a variable that is only ever set.
Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Discussed with: kib
MFC after: 1 week
2017-08-13 18:06:38 +00:00
Warner Losh
0038725697 Also provide a warning for geom_fox.
Differential Review: https://reviews.freebsd.org/D11935
Requested by: jhb@
MFC After: 3 days
2017-08-09 16:37:37 +00:00
Warner Losh
20995eab57 Mark geom classes as deprecated.
geom_bsd, geom_mbr and geom_sunlabel have been obsolete since Marcel
Moolenaar's geom_part was in FreeBSD 7. They haven't been in GENERIC
since FreeBSD 8. Add warning when used.

geom_vol_ffs has been obsolete since ufs support to geom_label was
committed in FreeBSD 5. It hasn't been in GENERIC since FreeBSD 5.
Add warning when used.

geom_fox has been obsolete since gmultipath was committed in FreeBSD 7.
(no warning added, since this is a very obscure class).

These will all be removed in FreeBSD 12.

MFC After: 3 days
Differential Revision: https://reviews.freebsd.org/D11935

Note: Classes will be removed after MFC
2017-08-09 16:15:24 +00:00
Warner Losh
36d6e01474 Eliminate useless adjustments of aliased device.
No need to set any fields in the cloned device. devfs uses symlinks,
so the adev entries returned won't be presented to the drivers. Since
we don't save copies, nothing else will see them. This code came from
the old compat code, and it appears to be obsolete or never needed.

Submitted by: kib@
Differential Review: https://reviews.freebsd.org/D11919
2017-08-07 22:42:46 +00:00
Warner Losh
d3517d306c Expose API to allow disks to ask for alias names in devfs.
Implement disk_add_alias to allow aliases to be added to disks. All
disk have a primary name (say "foo") can also have secondary names
(say "bar") such that all instances of "foo" also have a "bar"
alias. So if you have foo0, foo0p1, foo1, foo1s1 and foo1s1a nodes
created by the foo driver and gpart, device nodes bar0, bar0p1, bar1,
bar1s1 and bar1s1a will appear as symlinks back to the original nodes.
This generalizes to multiple aliases. However, since the unit number
follows the primary name, multiple device drivers can't create the
same aliases unless those drives coorinate the unit number space (eg
you couldn't add an alias 'disk' to both 'da' and 'ada' because it's
possible to have da0 and ada0, because 'disk0' is ambiguous).

Differential Revision: https://reviews.freebsd.org/D11873
2017-08-07 21:12:38 +00:00
Warner Losh
5d7d13290a Add alias support to gpart.
When we're creating new providers for each of the partitions, add
aliases to the geom before we create the provider so when geom_dev
tastes the provider, the aliases are in place so the proper /dev
entries are created. So foo5p6 gets created as an alias for bar5p6
when foo is an alias for bar in the geom we're partitioning with
g_part. This also copies aliases from the container geom (eg disk) to
the label geom (the disk with GPT partitioning) so that aliases nest
properly.

Differential Revision: https://reviews.freebsd.org/D11873
2017-08-07 21:12:33 +00:00
Warner Losh
c624eb2598 Add aliasing concept to geom.
Add an alias name list to geoms. Use them in geom_dev to create
aliases. Previously, geom_dev would create an device node for the name
of the geom. Now, additional nodes are created pointing back to the
primary node with make_dev_alias_p. Aliases must be in place on the
geom before any tasting occurs.

Differential Revision: https://reviews.freebsd.org/D11873
2017-08-07 21:12:28 +00:00
Kirk McKusick
6c6118b390 gjournal is broken in handling its flush_queue. If we have 10 bio's
in the flush_queue:
         1 2 3 4 5 6 7 8 9 10
and another 10 bio's go into the flush queue after only the first five
bio's are removed from the flush queue, the queue should look like:
         6 7 8 9 10 11 12 13 14 15 16 17 18 19 20,
but because of the bug we end up with
         6 11 12 13 14  15 16 17 18 19 20 7 8 9 10.
So the sequence of the bio's is damaged in the flush queue (and
therefore in the journal on disk !). This error can be triggered by
ffs_snapshot() when a block is read with readblock() and gjournal finds
this block in the broken flush queue before it goes to the correct
active queue.

The fix is to place all new blocks at the end of the queue.

Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Discussed with: kib
MFC after: 1 week
2017-08-07 19:40:03 +00:00
Kirk McKusick
683590b642 sysctl kern.geom.journal.cache.limit shows negative value for FreeBSD/amd64
system having over 4GB RAM. That's due to:

1) the limit being u_int instead of u_long like vm.kmem_size (the limit is
   half of vm.kmem_size by default for amd64);
2) sysctl handler g_journal_cache_limit_sysctl() using u_int instead of u_long.

The fix is to replace u_int with u_long for the kern.geom.journal.cache.limit
sysctl variable.

PR: 198500
Submitted by: Dr. Andreas Longwitz <longwitz@incore.de>
Reported by: Eugene Grosbein
Discussed with: kib
MFC after: 1 week
2017-08-07 19:18:27 +00:00
Alexander Motin
1631690677 Add GEOM::descr attribute for symmetry with GEOM::ident.
MFC after:	2 weeks
2017-07-06 08:36:14 +00:00
Ryan Libby
fb0e3235ea g_virstor.h: macro parenthesization
Build with gcc -Wint-in-bool-context revealed a macro parenthesization
error (invoking LOG_MSG with a ternary expression for lvl).

Reviewed by:	markj
Approved by:	markj (mentor)
Sponsored by:	Dell EMC Isilon
Differential revision:	https://reviews.freebsd.org/D11411
2017-06-30 22:01:18 +00:00
Marcelo Araujo
4323355e76 With r318394 seems it breaks gpart(8) in some embedded systems such like PCEngines,
RPI1-B, Alix and APU2 boards as well as NanoBSD with the following message:

vnode_pager_generic_getpages_done: I/O read error 5

Seems the breakage was because it was missed to include acr in glabel update.

Reported by:	Peter Blok <pblok@bsd4all.org>,
		madpilot, imp and trasz.
Reviewed by:	trasz
Tested by:	Peter Blok and madpilot.
MFC after:	3 days.
Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D11365
2017-06-27 01:22:27 +00:00
Stephen J. Kiernan
9a81ba0f24 Add MD_VERIFY option to enable O_VERIFY in open for vnode type.
Add -o [no]verify option to mdconfig (and document in man page.)
Implement GEOM attribute MNT::verified to ask md if the backing vnode is
  verified.
Check for MNT::verified in cd9660 mount to flag the mount as MNT_VERIFIED if
  the underlying device has been verified.

Reviewed by:	rwatson
Approved by:	sjg (mentor)
Obtained from:	Juniper Networks, Inc.
Differential Revision:	https://reviews.freebsd.org/D2902
2017-05-31 21:18:11 +00:00
Edward Tomasz Napierala
6635c8ed2f Fix typo.
MFC after:	2 weeks
2017-05-18 08:25:07 +00:00
Mark Johnston
db7c508323 Synchronize unclean mirrors before adding them to a running gmirror.
During gmirror startup, if component mirrors are found to be dirty as is
typical after a system crash, the mirrors are synchronized to the mirror
with highest priority. However if a gmirror starts without all of its
mirrors present, for example because of some transient delays during
tasting, the remaining mirrors must be synchronized before they may become
active.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-02 23:29:42 +00:00