1882 Commits

Author SHA1 Message Date
hselasky
1f41d295fb MFC r263710, r273377, r273378, r273423 and r273455:
- De-vnet hash sizes and hash masks.
- Fix multiple issues related to arguments passed to SYSCTL macros.

Sponsored by:	Mellanox Technologies
2014-10-27 14:38:00 +00:00
cperciva
925d357775 MFC r271664:
Cache GELI passphrases entered at the console during the boot process,
  in order to improve user-friendliness when a system has multiple disks
  encrypted using the same passphrase.

Relnotes:	yes
Approved by:	re (gjb)
2014-09-23 00:43:18 +00:00
emaste
37dd226a2e MFC EFI support for the installer
r264978 (nwhitehorn):

    Add EFI support to the installer. This requires that the kernel
    provide a sysctl to determine what firmware is in use. This sysctl
    does not exist yet, so the following blocks are in front of the
    wheels:
    - I've provisionally called this "hw.platform" after the equivalent
      thing on PPC
    - The logic to check the sysctl is short-circuited to always choose
      BIOS.  There's a comment in the top of the file about how to turn
      this off.

    If IA64 acquired a boot1.efifat-like thing (probably with very few
    modifications), the same code could be adapted there.

r265016 (nwhitehorn):

    Finish connecting up installer UEFI support. If the kernel was
    booted using EFI, set up the disks for an EFI system. If booted from
    BIOS/CSM, set up for BIOS.

r268256 (nwhitehorn):

    After EFI support was added to the installer, it needed to allow
    boot partitions of types other than "freebsd-boot" (in particular,
    "efi").  This allows the removal of some nasty hacks for supporting
    PowerPC systems, in particular aliasing freebsd-boot to apple-boot
    on APM and an IBM-specific code on MBR.

    This changes the installer to use the correct names, which also
    breaks a degeneracy in the meaning of "freebsd-boot" that allows the
    addition of support for some newer IBM systems that can boot from
    GPT in addition to MBR. Since I have no idea how to detect which
    those systems are, leave the default on IBM PPC systems as MBR for
    now.

Approved by:	re
PR:		193658
Relnotes:	Yes
2014-09-15 17:49:46 +00:00
smh
a7cb473513 MFC r256956:
Improve ZFS N-way mirror read performance by using load and locality
information.

MFC r260713:
Fix ZFS mirror code for handling multiple DVA's

Also make the addition of the d_rotation_rate binary compatible. This allows
storage drivers compiled for 10.0 to work by preserving the ABI for disks.

Approved by:	re (gjb)
Sponsored by:	Multiplay
2014-09-07 21:30:47 +00:00
ae
0f937e3532 MFC r268407 (by gjb):
Fix non-version text after .Fx macro usage.

MFC r269487 (by issyl0):
  Add generic list, status, load and unload docs to gpart(8)

  - In the style of gmirror(8).
  PR:		docs/191534

MFC r269852:
  Add sysctl and loader tunable kern.geom.part.mbr.enforce_chs that is set
  by default. It can be used to disable automatic alignment to CHS geometry,
  that GEOM_PART_MBR does.
2014-08-25 12:49:10 +00:00
marcel
250620f917 MFC 268986; fix file system corruption by creating as many BIOs as needed
to satisfy the original request -- in other words: no short reads.

Obtained from:	Juniper Networks, Inc.
2014-08-03 03:06:00 +00:00
ae
b6530ab0a7 MFC r267355:
Add UUIDs for DragonFlyBSD's partition types.

MFC r267356:
  Add DragonFlyBSD's Hammer FS types and type names.

MFC r267357:
  Add aliases for DragonFlyBSD's partition types.

MFC r267358:
  Allow dumping to DragonFlyBSD's swap partition.

MFC r267359:
  Add disklabel64 support to GEOM_PART class.

  This partitioning scheme is used in DragonFlyBSD. It is similar to
  BSD disklabel, but has the following improvements:
  * metadata has own dedicated place and isn't accessible through partitions;
  * all offsets are 64-bit;
  * supports 16 partitions by default (has reserved place for more);
  * has reserved place for backup label (but not yet implemented);
  * has UUIDs for partitions and partition types;

MFC r267360:
  Add disklabel64 support

Relnotes:	yes
2014-07-01 13:29:17 +00:00
marius
06338f8ccc MFC: r267145
Fix the keyfile being cleared prematurely after r259428 (MFCed to stable/10
in r266749).

PR:		185084
Submitted by:	fk@fabiankeil.de
Reviewed by:	pjd
2014-06-25 10:17:00 +00:00
ae
2c6af1420a MFC r266880:
Use g_conf_printf_escaped() to escape symbols, which can break
  an XML tree.
2014-06-06 12:37:56 +00:00
ae
af0a9c5a59 MFC r266444:
We have two functions from where a geom orphan method could be called:
  g_orphan_register and g_resize_provider_event. Both are called from the
  event queue. Also we have GEOM_DEV class, which does deferred destroy
  for its consumers via g_dev_destroy (also called from the event queue).
  So it is possible, that for some consumers an orphan method will be
  called twice. This triggers panic in g_dev_orphan.
  Check that consumer isn't already orphaned before call orphan method.
2014-06-02 10:14:03 +00:00
marius
d22760cc90 MFC: r259428
Clear content of keyfiles loaded by the loader after processing them.

MFC: r259429

Clear some more places with potentially sensitive data.
2014-05-27 14:55:09 +00:00
ae
b0a39c9645 MFC r266445:
Add a topology trace to the g_spoil_event.
2014-05-26 07:04:30 +00:00
mav
f56a48e361 MFC r266319:
Make GEOM DISK to account also BIO_FLUSH operations.
2014-05-24 10:20:56 +00:00
loos
058ac53f3f MFC r260522, r260523, r261439, r261440, r261586, r264504, r264769, r265193,
r265194, r265197

r260522:
Add the manual page for geom_uncompress(4).

r260523:
Build the geom_uncompress(4) module by default.

Fix geom_uncompress(4) module loading.  Don't link zlib.c (which is a module
itself) directly.

r261439:
Remove some unnecessary code.  The offsets read from the first block are
overwritten a few lines bellow.

r261440:
Fix a logic error.  Because of this inflateReset() wasn't being called and
the output buffer wasn't being cleared between the inflate() calls,
producing zeroed output after the first inflate() call.

This fixes the read of mkuzip(8) images with geom_uncompress(4).

r261586:
Fix the build with DEBUG enabled.  Where possible, fix style(9) issues.

r264504:
Make sure not to do I/O for more than MAXPHYS bytes. Doing so can cause
problems in our providers, such as a KASSERT in md(4). We can initiate
I/O for more than MAXPHYS bytes if we've been given a BIO for MAXPHYS
bytes, the blocks from which we're reading couldn't be compressed and
we had compression in preceeding blocks resulting in misalignment of
the blocks we're trying to read relative to the sector. We're forced to
round up the I/O length to make it an multiple of the sector size.

When we detect the condition, we'll reduce the block count and perform
a "short" read. In g_uzip_done() we need to consider the original I/O
length and stop early if we're about to deflate a block that we didn't
read. By using bio_completed in the cloned BIO and not bio_length to
check for this, we automatically and gracefully handle short reads that
our providers may be doing on top of the short reads we may initiate
ourselves.

r264769:
Keep geom_uncompress(4) in line with geom_uzip(4), bring in the r264504 fix.

Make sure not to start I/O bigger than MAXPHYS bytes.

r265193:
Some style and whitespace fixes.  Reduce the difference between geom_uzip(4)
and geom_uncompress(4).  Now, they produce an almost clean diff(1) output.

Remove a duplicated variable from g_uncompress.c and an unnecessary header
from g_uzip.c.

r265194:
Actually the FEATURE() macro is defined on sys/sysctl.h.

r265197:
Fix a leak in g_uzip_taste().  After retrieve all the block offsets from
the uzip image, free the last data read.
2014-05-16 14:28:55 +00:00
bdrewery
21b23c5efb MFC r265072:
Remove redundant include
2014-05-14 15:35:01 +00:00
bdrewery
b53c2b05cd MFC r264499:
Make g_access() KASSERT() more useful.
2014-05-14 15:30:49 +00:00
ae
8e5289f450 MFC r256690:
Add an automatic resize support to the GEOM_PART class.

  When parent provider has been resized, the scheme specific G_PART_RESIZE
  method does an update of scheme's metadata. But all changes are not saved
  to disk, until `gpart commit` will be called.

MFC r265336:
  Add an advice what to do when partition was automatically resized.
2014-05-12 12:04:44 +00:00
ae
8bd4b60954 MFC r265318:
For schemes that do an automatic partition aligning move this code to
  separate function.

MFC r265331:
  Prevent an unexpected shrinking on resizing due to alignment for MBR,
  PC98 and VTOC8 schemes.

MFC r265333:
  Add better error description for case when we are doing resize and
  scheme-specific method returns EBUSY.

MFC r265539:
  It is safe to allow shrinking, when aligned size is bigger than current.
2014-05-12 10:19:31 +00:00
mav
df9f601cbc MFC r265054:
Reduce number of opens by REOM RAID during provider taste.

Instead opening/closing provider by each of metadata classes, do it only
once in core code.  Since for SCSI disks open/close means sending some
SCSI commands to the device, this change reduces taste time.

Sponsored by:	iXsystems, Inc.
2014-05-08 12:07:40 +00:00
mav
69f94d7a6b MFC r264313:
Do not increment bio_data in case of BIO_DELETE.

This fixes KASSERT() panic in g_io_request().
2014-05-08 12:04:40 +00:00
mav
c2e3819ca8 MFC r264318:
Fix wrong sizes used to access PD_Type and PD_State DDF metadata fields.

This caused incorrect behavior of arrays with big-endian DDF metadata.
Little-endian (like used by Adaptec controllers) should not be harmed.
Add workaround should be enough to manage compatibility.
2014-04-24 10:22:00 +00:00
bdrewery
73c210dce0 MFC r264142:
Show error code when failing to destroy a mirror on delay
2014-04-21 02:49:41 +00:00
bdrewery
5d04f39f50 MFC r264320:
Fix spelling error in g_trace() call.
2014-04-21 02:48:04 +00:00
delphij
4d10945810 MFC r261618:
In g_eli_crypto_hmac_init(), zero out after using the ipad buffer,
k_ipad.

Note that the two consumers in geli(4) are not affected by this
issue because the way the code is constructed and as such, we
believe there is no security impact with or without this change
with geli(4)'s usage.

Reported by:	Serge van den Boom <serge vdboom.org>
Reviewed by:	pjd
2014-02-22 00:30:33 +00:00
marcel
035a990673 MFC r258448:
Have the GPT probe return a lower priority when the MBR is not a PMBR.
2014-02-16 19:46:20 +00:00
eadler
ec294fd7f5 MFC r258779,r258780,r258787,r258822:
Fix undefined behavior: (1 << 31) is not defined as 1 is an int and this
shifts into the sign bit.  Instead use (1U << 31) which gets the
expected result.

Similar to the (1 << 31) case it is not defined to do (2 << 30).

This fix is not ideal as it assumes a 32 bit int, but does fix the issue
for most cases.

A similar change was made in OpenBSD.
2014-02-04 03:36:42 +00:00
mav
4d04af12a8 MFC r260883:
Remove unneeded and dangerous assignment.  It would probably cause NULL
refererence panic if compiler not optimize it out.
2014-02-02 00:33:17 +00:00
ae
11f15ea787 MFC r261084:
malloc() with M_WAITOK doesn't return NULL.

MFC r261085:
  Fix typo in r261084.
  Add to the gctl_error() an ability to specify error description even
  if numeric error code is already specified. Also by default set
  error code to EINVAL.

  PR:		185852

MFC r261086:
  In gctl_copyin() remove unused error variable.
  geom_alloc_copyin() can't return ENOMEM, so describe its fail as bad
  control request. Add check for NULL pointer in gctl_dump(), since it
  can be NULL when geom_alloc_copyin() failed.

MFC r261089:
  Remove another unneeded NULL check from geom_alloc_copyin().
  Do copyout in case of gctl version mismatch and fix sbuf leak in
  g_ctl_ioctl_ctl().

MFC r261091:
  Always free sbuf in gctl_free().
2014-01-30 10:53:29 +00:00
marck
bc9a363c26 MFC r259925-259926:
Add GPT UUID for VMware vSAN meta-data partition.

Approved by:	ae
2014-01-21 09:19:28 +00:00
ae
2865f87f4a MFC r259634:
Prevent users from deactivating the last component of a mirror.

MFC r259929:
  Add an ability to stop gmirror and clear its metadata in one command.
  This fixes the problem, when gmirror starts again just after stop.

  The problem occurs when gmirror's component has geom label with equal size.
  E.g. gpt and gptid have the same size as partition, diskid has the same
  size as entire disk. When gmirror's geom has been destroyed, glabel
  creates its providers and this initiate retaste.

  Now "gmirror destroy" command is available. It destroys geom and also
  erases gmirror's metadata.

PR:		184985
2014-01-10 07:48:36 +00:00
ae
5308ec5802 MFC r258357:
Add "resize" verb to gmirror(8) and such functionality to geom_mirror(4).
  Now it is easy to expand the size of the mirror when all its components
  are replaced. Also add g_resize method to geom_mirror class. It will write
  updated metadata to new last sector, when parent provider is resized.
2014-01-10 07:43:40 +00:00
mav
263d261fe5 MFC r258683:
Escape special XML chars, returned by some devices, confusing XML parsers.
2014-01-09 11:13:03 +00:00
mav
3b6d8c3206 MFC r258220, r258251:
Implement automatic live resize support for GEOM MULTIPATH class.

In "manual" mode just automatically resize provider in any direction.
In "automatic" mode allow growth (with new metadata write); in case of
shrinking check if there is already valid metadata found at the new
location.  This should allow easy transparent recovery if first resize
was done by mistake.

While there, unify metadata write code and fix minor memory leak.
2014-01-09 11:11:47 +00:00
scottl
0a34594b9c MFC Alexander Motin's GEOM direct dispatch work:
r256603:
Introduce new function devstat_end_transaction_bio_bt(), adding new argument
to specify present time.  Use this function to move binuptime() out of lock,
substantially reducing lock congestion when slow timecounter is used.

r256606:
Move g_io_deliver() out of the lock, as required for direct dispatch.
Move g_destroy_bio() out too to reduce lock scope even more.

r256607:
Fix passing uninitialized bio_resid argument to g_trace().

r256610:
Add unmapped I/O support to GEOM RAID.

r256830:
Restore BIO_UNMAPPED and BIO_TRANSIENT_MAPPING in biodonne() when unmapping
temporary mapped buffer.  That fixes double unmap if biodone() called twice
for the same BIO (but with different done methods).

r256880:
Merge GEOM direct dispatch changes from the projects/camlock branch.

When safety requirements are met, it allows to avoid passing I/O requests
to GEOM g_up/g_down thread, executing them directly in the caller context.
That allows to avoid CPU bottlenecks in g_up/g_down threads, plus avoid
several context switches per I/O.

r259247:
Fix bug introduced at r256607.  We have to recalculate bp_resid here since
sizes of original and completed requests may differ due to end of media.

Testing of the stable/10 merge was done by Netflix, but all of the credit
goes to Alexander and iX Systems.

Submitted by:   mav
Sponsored by:   iX Systems
2014-01-07 01:32:23 +00:00
ae
a210965c36 MFC r257965:
Add missing line breaks.

  PR:		181900
2013-12-14 04:16:27 +00:00
trasz
a82d470c95 MFC r256724:
Make geom_label(4) resize-aware.  This fixes a situation when "gpart resize"
would resize a partition, but label providers - e.g. /dev/gptid/XXX - would
stay the same size.

MFC r256766:

Fix build with gcc by spelling unused format string as "unused" instead of NULL.

Sponsored by:	The FreeBSD Foundation
2013-12-13 20:33:59 +00:00
mjg
8718fd5993 MFC r256951:
gnop: make sure that newly allocated memory for softc is zeroed

This prevents mtx_init from encountering non-zeros and panicking
the kernel as a result.

Approved by:	re
2013-11-23 23:54:38 +00:00
delphij
5b5f66ad93 MFC r257539:
When zero'ing out a buffer, make sure we are using right size.

Without this change, in the worst but unlikely case scenario, certain
administrative operations, including change of configuration, set or
delete key from a GEOM ELI provider, may leave potentially sensitive
information in buffer allocated from kernel memory.

We believe that it is not possible to actively exploit these issues, nor
does it impact the security of normal usage of GEOM ELI providers when
these operations are not performed after system boot.

Security:	possible sensitive information disclosure
Submitted by:	Clement Lecigne <clecigne google com>
Approved by:	re (glebius)
2013-11-05 19:58:40 +00:00
des
140807754c Introduce a kern.geom.notaste sysctl that can be used to temporarily
disable GEOM tasting to avoid the "bouncing GEOM" problem where, when
you shut down the consumer of a provider which can be viewed in multiple
ways (typically a mirror whose members are labeled partitions), GEOM
will immediately taste that provider's alter ego and reattach the
consumer.

Approved by:	re (glebius)
2013-09-24 20:05:16 +00:00
ae
7f30f5be1c Remove stub implementation.
MFC after:	1 week
2013-09-05 09:44:09 +00:00
mav
8324fc3480 Make ELI destruction (including orphanization) less aggressive, making it
always wait for provider close.  Old algorithm was reported to cause NULL
dereference panic on attempt to close provider after softc destruction.
If not global workaroung in GEOM, that could even cause destruction with
requests still in flight.
2013-09-02 10:44:54 +00:00
mav
3380a03b00 MFprojects/camlock r254895:
Add unmapped BIO support to GEOM ZERO if kern.geom.zero.clear is cleared.
2013-08-26 20:39:02 +00:00
mav
e8031ce26c Add new attribute lunname to report only textual LUN-specific device IDs.
While lunid attribute prefers to report numeric ones, having both may be
useful in some situations.
2013-08-24 09:42:14 +00:00
ken
5591de079d Change the way that unmapped I/O capability is advertised.
The previous method was to set the D_UNMAPPED_IO flag in the cdevsw
for the driver.  The problem with this is that in many cases (e.g.
sa(4)) there may be some instances of the driver that can handle
unmapped I/O and some that can't.  The isp(4) driver can handle
unmapped I/O, but the esp(4) driver currently cannot.  The cdevsw
is shared among all driver instances.

So instead of setting a flag on the cdevsw, set a flag on the cdev.
This allows drivers to indicate support for unmapped I/O on a
per-instance basis.

sys/conf.h:	Remove the D_UNMAPPED_IO cdevsw flag and replace it
		with an SI_UNMAPPED cdev flag.

kern_physio.c:	Look at the cdev SI_UNMAPPED flag to determine
		whether or not a particular driver can handle
		unmapped I/O.

geom_dev.c:	Set the SI_UNMAPPED flag for all GEOM cdevs.
		Since GEOM will create a temporary mapping when
		needed, setting SI_UNMAPPED unconditionally will
		work.

		Remove the D_UNMAPPED_IO flag.

nvme_ns.c:	Set the SI_UNMAPPED flag on cdevs created here
		if NVME_UNMAPPED_BIO_SUPPORT is enabled.

vfs_aio.c:	In aio_qphysio(), check the SI_UNMAPPED flag on a
		cdev instead of the D_UNMAPPED_IO flag on the cdevsw.

sys/param.h:	Bump __FreeBSD_version to 1000045 for the switch from
		setting the D_UNMAPPED_IO flag in the cdevsw to setting
		SI_UNMAPPED in the cdev.

Reviewed by:	kib, jimharris
MFC after:	1 week
Sponsored by:	Spectra Logic
2013-08-15 22:52:39 +00:00
mav
eba4a485b2 Return error when opening read-only volumes (like RAID4/5/...) for writing.
Previously opens succeeded, but actual write operations returned errors.

Requested by:	peter
MFC after:	2 weeks
2013-08-13 07:56:40 +00:00
mav
d9e76bbffc Oops, wrong constant at r254269. 2013-08-13 06:25:34 +00:00
mav
1ddae2c9b4 Fix reasonable but safe Clang warnings. 2013-08-13 06:21:36 +00:00
ed
e591d48c3e Fix the formatting of the error message.
The G_MIRROR_DEBUG() macro already appends a newline. Also, most of the
log messages emitted by gmirror start with an uppercase letter.
2013-08-12 18:17:45 +00:00
ae
4c7750d3a8 gpt_entries is used as limit for the number of partition entries in
the GEOM_PART. Instead of just using number of entries from the GPT
header, calculate this limit based on the reserved space between
GPT header and first available LBA.

MFC after:	2 weeks
2013-08-08 16:09:20 +00:00
marcel
9f2f2e171a Change <sys/diskpc98.h> to not redefine the same symbols that are
being defined in <sys/diskmbr.h>. Instead give the symbols here a
"PC98_" prefix. This way, both <sys/diskmbr.h> and <sys/diskpc98.h>
can be included in the same C source file.

The renaming is trivial. The only gotcha is that DOSBBSECTOR is
also redefined from 0 to 1. This because DOSBBSECTOR was always
used in conjunction with an addition of 1. The PC98_BBSECTOR symbol
is defined as 1 and the expression is simplified.

Note: it is not believed that ports are seriously impacted; or at
all for that matter.

Approved by: nyan@
2013-08-07 00:00:48 +00:00