Commit Graph

93 Commits

Author SHA1 Message Date
Li-Wen Hsu
78b42ee37c Fix LINT build.
Approved by:	delphij
2018-01-26 06:21:24 +00:00
Enji Cooper
193d9e768b sys/modules: normalize .CURDIR-relative paths to SRCTOP
This simplifies make output/logic

Tested with:	`cd sys/modules; make ALL_MODULES=` on amd64
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 10:10:17 +00:00
Enji Cooper
38ede98e9a Fix "make depend" with geom_uzip.ko: add opt_geom.h to SRCS
MFC after:	1 week
Sponsored by:	Dell EMC Isilon
2017-03-04 08:28:33 +00:00
Marius Strobl
4874af73c1 - Allow different slicers for different flash types to be registered
with geom_flashmap(4) and teach it about MMC for slicing enhanced
  user data area partitions. The FDT slicer still is the default for
  CFI, NAND and SPI flash on FDT-enabled platforms.
- In addition to a device_t, also pass the name of the GEOM provider
  in question to the slicers as a single device may provide more than
  provider.
- Build a geom_flashmap.ko.
- Use MODULE_VERSION() so other modules can depend on geom_flashmap(4).
- Remove redundant/superfluous GEOM routines that either do nothing
  or provide/just call default GEOM (slice) functionality.
- Trim/adjust includes

Submitted by:	jhibbits (RouterBoard bits)
Reviewed by:	jhibbits
2017-02-22 10:21:39 +00:00
Yoshihiro Takahashi
2b375b4edd Remove pc98 support completely.
I thank all developers and contributors for pc98.

Relnotes:	yes
2017-01-28 02:22:15 +00:00
Ruslan Bukin
ae8b1f90fe Fix alignment issues on MIPS: align the pointers properly.
All the 5520 GEOM_ELI tests passed successfully on MIPS64EB.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
Differential Revision:	https://reviews.freebsd.org/D7905
2016-10-31 16:55:14 +00:00
Ruslan Bukin
68a18b821e Disable geom_eli module build on MIPS64 as it has alignment
issues and causes kernel panic.
ELI metadata is also not aligned properly for MIPS64 case.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-10-20 17:10:26 +00:00
Maxim Sobolev
5497acc527 Obsolete mkulzma(8) and geom_uncompress(4), their functionality
is now provided by mkuzip(8) and geom_uzip(4) respectively.

MFC after:	1 month
2016-02-24 00:39:36 +00:00
Maxim Sobolev
8f8cb840b0 Improve mkuzip(8) and geom_uzip(4), merge in LZMA support from mkulzma(8)
and geom_uncompress(4):

1. mkuzip(8):

 - Proper support for eliminating all-zero blocks when compressing an
   image. This feature is already supported by the geom_uzip(4) module
   and CLOOP format in general, so it's just a matter of making mkuzip(8)
   match. It should be noted, however that this feature while it sounds
   great, results in very slight improvement in the overall compression
   ratio, since compressing default 16k all-zero block produces only 39
   bytes compressed output block, which is 99.8% compression ratio. With
   typical average compression ratio of amd64 binaries and data being
   around 60-70% the difference between 99.8% and 100.0% is not that
   great further diluted by the ratio of number of zero blocks in the
   uncompressed image to the overall number of blocks being less than
   0.5 (typically). However, this may be important from performance
   standpoint, so that kernel are not spinning its wheels decompressing
   those empty blocks every time this zero region is read. It could also
   be important when you create huge image mostly filled with zero
   blocks for testing purposes.

 - New feature allowing to de-duplicate output image. It turns out that
   if you twist CLOOP format a bit you can do that as well. And unlike
   zero-blocks elimination, this gives a noticeable improvement in the
   overall compression ratio, reducing output image by something like
   3-4% on my test UFS2 3GB image consisting of full FreeBSD base system
   plus some of the packages (openjdk, apache etc), about 2.3GB worth of
   file data (800+MB compressed). The only caveat is that images created
   with this feature "on" would not work on older versions of FeeBSDxi
   kernel, hence it's turned off by default.

 - provide options to control both features and document them in manual
   page.

 - merge in all relevant LZMA compression support from the mkulzma(8),
   add new option to select between both.

 - switch license from ad-hoc beerware into standard 2-clause BSD.

2. geom_uzip(4):

 - implement support for de-duplicated images;

 - optimize some code paths to handle "all-zero" blocks without reading
   any compressed data;

 - beef up manual page to explain that geom_uzip(4) is not limited only
   to md(4) images. The compressed data can be written to the block
   device and accessed directly via magic of GEOM(4) and devfs(4),
   including to mount root fs from a compressed drive.

 - convert debug log code from being compiled in conditionally into
   being present all the time and provide two sysctls to turn it on or
   off. Due to intended use of the module, it can be used in
   environments where there may not be a luxury to put new kernel with
   debug code enabled. Having those options handy allows debug issues
   without as much problem by just having access to serial console or
   network shell access to a box/appliance. The resulting additional
   CPU cycles are just few int comparisons and branches, and those are
   minuscule when compared to data decompression which is the main
   feature of the module.

 - hopefully improve robustness and resiliency of the geom_uzip(4) by
   performing some of the data validation / range checking on the TOC
   entries and rejecting to attach to an image if those checks fail.

 - merge in all relevant LZMA decompression support from the
   geom_uncompress(4), enable automatically when appropriate format is
   indicated in the header.

 - move compilation work into its own worker thread so that it does not
   clog g_up. This allows multiple instances work in parallel utilizing
   smp cores.

 - document new knobs in the manual page.

Reviewed by:		adrian
MFC after:		1 month
Differential Revision:	https://reviews.freebsd.org/D5333
2016-02-23 23:59:08 +00:00
Allan Jude
4332feca4b Make additional parts of sys/geom/eli more usable in userspace
The upcoming GELI support in the loader reuses parts of this code
Some ifdefs are added, and some code is moved outside of existing ifdefs

The HMAC parts of GELI are broken out into their own file, to separate
them from the kernel crypto/openssl dependant parts that are replaced
in the boot code.

Passed the GELI regression suite (tools/regression/geom/eli)
 Files=20 Tests=14996
 Result: PASS

Reviewed by:	pjd, delphij
MFC after:	1 week
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D4699
2016-01-07 05:47:34 +00:00
Allan Jude
7a3f5d11fb Replace sys/crypto/sha2/sha2.c with lib/libmd/sha512c.c
cperciva's libmd implementation is 5-30% faster

The same was done for SHA256 previously in r263218

cperciva's implementation was lacking SHA-384 which I implemented, validated against OpenSSL and the NIST documentation

Extend sbin/md5 to create sha384(1)

Chase dependancies on sys/crypto/sha2/sha2.{c,h} and replace them with sha512{c.c,.h}

Reviewed by:	cperciva, des, delphij
Approved by:	secteam, bapt (mentor)
MFC after:	2 weeks
Sponsored by:	ScaleEngine Inc.
Differential Revision:	https://reviews.freebsd.org/D3929
2015-12-27 17:33:59 +00:00
Warner Losh
6a607537da Scheduling module to introduce a fixed delay into the I/O path. 2015-12-18 05:39:25 +00:00
Warner Losh
12f05b8446 Kill MFILES and find things automatically. It turned out to be only
lightly used. Find the proper .m file when we depend on *_if.[ch] in
the srcs line, with seat-belts for false positive matches.  This uses
make's path mechanism. A further refinement would be to calculate this
once, and then pass the resulting _MPATH to modules submakes.

Differential Revision: https://reviews.freebsd.org/D2327
2015-07-03 01:50:26 +00:00
Andrey V. Elsukov
fcdb1ffc49 Add makefile to build geom_map kld. Document some GEOM_* options
in NOTES and geom(4).
2015-06-08 13:23:56 +00:00
Enji Cooper
18cc317233 Add MK_CCD knob for building and installing ccd(4), ccdconfig, etc
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
2015-01-25 04:52:48 +00:00
Warner Losh
cf26c00351 Remove comments relevant to 6.x only. 2014-12-20 00:07:53 +00:00
Warner Losh
0acf08d985 Remove support for FreeBSD 7 and really old FreeBSD 8. The classifiers
have been in the base for a while, so the gymnastics here aren't
needed. In addition, the bugs in subr_disk.c have been fixed since
2009, so there's no need for an identical copy of it in the tree
anymore. There's really no need to binary patch g_io_request, so let's
get rid of the code (not compiled in anymore) lest others think it is
a good idea.
2014-12-20 00:04:01 +00:00
Warner Losh
73f49e9eef Implement the historic DIOCGDINFO ioctl for gpart on BSD
partitions. Several utilities still use this interface and require
additional information since gpart was activated than before. This
allows fsck of a UFS partition without having to specify it is UFS,
per historic behavior.
2014-11-18 17:06:40 +00:00
Andrey V. Elsukov
91ca76a590 Add disklabel64 support to GEOM_PART class.
This partitioning scheme is used in DragonFlyBSD. It is similar to
BSD disklabel, but has the following improvements:
* metadata has own dedicated place and isn't accessible through partitions;
* all offsets are 64-bit;
* supports 16 partitions by default (has reserved place for more);
* has reserved place for backup label (but not yet implemented);
* has UUIDs for partitions and partition types;

No objections from:	geom
MFC after:	2 weeks
Relnotes:	yes
2014-06-11 10:42:34 +00:00
Andrey V. Elsukov
303a8b5985 Disconnect the following geom classes from the kernel modules build:
BSD, FOX, MBR, PC98, SUNLABEL and VOL_FFS. They all have a modern
replacement. Also it is still possible build them manually.

Discussed with:	geom
2014-05-27 10:21:49 +00:00
Marcel Moolenaar
e8c166e85a An all-or-nothing approach to labels isn't flexible enough. Embedded
systems need fine-grained control over what's in and what's out.
That's ideal. For now, separate GPT labels from the rest and allow
g_label to be built with just GPT labels.

Obtained from:	Juniper Networks, Inc.
2014-04-06 02:44:37 +00:00
John-Mark Gurney
f225bcd157 I clearly didn't test the modules... add sha256c.c to the various
modules that included sha2.c...
2014-03-16 01:55:30 +00:00
Luiz Otavio O Souza
a1f5df2215 Fix a leftover of r260523. Remove the unnecessary dependency to zlib.h.
Approved by:	adrian (mentor)
2014-02-27 13:29:26 +00:00
Luiz Otavio O Souza
67619a4120 Build the geom_uncompress(4) module by default.
Fix geom_uncompress(4) module loading.  Don't link zlib.c (which is a module
itself) directly.

The built module was verified and used to read a few mkulzma(8) images on
amd64 to validate some of the informations on the manual page.

While here, don't overwrite CFLAGS.

Reviewed by:	ray
Approved by:	adrian (mentor)
2014-01-10 20:29:46 +00:00
Ivan Voras
7ceaf939d6 Link g_label_disk_ident when building geom_label as a module 2013-04-17 09:19:29 +00:00
Attilio Rao
682ee99e7a It seems that it is preferable to keep support for glabel also for
filesystems that we don't support natively.
Revert part of r241636 to do so.

This patch is not targeted for MFC.

Requested by:	gleb, jhb
2012-10-18 22:18:11 +00:00
Attilio Rao
a42ac676f5 Disconnect non-MPSAFE NTFS from the build in preparation for dropping
GIANT from VFS. This code is particulary broken and fragile and other
in-kernel implementations around, found in other operating systems,
don't really seem clean and solid enough to be imported at all.
If someone wants to reconsider in-kernel NTFS implementation for
inclusion again, a fair effort for completely fixing and cleaning it
up is expected.

In the while NTFS regular users can use FUSE interface and ntfs-3g
port to work with their NTFS partitions.

This is not targeted for MFC.
2012-10-17 11:30:00 +00:00
Alexander Motin
00f32ecbd0 Add to GEOM RAID class module, supporting the DDF metadata format, as
defined by the SNIA Common RAID Disk Data Format Specification v2.0.

Supports multiple volumes per array and multiple partitions per disk.
Supports standard big-endian and Adaptec's little-endian byte ordering.
Supports all single-layer RAID levels. Dual-layer RAID levels except
RAID10 are not supported now because of GEOM RAID design limitations.

Some work is still to be done, but the present code already manages basic
interoperation with RAID BIOS of the Adaptec 1430SA SATA RAID controller.

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2012-04-30 17:53:02 +00:00
Alexander Motin
fc1de96060 Add to GEOM RAID class module for reading non-degraded RAID5 volumes and
some environment to differentiate 4 possible RAID5 on-disk layouts.

Tested with Intel and AMD RAID BIOSes.

MFC after:	2 weeks
2012-04-19 12:30:12 +00:00
Andrey V. Elsukov
8bb4272910 Connect geom_part_ldm module to the build.
MFC after:	1 month
2012-03-19 13:15:48 +00:00
Aleksandr Rybalko
ce96bb7942 GEOM_UNCOMPRESS module, can be used with uzip images and with new ulzma images.
Approved by:	adrian (mentor)
2012-01-04 23:39:11 +00:00
Pawel Jakub Dawidek
1e09ff3dc3 Instead of allocating memory for all the keys at device attach,
create reasonably large cache for the keys that is filled when
needed. The previous version was problematic for very large providers
(hundreds of terabytes or serval petabytes). Every terabyte of data
needs around 256kB for keys. Make the default cache limit big enough
to fit all the keys needed for 4TB providers, which will eat at most
1MB of memory.

MFC after:	2 weeks
2011-04-21 13:31:43 +00:00
Alexander Motin
89b172238a MFgraid/head:
Add new RAID GEOM class, that is going to replace ataraid(4) in supporting
various BIOS-based software RAIDs. Unlike ataraid(4) this implementation
does not depend on legacy ata(4) subsystem and can be used with any disk
drivers, including new CAM-based ones (ahci(4), siis(4), mvs(4), ata(4)
with `options ATA_CAM`). To make code more readable and extensible, this
implementation follows modular design, including core part and two sets
of modules, implementing support for different metadata formats and RAID
levels.

Support for such popular metadata formats is now implemented:
Intel, JMicron, NVIDIA, Promise (also used by AMD/ATI) and SiliconImage.

Such RAID levels are now supported:
RAID0, RAID1, RAID1E, RAID10, SINGLE, CONCAT.

For any all of these RAID levels and metadata formats this class supports
full cycle of volume operations: reading, writing, creation, deletion,
disk removal and insertion, rebuilding, dirty shutdown detection
and resynchronization, bad sector recovery, faulty disks tracking,
hot-spare disks. For Intel and Promise formats there is support multiple
volumes per disk set.

Look graid(8) manual page for additional details.

Co-authored by:	imp
Sponsored by:	Cisco Systems, Inc. and iXsystems, Inc.
2011-03-24 21:31:32 +00:00
Luigi Rizzo
1831a90ac5 Bring in geom_sched, support for scheduling disk I/O requests
in a device independent manner. Also include an example anticipatory
scheduler, gsched_rr, which gives very nice performance improvements
in presence of competing random access patterns.

This is joint work with Fabio Checconi, developed last year
and presented at BSDCan 2009. You can find details in the
README file or at

http://info.iet.unipi.it/~luigi/geom_sched/
2010-04-12 16:37:45 +00:00
Edward Tomasz Napierala
b3f9d8c804 Add gmountver, disk mount verification GEOM class.
Note that due to e.g. write throttling ('wdrain'), it can stall all the disk
I/O instead of just the device it's configured for.  Using it for removable
media is therefore not a good idea.

Reviewed by:	pjd (earlier version)
2010-01-16 09:52:49 +00:00
Ivan Voras
cd2dd44393 Add missing reference to GPT support.
Submitted by:	Paul B. Mahol onemda at gmail.com
Approved by:	re (kib)
2009-07-05 14:03:41 +00:00
Marcel Moolenaar
a90e8937be g_part_ebr.c includes opt_geom.h 2009-05-30 22:04:18 +00:00
Ulf Lilleengen
c0b9797aa8 Import the gvinum work that have been done during and after Summer of Code 2007.
The work have been under testing and fixing since then, and it is mature enough
to be put into HEAD for further testing.

A lot have changed in this time, and here are the most important:
- Gvinum now uses one single workerthread instead of one thread for each
  volume and each plex. The reason for this is that the previous scheme was
  very complex, and was the cause of many of the bugs discovered in gvinum.
  Instead, gvinum now uses one worker thread with an event queue, quite
  similar to what used in gmirror.
- The rebuild/grow/initialize/parity check routines no longer runs in
  separate threads, but are run as regular I/O requests with special flags.
  This made it easier to support mounted growing and parity rebuild.
- Support for growing striped and raid5-plexes, meaning that one can extend the
  volumes for these plex types in addition to the concat type. Also works while
  the volume is mounted.
- Implementation of many of the missing commands from the old vinum:
  attach/detach, start (was partially implemented), stop (was partially
  implemented), concat, mirror, stripe, raid5 (shortcuts for creating volumes
  with one plex of these organizations).
- The parity check and rebuild no longer goes between userland/kernel, meaning
  that the gvinum command will not stay and wait forever for the rebuild to
  finish. You can instead watch the status with the list command.
- Many problems with gvinum have been reported since 5.x, and some has been hard
  to fix due to the complicated architecture. Hopefully, it should be more
  stable and better handle edge cases that previously made gvinum crash.
- Failed drives no longer disappears entirely, but now leave behind a dummy
  drive that makes sure the original state is not forgotten in case the system
  is rebooted between drive failures/swaps.
- Update manpage to reflect new commands and extend it with some examples.

Sponsored by:   Google Summer of Code 2007
Mentored by:    le
Tested by:      Rick C. Petty <rick-freebsd2008 -at- kiwi-computer.com>
2009-03-28 17:20:08 +00:00
Marcel Moolenaar
5d68db5bc8 Add the EBR scheme. The EBR scheme supports the Extended Boot Records
found inside extended partitions and used to create logical partitions.
At this time write/modify support is not (yet) present.
The EBR and MBR schemes both check the parent scheme. The MBR will
back-off when nested under another MBR, whereas the EBR only nests
under a MBR.
2009-02-08 23:51:44 +00:00
Marcel Moolenaar
b03fab128b Add support for PC-9800 partition tables. 2008-03-28 17:58:55 +00:00
Marcel Moolenaar
807e684076 Instead of making a single geom_part.ko module, make a module
for each partitioning scheme. The gpart code is currently non-
optional.
2008-03-23 01:42:47 +00:00
Marcel Moolenaar
252dfdbdbd Remove makefiles for geom_apple and geom_gpt. These are non-existent
for while.
2008-03-02 05:59:25 +00:00
Marcel Moolenaar
da50cd3932 Add geom_part module. 2008-03-02 05:54:34 +00:00
Ruslan Ermilov
995dc98447 Remove WARNS from here and compile with default kernel flags. 2008-02-21 11:09:59 +00:00
Andrew Thompson
00620931e1 geom_lvm(4) is now known as geom_linux_lvm(4). 2008-02-20 11:32:12 +00:00
Andrew Thompson
b61033d925 Rename geom_lvm(4) to geom_linux_lvm(4).
Requested by:   des, phk
2008-02-20 11:30:49 +00:00
Andrew Thompson
1669d8afc6 Rename geom_lvm(4) to geom_linux_lvm(4).
Requested by:	des, phk
2008-02-20 07:50:13 +00:00
Andrew Thompson
2b8d4f5bd4 Hook geom_lvm(4) up to the build. 2008-02-11 03:10:40 +00:00
Pawel Jakub Dawidek
88882dcf11 Add GEOM VIRSTOR module to the build. 2007-10-26 08:23:57 +00:00
Pawel Jakub Dawidek
f854db0bf5 Bring in the GEOM Virtualisation class, which allows to create huge GEOM
providers with limited physical storage and add physical storage as
needed.

Submitted by:	Ivan Voras
Sponsored by:	Google Summer of Code 2006
Approved by:	re (kensmith)
2007-09-23 07:34:23 +00:00