Commit Graph

44 Commits

Author SHA1 Message Date
Pawel Jakub Dawidek
bcb77be2b7 Add TRIM support.
The code builds a map of regions that were freed. On every write the
code consults the map and eventually removes ranges that were freed
before, but are now overwritten.

Freed blocks are not TRIMed immediately. There is a tunable that defines
how many txg we should wait with TRIMming freed blocks (64 by default).

There is a low priority thread that TRIMs ranges when the time comes.
During TRIM we keep in-flight ranges on a list to detect colliding
writes - we have to delay writes that collide with in-flight TRIMs in
case something will be reordered and write will reached the disk before
the TRIM. We don't have to do the same for in-flight writes, as
colliding writes just remove ranges to TRIM.

Sponsored by:	multiplay.co.uk

This work includes some important fixes and some improvements obtained
from the zfsonlinux project, including TRIMming entire vdevs on pool
create/add/attach and on pool import for spare and cache vdevs.

Obtained from:	zfsonlinux
Submitted by:	Etienne Dechamps <etienne.dechamps@ovh.net>
2012-09-23 19:40:58 +00:00
Martin Matuska
4c5238d576 Merge recent zfs vendor changes, sync code and adjust userland DEBUG.
Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
     is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-12 18:05:43 +00:00
Rui Paulo
e04dfc407c Finish porting execsnoop to FreeBSD. This includes replacing the zonename
with a jail ID and removing the project ID from the list of options.
2012-09-01 08:14:21 +00:00
Martin Matuska
e9832bb1da Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after:	2 weeks
2012-07-30 23:14:24 +00:00
George V. Neville-Neil
3fac94ba94 Initial commit of an I/O provider for DTrace on FreeBSD.
These probes are most useful when looking into the structures
they provide, which are listed in io.d.  For example:

dtrace -n 'io:genunix::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.
MFC after:	1 month
2012-07-11 16:27:02 +00:00
Martin Matuska
2d9cf57e18 Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month
2012-06-11 11:35:22 +00:00
George V. Neville-Neil
c5069a4756 Add DTrace's io.d, which handles tranlsations for file, buffer and
device info structures as well as the fds[] array.  This is a raw
version of the file, unmodified, to be used as a baseline.
2012-06-05 18:58:05 +00:00
Oleksandr Tymoshenko
457c7edfbd Enable build of DTrace-related userland parts for MIPS 2012-03-24 05:29:07 +00:00
Martin Matuska
2f7f0f4112 Merge new ZFS features from illumos:
1644 add ZFS "clones" property
https://www.illumos.org/issues/1644

1645 add ZFS "written" and "written@..." properties
https://www.illumos.org/issues/1645

1646 "zfs send" should estimate size of stream
https://www.illumos.org/issues/1646

1647 "zfs destroy" should determine space reclaimed by destroying multiple
snapshots
https://www.illumos.org/issues/1647

1693 persistent 'comment' field for a zpool
https://www.illumos.org/issues/1693

1708 adjust size of zpool history data
https://www.illumos.org/issues/1708

1748 desire support for reguid in zfs
https://www.illumos.org/issues/1748

Obtained from:	illumos (changesets 13514, 13524, 13525)
MFC after:	1 month
2011-11-28 21:40:00 +00:00
Robert Watson
d04fbf251a Change the Makefile in cddl/lib/drti to use bsd.lib.mk instead of
bsd.prog.mk -- we need to compile PIC, which requires a library build.
With this change, USDT (userspace DTrace probes) work from within
shared libraries.

PR:		kern/159046
Submitted by:	Alex Samorukov <samm at os2.kiev.ua>
Comments by:	Scott Lystig Fritchie <slfritchie at snookles.com>
MFC after:	3 days
2011-11-28 10:01:36 +00:00
Robert Millan
3536206713 Fix a few gratuitous library dependencies. Some of the ZFS utilities
are linked with libraries they don't use:

 - zinject doesn't use libavl
 - ztest doesn't use libz
 - zdb uses neither libavl nor libz
 - zfs uses neither libbsdxml nor libm, nor libsbuf
 - zpool uses neither libbsdxml nor libm, nor libsbuf

In addition, libzfs needs libm because it uses pow(), however it isn't
linked with -lm.  This went unnoticed because all its users had -lm before.

Reviewed by:	pjd, mm
Approved by:	kib (mentor)
MFC after:	1 week
2011-10-30 16:29:04 +00:00
Pawel Jakub Dawidek
10b9d77bf1 Finally... Import the latest open-source ZFS version - (SPA) 28.
Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
  transaction group.
- Possibility to import pool in read-only mode.

MFC after:	1 month
2011-02-27 19:41:40 +00:00
Dimitry Andric
38d82872c6 When building libzpool on ia64 or sparc64, don't add the .note.GNU-stack
section.

Submitted by:	kib
2011-02-15 22:28:15 +00:00
Dimitry Andric
152e60f2fe Fix some leftover binaries and shared libraries in the system that still
have an executable stack, due to linking in hand-assembled .S or .s
files, that have no .GNU-stack sections:

RWX --- ---  /lib/libcrypto.so.6
RWX --- ---  /lib/libmd.so.5
RWX --- ---  /lib/libz.so.6
RWX --- ---  /lib/libzpool.so.2
RWX --- ---  /usr/lib/liblzma.so.5

These were found using scanelf, from the sysutils/pax-utils port.

Reviewed by:	kib
2011-02-15 22:03:09 +00:00
Rui Paulo
8399e5f307 Use ${PICFLAG} instead of -fpic. 2010-10-27 18:46:56 +00:00
Warner Losh
25faff346c MFtbemd:
Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.
2010-08-23 22:24:11 +00:00
Marius Strobl
222447d5e0 Use real atomic operations for sparc64.
MFC after:	1 week
2010-08-23 22:04:30 +00:00
Nathan Whitehorn
de563149f5 Add powerpc64 to the list of architectures with real atomic operations.
Submitted by:	imp
2010-08-23 16:47:29 +00:00
Rui Paulo
029a737371 Don't link drti.o with libelf_pic.a. This means that all software with
userland SDT probes must be linked with libelf explicitly.

Requested by:	kib
Sponsored by:	The FreeBSD Foundation
2010-08-21 17:29:59 +00:00
Rui Paulo
513bdf9ac0 Link drti.o with a PIC version of libelf. This is needed because
drti.o depends on libelf and this avoids linking every other drti.o
program (namely programs with USDT probes) with libelf.

Sponsored by:	The FreeBSD Foundation
2010-08-21 13:48:04 +00:00
Rui Paulo
0f2bd1e89d Add libdtrace support for tracing userland programs.
Summary of changes:
* Implement a compatibility shim between Solaris libproc and our
libproc and remove several ifdefs because of this.
* Port the drti to FreeBSD.
* Implement the missing DOODAD sections
* Link with libproc and librtld_db
* Support for ustack, jstack and uregs (by sson@)
* Misc bugfixing

When writing the SUWN_dof section, we had to resort to building the ELF
file layout by "hand". This is the job of libelf, but our libelf doesn't
support this yet. When libelf is fixed, we can remove the code under
#ifdef BROKEN_LIBELF.

Sponsored by:	The FreeBSD Foundation
2010-08-21 11:50:53 +00:00
Rui Paulo
d6f390f2f1 Use double quotes when checking the value of MACHINE_ARCH.
Sponsored by:	The FreeBSD Foundation
2010-08-21 11:14:50 +00:00
Rui Paulo
0c0b3feeed The DTrace instrumentation object is going to be i386/amd64 only.
Sponsored by:	The FreeBSD Foundation
2010-08-21 11:09:03 +00:00
Rui Paulo
fd14f3b4a9 libdtrace, dtrace(1) and lockstat(1) are for i386 and amd64 only right
now.

Sponsored by:	The FreeBSD Foundation
2010-07-31 17:13:14 +00:00
Ulrich Spörlein
4ef20db290 Remove manual .includes in cddl Makefiles
- Break the dependency on ../Makefile.inc for .PATH, and include
  ../Makefile.inc implicitly. This is required to ...
- Set WARNS?=6 in top-level Makefile.inc
- Remove now redundant WARNS settings, add WARNS?=0 where appropriate
- Remove redundant SHLIB_MAJOR overrides
- Use NO_MAN, not MK_MAN=no
- Remove redundant inclusion of bsd.own.mk
- Order Makefiles more according to style.Makefile(9)
- Reduce diff of cddl Makefiles against each other

No objection:	pjd
Approved by:	ed (co-mentor)
2010-03-02 19:04:07 +00:00
Pawel Jakub Dawidek
830940567b Remove files that are no longer used.
Discussed with:	kmacy
Approved by:	re (kib)
2009-08-17 08:03:02 +00:00
Ken Smith
3ca3047aee Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE.  Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by:    kib
Approved by:    re (rwatson)
2009-07-19 17:25:24 +00:00
Kip Macy
3f9fc7fea0 atomic.S has been renamed opensolaris_atomic.S to avoid collisions 2009-05-09 05:39:35 +00:00
Roman Divacky
300d03a832 Switch over to gnu99 compilation on default for userland.
Tested by:	make universe
Tested by:	ports exp build (done by pav)
Reviewed by:	ru
Reviewed by:	silence on arch
Approved by:	ed (mentor)
2009-03-14 17:55:16 +00:00
Pawel Jakub Dawidek
1ba4a712dd Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.
This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

	Allows regular users to perform ZFS operations, like file system
	creation, snapshot creation, etc.

- L2ARC

	Level 2 cache for ZFS - allows to use additional disks for cache.
	Huge performance improvements mostly for random read of mostly
	static content.

- slog

	Allow to use additional disks for ZFS Intent Log to speed up
	operations like fsync(2).

- vfs.zfs.super_owner

	Allows regular users to perform privileged operations on files stored
	on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

	Not all the flags are supported. This still needs work.

- ZFSBoot

	Support to boot off of ZFS pool. Not finished, AFAIK.

	Submitted by:	dfr

- Snapshot properties

- New failure modes

	Before if write requested failed, system paniced. Now one
	can select from one of three failure modes:
	- panic - panic on write error
	- wait - wait for disk to reappear
	- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

	Just quota and reservation properties, but don't count space consumed
	by children file systems, clones and snapshots.

- Sparse volumes

	ZVOLs that don't reserve space in the pool.

- External attributes

	Compatible with extattr(2).

- NFSv4-ACLs

	Not sure about the status, might not be complete yet.

	Submitted by:	trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from:	OpenSolaris
2008-11-17 20:49:29 +00:00
John Birrell
ebe86aacbe Add the DTrace libraries and D library scripts. 2008-05-22 04:26:42 +00:00
Ruslan Ermilov
dbdb679c6f Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse.  This should fix WITHOUT_LIBTHR builds as a side effect.
2008-03-29 17:44:40 +00:00
John Birrell
e327f52446 The sources covered by Sun's CDDL have been repo copied below the
src/cddl and src/sys/cddl directories per the core@ decision following
the license review.

This change modifies the affected Makefiles to reference the sources
in their new location.
2008-03-27 23:21:25 +00:00
John Birrell
ee8a5fa77d Remove _SOLARIS_C_SOURCE now that it doesn't do anything in FreeBSD
headers. All OpenSolaris compatibility comes via the set of specific
compatibility headers in src/compat/opensolaris and
src/sys/compat/opensolaris.
2007-11-28 22:58:09 +00:00
David E. O'Brien
946367b8e2 Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by:	ru
Approved by:	re(kensmith)
2007-10-09 23:31:11 +00:00
David E. O'Brien
65c045e964 Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by:	re(kensmith)
2007-10-09 13:42:34 +00:00
Ruslan Ermilov
bcf72246c5 Don't build bits that depend on the pthreads support if a
system was configured without such support.

Approved by:	re (kensmith)
2007-10-01 18:23:24 +00:00
Pawel Jakub Dawidek
3b7917d766 - Reduce number of atomic operations needed to be implemented in asm by
implementing some of them using existing ones.
- Allow to compile ZFS on all archs and use atomic operations surrounded
  by global mutex on archs we don't have or can't have all atomic
  operations needed by ZFS.
2007-06-08 12:35:47 +00:00
Pawel Jakub Dawidek
7baf73a6c2 Use provider's ident to handle situations when disks are moved around
and show up with different names: first try to open provider using
remembered name and compare its ident, if equal, this is our provider,
if not equal or there is no provider with such name, find provider with
remembered ident and don't care about the name.
2007-05-06 01:39:39 +00:00
Pawel Jakub Dawidek
1c22c47c35 Link libzpool, ztest and zdb against libpthread.
Requested by:	ru
2007-04-17 15:52:37 +00:00
Ruslan Ermilov
5941f0355c Apply a set of style.Makefile(5) changes to src/cddl/ makefiles.
There are some insignificant non-style changes as well.

Not fixed: makefiles use ${LIBTHR} that doesn't exist, thus
breaking "make checkdpadd" and not tracking dependencies
properly.

Approved by:	pjd
2007-04-16 21:20:26 +00:00
Pawel Jakub Dawidek
3dc4488c91 Move atomic.S files to directories that better fit OpenSolaris directory
layout.
2007-04-07 23:54:54 +00:00
Dag-Erling Smørgrav
48be553b82 Build ZFS on amd64 and pc98.
Approved by:	pjd@
2007-04-07 19:12:10 +00:00
Pawel Jakub Dawidek
f0a75d274a Please welcome ZFS - The last word in file systems.
ZFS file system was ported from OpenSolaris operating system. The code in under
CDDL license.

I'd like to thank all SUN developers that created this great piece of software.

Supported by:	Wheel LTD (http://www.wheel.pl/)
Supported by:	The FreeBSD Foundation (http://www.freebsdfoundation.org/)
Supported by:	Sentex (http://www.sentex.net/)
2007-04-06 01:09:06 +00:00