Commit Graph

44 Commits

Author SHA1 Message Date
pjd
618888b019 Add TRIM support.
The code builds a map of regions that were freed. On every write the
code consults the map and eventually removes ranges that were freed
before, but are now overwritten.

Freed blocks are not TRIMed immediately. There is a tunable that defines
how many txg we should wait with TRIMming freed blocks (64 by default).

There is a low priority thread that TRIMs ranges when the time comes.
During TRIM we keep in-flight ranges on a list to detect colliding
writes - we have to delay writes that collide with in-flight TRIMs in
case something will be reordered and write will reached the disk before
the TRIM. We don't have to do the same for in-flight writes, as
colliding writes just remove ranges to TRIM.

Sponsored by:	multiplay.co.uk

This work includes some important fixes and some improvements obtained
from the zfsonlinux project, including TRIMming entire vdevs on pool
create/add/attach and on pool import for spare and cache vdevs.

Obtained from:	zfsonlinux
Submitted by:	Etienne Dechamps <etienne.dechamps@ovh.net>
2012-09-23 19:40:58 +00:00
mm
063b8917f2 Merge recent zfs vendor changes, sync code and adjust userland DEBUG.
Illumos issued covered:
1884 Empty "used" field for zfs *space commands
3006 VERIFY[S,U,P] and ASSERT[S,U,P] frequently check if first argument
     is zero
3028 zfs {group,user}space -n prints (null) instead of numeric GID/UID
3048 zfs {user,group}space [-s|-S] is broken
3049 zfs {user,group}space -t doesn't really filter the results
3060 zfs {user,group}space -H output isn't tab-delimited
3061 zfs {user,group}space -o doesn't use specified fields order
3064 usr/src/cmd/zpool/zpool_main.c misspells "successful"
3093 zfs {user,group}space's -i is noop
3098 zfs userspace/groupspace fail without saying why when run as non-root

References:
  https://www.illumos.org/issues/ + [issue_id]

Obtained from:	illumos (vendor/illumos, vendor/illumos-sys)
MFC after:	2 weeks
2012-09-12 18:05:43 +00:00
rpaulo
d430c71377 Finish porting execsnoop to FreeBSD. This includes replacing the zonename
with a jail ID and removing the project ID from the list of options.
2012-09-01 08:14:21 +00:00
mm
cf6d705d5b Partial MFV (illumos-gate 13753:2aba784c276b)
2762 zpool command should have better support for feature flags

References:
https://www.illumos.org/issues/2762

MFC after:	2 weeks
2012-07-30 23:14:24 +00:00
gnn
786ac82553 Initial commit of an I/O provider for DTrace on FreeBSD.
These probes are most useful when looking into the structures
they provide, which are listed in io.d.  For example:

dtrace -n 'io:genunix::start { printf("%d\n", args[0]->bio_bcount); }'

Note that the I/O systems in FreeBSD and Solaris/Illumos are sufficiently
different that there is not a 1:1 mapping from scripts that work
with one to the other.
MFC after:	1 month
2012-07-11 16:27:02 +00:00
mm
cc61ab2f13 Introduce "feature flags" for ZFS pools (bump SPA version to 5000).
Add first feature "com.delphix:async_destroy" (asynchronous destroy
of ZFS datasets).
Implement features support in ZFS boot code.

Illumos revisions merged:
13700:2889e2596bd6
13701:1949b688d5fb
2619 asynchronous destruction of ZFS file systems
2747 SPA versioning with zfs feature flags

References:
https://www.illumos.org/issues/2619
https://www.illumos.org/issues/2747

Obtained from:	illumos (issue #2619, #2747)
MFC after:	1 month
2012-06-11 11:35:22 +00:00
gnn
2d93dc867d Add DTrace's io.d, which handles tranlsations for file, buffer and
device info structures as well as the fds[] array.  This is a raw
version of the file, unmodified, to be used as a baseline.
2012-06-05 18:58:05 +00:00
gonzo
a704c8860f Enable build of DTrace-related userland parts for MIPS 2012-03-24 05:29:07 +00:00
mm
73cab71971 Merge new ZFS features from illumos:
1644 add ZFS "clones" property
https://www.illumos.org/issues/1644

1645 add ZFS "written" and "written@..." properties
https://www.illumos.org/issues/1645

1646 "zfs send" should estimate size of stream
https://www.illumos.org/issues/1646

1647 "zfs destroy" should determine space reclaimed by destroying multiple
snapshots
https://www.illumos.org/issues/1647

1693 persistent 'comment' field for a zpool
https://www.illumos.org/issues/1693

1708 adjust size of zpool history data
https://www.illumos.org/issues/1708

1748 desire support for reguid in zfs
https://www.illumos.org/issues/1748

Obtained from:	illumos (changesets 13514, 13524, 13525)
MFC after:	1 month
2011-11-28 21:40:00 +00:00
rwatson
1a5f36a1c6 Change the Makefile in cddl/lib/drti to use bsd.lib.mk instead of
bsd.prog.mk -- we need to compile PIC, which requires a library build.
With this change, USDT (userspace DTrace probes) work from within
shared libraries.

PR:		kern/159046
Submitted by:	Alex Samorukov <samm at os2.kiev.ua>
Comments by:	Scott Lystig Fritchie <slfritchie at snookles.com>
MFC after:	3 days
2011-11-28 10:01:36 +00:00
rmh
1ab901bfdd Fix a few gratuitous library dependencies. Some of the ZFS utilities
are linked with libraries they don't use:

 - zinject doesn't use libavl
 - ztest doesn't use libz
 - zdb uses neither libavl nor libz
 - zfs uses neither libbsdxml nor libm, nor libsbuf
 - zpool uses neither libbsdxml nor libm, nor libsbuf

In addition, libzfs needs libm because it uses pow(), however it isn't
linked with -lm.  This went unnoticed because all its users had -lm before.

Reviewed by:	pjd, mm
Approved by:	kib (mentor)
MFC after:	1 week
2011-10-30 16:29:04 +00:00
pjd
1b03c5bf41 Finally... Import the latest open-source ZFS version - (SPA) 28.
Few new things available from now on:

- Data deduplication.
- Triple parity RAIDZ (RAIDZ3).
- zfs diff.
- zpool split.
- Snapshot holds.
- zpool import -F. Allows to rewind corrupted pool to earlier
  transaction group.
- Possibility to import pool in read-only mode.

MFC after:	1 month
2011-02-27 19:41:40 +00:00
dim
8ca1e470b6 When building libzpool on ia64 or sparc64, don't add the .note.GNU-stack
section.

Submitted by:	kib
2011-02-15 22:28:15 +00:00
dim
f58eeaae2e Fix some leftover binaries and shared libraries in the system that still
have an executable stack, due to linking in hand-assembled .S or .s
files, that have no .GNU-stack sections:

RWX --- ---  /lib/libcrypto.so.6
RWX --- ---  /lib/libmd.so.5
RWX --- ---  /lib/libz.so.6
RWX --- ---  /lib/libzpool.so.2
RWX --- ---  /usr/lib/liblzma.so.5

These were found using scanelf, from the sysutils/pax-utils port.

Reviewed by:	kib
2011-02-15 22:03:09 +00:00
rpaulo
28a426fd14 Use ${PICFLAG} instead of -fpic. 2010-10-27 18:46:56 +00:00
imp
c3a399c4ba MFtbemd:
Prefer MACHNE_CPUARCH to MACHINE_ARCH in most contexts where you want
to test of all the CPUs of a given family conform.
2010-08-23 22:24:11 +00:00
marius
a1a6e5ff56 Use real atomic operations for sparc64.
MFC after:	1 week
2010-08-23 22:04:30 +00:00
nwhitehorn
c508e8d056 Add powerpc64 to the list of architectures with real atomic operations.
Submitted by:	imp
2010-08-23 16:47:29 +00:00
rpaulo
6a5d9f2b9f Don't link drti.o with libelf_pic.a. This means that all software with
userland SDT probes must be linked with libelf explicitly.

Requested by:	kib
Sponsored by:	The FreeBSD Foundation
2010-08-21 17:29:59 +00:00
rpaulo
26d140a225 Link drti.o with a PIC version of libelf. This is needed because
drti.o depends on libelf and this avoids linking every other drti.o
program (namely programs with USDT probes) with libelf.

Sponsored by:	The FreeBSD Foundation
2010-08-21 13:48:04 +00:00
rpaulo
b325e69c92 Add libdtrace support for tracing userland programs.
Summary of changes:
* Implement a compatibility shim between Solaris libproc and our
libproc and remove several ifdefs because of this.
* Port the drti to FreeBSD.
* Implement the missing DOODAD sections
* Link with libproc and librtld_db
* Support for ustack, jstack and uregs (by sson@)
* Misc bugfixing

When writing the SUWN_dof section, we had to resort to building the ELF
file layout by "hand". This is the job of libelf, but our libelf doesn't
support this yet. When libelf is fixed, we can remove the code under
#ifdef BROKEN_LIBELF.

Sponsored by:	The FreeBSD Foundation
2010-08-21 11:50:53 +00:00
rpaulo
45dce14583 Use double quotes when checking the value of MACHINE_ARCH.
Sponsored by:	The FreeBSD Foundation
2010-08-21 11:14:50 +00:00
rpaulo
c3d495b24d The DTrace instrumentation object is going to be i386/amd64 only.
Sponsored by:	The FreeBSD Foundation
2010-08-21 11:09:03 +00:00
rpaulo
6a96e8c49c libdtrace, dtrace(1) and lockstat(1) are for i386 and amd64 only right
now.

Sponsored by:	The FreeBSD Foundation
2010-07-31 17:13:14 +00:00
uqs
ce11b17db8 Remove manual .includes in cddl Makefiles
- Break the dependency on ../Makefile.inc for .PATH, and include
  ../Makefile.inc implicitly. This is required to ...
- Set WARNS?=6 in top-level Makefile.inc
- Remove now redundant WARNS settings, add WARNS?=0 where appropriate
- Remove redundant SHLIB_MAJOR overrides
- Use NO_MAN, not MK_MAN=no
- Remove redundant inclusion of bsd.own.mk
- Order Makefiles more according to style.Makefile(9)
- Reduce diff of cddl Makefiles against each other

No objection:	pjd
Approved by:	ed (co-mentor)
2010-03-02 19:04:07 +00:00
pjd
6ccc3544c4 Remove files that are no longer used.
Discussed with:	kmacy
Approved by:	re (kib)
2009-08-17 08:03:02 +00:00
kensmith
9c2c634ee9 Bump the version of all non-symbol-versioned shared libraries in
preparation for 8.0-RELEASE.  Add the previous version of those
libraries to ObsoleteFiles.inc and bump __FreeBSD_Version.

Reviewed by:    kib
Approved by:    re (rwatson)
2009-07-19 17:25:24 +00:00
kmacy
78c3549a9f atomic.S has been renamed opensolaris_atomic.S to avoid collisions 2009-05-09 05:39:35 +00:00
rdivacky
4cd01fc223 Switch over to gnu99 compilation on default for userland.
Tested by:	make universe
Tested by:	ports exp build (done by pav)
Reviewed by:	ru
Reviewed by:	silence on arch
Approved by:	ed (mentor)
2009-03-14 17:55:16 +00:00
pjd
bbe899b96e Update ZFS from version 6 to 13 and bring some FreeBSD-specific changes.
This bring huge amount of changes, I'll enumerate only user-visible changes:

- Delegated Administration

	Allows regular users to perform ZFS operations, like file system
	creation, snapshot creation, etc.

- L2ARC

	Level 2 cache for ZFS - allows to use additional disks for cache.
	Huge performance improvements mostly for random read of mostly
	static content.

- slog

	Allow to use additional disks for ZFS Intent Log to speed up
	operations like fsync(2).

- vfs.zfs.super_owner

	Allows regular users to perform privileged operations on files stored
	on ZFS file systems owned by him. Very careful with this one.

- chflags(2)

	Not all the flags are supported. This still needs work.

- ZFSBoot

	Support to boot off of ZFS pool. Not finished, AFAIK.

	Submitted by:	dfr

- Snapshot properties

- New failure modes

	Before if write requested failed, system paniced. Now one
	can select from one of three failure modes:
	- panic - panic on write error
	- wait - wait for disk to reappear
	- continue - serve read requests if possible, block write requests

- Refquota, refreservation properties

	Just quota and reservation properties, but don't count space consumed
	by children file systems, clones and snapshots.

- Sparse volumes

	ZVOLs that don't reserve space in the pool.

- External attributes

	Compatible with extattr(2).

- NFSv4-ACLs

	Not sure about the status, might not be complete yet.

	Submitted by:	trasz

- Creation-time properties

- Regression tests for zpool(8) command.

Obtained from:	OpenSolaris
2008-11-17 20:49:29 +00:00
jb
1f1c4e3e67 Add the DTrace libraries and D library scripts. 2008-05-22 04:26:42 +00:00
ru
0f0375e36a Remove options MK_LIBKSE and DEFAULT_THREAD_LIB now that we no longer
build libkse.  This should fix WITHOUT_LIBTHR builds as a side effect.
2008-03-29 17:44:40 +00:00
jb
5794ada908 The sources covered by Sun's CDDL have been repo copied below the
src/cddl and src/sys/cddl directories per the core@ decision following
the license review.

This change modifies the affected Makefiles to reference the sources
in their new location.
2008-03-27 23:21:25 +00:00
jb
769c37d0ba Remove _SOLARIS_C_SOURCE now that it doesn't do anything in FreeBSD
headers. All OpenSolaris compatibility comes via the set of specific
compatibility headers in src/compat/opensolaris and
src/sys/compat/opensolaris.
2007-11-28 22:58:09 +00:00
obrien
a1598920aa Tweak the handling of "WITHOUT_LIBPTHREAD". Also remove the accidental
treatment of 'LIBKSE' as an "old style" knob.

Submitted by:	ru
Approved by:	re(kensmith)
2007-10-09 23:31:11 +00:00
obrien
eb3f6a9e74 Repo copy libpthreads to libkse.
This introduces the WITHOUT_LIBKSE nob,
and changes WITHOUT_LIBPTHREADS to mean with neither threading libs.
Approved by:	re(kensmith)
2007-10-09 13:42:34 +00:00
ru
c05a2547c6 Don't build bits that depend on the pthreads support if a
system was configured without such support.

Approved by:	re (kensmith)
2007-10-01 18:23:24 +00:00
pjd
9eba2904d1 - Reduce number of atomic operations needed to be implemented in asm by
implementing some of them using existing ones.
- Allow to compile ZFS on all archs and use atomic operations surrounded
  by global mutex on archs we don't have or can't have all atomic
  operations needed by ZFS.
2007-06-08 12:35:47 +00:00
pjd
1effa32b66 Use provider's ident to handle situations when disks are moved around
and show up with different names: first try to open provider using
remembered name and compare its ident, if equal, this is our provider,
if not equal or there is no provider with such name, find provider with
remembered ident and don't care about the name.
2007-05-06 01:39:39 +00:00
pjd
5c0f9c4bd3 Link libzpool, ztest and zdb against libpthread.
Requested by:	ru
2007-04-17 15:52:37 +00:00
ru
fc861d6ddf Apply a set of style.Makefile(5) changes to src/cddl/ makefiles.
There are some insignificant non-style changes as well.

Not fixed: makefiles use ${LIBTHR} that doesn't exist, thus
breaking "make checkdpadd" and not tracking dependencies
properly.

Approved by:	pjd
2007-04-16 21:20:26 +00:00
pjd
65ca3ed324 Move atomic.S files to directories that better fit OpenSolaris directory
layout.
2007-04-07 23:54:54 +00:00
des
fa363796e1 Build ZFS on amd64 and pc98.
Approved by:	pjd@
2007-04-07 19:12:10 +00:00
pjd
3b005d3302 Please welcome ZFS - The last word in file systems.
ZFS file system was ported from OpenSolaris operating system. The code in under
CDDL license.

I'd like to thank all SUN developers that created this great piece of software.

Supported by:	Wheel LTD (http://www.wheel.pl/)
Supported by:	The FreeBSD Foundation (http://www.freebsdfoundation.org/)
Supported by:	Sentex (http://www.sentex.net/)
2007-04-06 01:09:06 +00:00