Commit Graph

79 Commits

Author SHA1 Message Date
Matt Macy
9e5787d228 Merge OpenZFS support in to HEAD.
The primary benefit is maintaining a completely shared
code base with the community allowing FreeBSD to receive
new features sooner and with less effort.

I would advise against doing 'zpool upgrade'
or creating indispensable pools using new
features until this change has had a month+
to soak.

Work on merging FreeBSD support in to what was
at the time "ZFS on Linux" began in August 2018.
I first publicly proposed transitioning FreeBSD
to (new) OpenZFS on December 18th, 2018. FreeBSD
support in OpenZFS was finally completed in December
2019. A CFT for downstreaming OpenZFS support in
to FreeBSD was first issued on July 8th. All issues
that were reported have been addressed or, for
a couple of less critical matters there are
pull requests in progress with OpenZFS. iXsystems
has tested and dogfooded extensively internally.
The TrueNAS 12 release is based on OpenZFS with
some additional features that have not yet made
it upstream.

Improvements include:
  project quotas, encrypted datasets,
  allocation classes, vectorized raidz,
  vectorized checksums, various command line
  improvements, zstd compression.

Thanks to those who have helped along the way:
Ryan Moeller, Allan Jude, Zack Welch, and many
others.

Sponsored by:	iXsystems, Inc.
Differential Revision:	https://reviews.freebsd.org/D25872
2020-08-25 02:21:27 +00:00
Mark Johnston
858e8e41dd Stop compiling dtrace modules with -DSMP.
I believe this is left over from when dtrace was being ported and
developed out-of-tree.  Now it just ensures that dtrace.ko and a non-SMP
kernel have incompatible KBIs.

PR:		243711
Sponsored by:	The FreeBSD Foundation
2020-02-05 19:08:45 +00:00
Mark Johnston
efe88d92da Disconnect fasttrap from the 32-bit powerpc build.
An upcoming bug fix requires 64-bit atomics, which aren't implemented on
powerpc.  The powerpc port of fasttrap is incomplete anyway and doesn't
get loaded by dtraceall.ko on powerpc because of a missing dependency;
it's presumed that it's effectively unused.

Discussed with:	jhibbits
MFC after:	2 weeks
2019-02-21 22:49:21 +00:00
Brooks Davis
6469bdcdb6 Move most of the contents of opt_compat.h to opt_global.h.
opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c.  A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by:	kib, cem, jhb, jtl
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14941
2018-04-06 17:35:35 +00:00
Ed Maste
fc2a8776a2 Rename assym.s to assym.inc
assym is only to be included by other .s files, and should never
actually be assembled by itself.

Reviewed by:	imp, bdrewery (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14180
2018-03-20 17:58:51 +00:00
Michael Tuexen
05f08f2978 Build systrace_freebsd32 on arm64.
Since r326501 which added COMPAT_FREEBSD32 to the GENERIC config for arm64,
we need to build systrace_freebsd32, since dtraceall depends on it.

Reviewed by:		ed, gnn
Differential Revision:	https://reviews.freebsd.org/D13670
2018-01-11 21:17:05 +00:00
Pedro F. Giffuni
fe267a5590 sys: general adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

No functional change intended.
2017-11-27 15:23:17 +00:00
Mark Johnston
8bb47d1432 Don't redefine _KERNEL.
MFC after:	1 week
2017-11-24 19:08:54 +00:00
Kurt Lidl
a8273e4371 Enable dtrace support for mips64 and the ERL kernel config
Turn on the required options in the ERL config file, and ensure
that the fbt module is listed as a dependency for mips in
the modules/dtrace/dtraceall/dtraceall.c file.

PR: 		220346
Reviewed by:	gnn, markj
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D12227
2017-09-06 03:19:52 +00:00
Ruslan Bukin
ca20f8ec29 o Replace __riscv__ with __riscv
o Replace __riscv64 with (__riscv && __riscv_xlen == 64)

This is required to support new GCC 7.1 compiler.
This is compatible with current GCC 6.1 compiler.

RISC-V is extensible ISA and the idea here is to have built-in define
per each extension, so together with __riscv we will have some subset
of these as well (depending on -march string passed to compiler):

__riscv_compressed
__riscv_atomic
__riscv_mul
__riscv_div
__riscv_muldiv
__riscv_fdiv
__riscv_fsqrt
__riscv_float_abi_soft
__riscv_float_abi_single
__riscv_float_abi_double
__riscv_cmodel_medlow
__riscv_cmodel_medany
__riscv_cmodel_pic
__riscv_xlen

Reviewed by:	ngie
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D11901
2017-08-07 14:09:57 +00:00
Enji Cooper
5c63f26129 Normalize SYSDIR on SRCTOP instead of .CURDIR
This is being done to simplify pathing for CFLAGS and source files.

MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
2017-05-18 06:15:42 +00:00
Bryan Drewery
9c0e1d45c5 Stop building assym.o into the module.
DTrace includes assym.s, to build this we build assym.o, however
this is unneeded as assym.s only contains macros.

Remove the need to build this by removing it from OBJS, but keep assym.s
in the module dependencies via DPSRCS.

This fixes the build when there is no assembler, e.g. on arm64 without
the external binutils.

Submitted by:	andrew
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D10041
2017-04-05 17:56:21 +00:00
Robert Watson
1811d6bf7f Add an experimental DTrace audit provider, which allows users of DTrace to
instrument security event auditing rather than relying on conventional BSM
trail files or audit pipes:

- Add a set of per-event 'commit' probes, which provide access to
  particular auditable events at the time of commit in system-call return.
  These probes gain access to audit data via the in-kernel audit_record
  data structure, providing convenient access to system-call arguments and
  return values in a single probe.

- Add a set of per-event 'bsm' probes, which provide access to particular
  auditable events at the time of BSM record generation in the audit
  worker thread. These probes have access to the in-kernel audit_record
  data structure and BSM representation as would be written to a trail
  file or audit pipe -- i.e., asynchronously in the audit worker thread.

DTrace probe arguments consist of the name of the audit event (to support
future mechanisms of instrumenting multiple events via a single probe --
e.g., using classes), a pointer to the in-kernel audit record, and an
optional pointer to the BSM data and its length. For human convenience,
upper-case audit event names (AUE_...) are converted to lower case in
DTrace.

DTrace scripts can now cause additional audit-based data to be collected
on system calls, and inspect internal and BSM representations of the data.
They do not affect data captured in the audit trail or audit pipes
configured in the system. auditd(8) must be configured and running in
order to provide a database of event information, as well as other audit
configuration parameters (e.g., to capture command-line arguments or
environmental variables) for the provider to operate.

Reviewed by:	gnn, jonathan, markj
Sponsored by:	DARPA, AFRL
MFC after:	3 weeks
Differential Revision:	https://reviews.freebsd.org/D10149
2017-03-29 19:58:00 +00:00
Enji Cooper
193d9e768b sys/modules: normalize .CURDIR-relative paths to SRCTOP
This simplifies make output/logic

Tested with:	`cd sys/modules; make ALL_MODULES=` on amd64
MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 10:10:17 +00:00
George V. Neville-Neil
00bb01a40c Replace the implementation of DTrace's RAND subroutine for generating
low-quality random numbers with a modern implementation (xoroshiro128+)
that is capable of generating better quality randomness without compromising performance.

Submitted by:	Graeme Jenkinson
Reviewed by:	markj
MFC after:	2 weeks
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D9051
2017-02-03 22:26:19 +00:00
Ruslan Bukin
7804dd5212 Add full softfloat and hardfloat support for RISC-V.
Hardfloat is now default (use riscv64sf as TARGET_ARCH
for softfloat).

Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D8529
2016-11-16 15:21:32 +00:00
Ruslan Bukin
2ad1d09f16 o Add support for long double.
o Add support for latest RISC-V GNU toolchain.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-11-03 13:06:17 +00:00
Ruslan Bukin
01a62066c3 Revert r303911 "Remove extra -msoft-float flags settings."
This was not properly tested.
2016-08-11 13:42:31 +00:00
Ruslan Bukin
2d700cb557 Remove extra -msoft-float flags settings.
This helps to build firmware modules.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-08-10 13:32:27 +00:00
Ruslan Bukin
92bf0e5e2a Include FBT to modules build on RISC-V. 2016-07-29 12:30:33 +00:00
Ruslan Bukin
c760a23737 Build DTrace assym.o with -msoft-float flag for RISC-V so we have
correct flag in ELF file.

Sponsored by:	DARPA, AFRL
Sponsored by:	HEIF5
2016-07-28 13:18:10 +00:00
John Baldwin
2f99bcce1e Rename remaining linux32 symbols such as linux_sysent[] and
linux_syscallnames[] from linux_* to linux32_* to avoid conflicts with
linux64.ko.  While here, add support for linux64 binaries to systrace.
- Update NOPROTO entries in amd64/linux/syscalls.master to match the
  main table to fix systrace build.
- Add a special case for union l_semun arguments to the systrace
  generation.
- The systrace_linux32 module now only builds the systrace_linux32.ko.
  module on amd64.
- Add a new systrace_linux module that builds on both i386 and amd64.
  For i386 it builds the existing systrace_linux.ko.  For amd64 it
  builds a systrace_linux.ko for 64-bit binaries.

Reviewed by:	markj
Differential Revision:	https://reviews.freebsd.org/D3954
2015-10-22 21:28:20 +00:00
Mark Johnston
32cd0147fa Implement the lockstat provider using SDT(9) instead of the custom provider
in lockstat.ko. This means that lockstat probes now have typed arguments and
will utilize SDT probe hot-patching support when it arrives.

Reviewed by:	gnn
Differential Revision:	https://reviews.freebsd.org/D2993
2015-07-19 22:14:09 +00:00
Ruslan Bukin
b78ee15e9f First cut of DTrace for AArch64.
Reviewed by:	andrew, emaste
Sponsored by:	ARM Limited
Differential Revision:	https://reviews.freebsd.org/D2738
2015-07-01 15:51:11 +00:00
Ruslan Bukin
a1faeea684 Set a dependancy on fbt module for ARM. 2015-03-06 16:08:03 +00:00
George V. Neville-Neil
fcb5606706 Initial version of DTrace on ARM32.
Submitted by:	Howard Su based on work by Oleksandr Tymoshenko
Reviewed by:	ian, andrew, rpaulo, markj
2015-02-10 19:41:30 +00:00
Warner Losh
4906cdc8c5 Re-support CWARNFLAGS, lost when bsd.sys.mk was removed. Also, support
CWARNFALGS.$file centrally so we don't have to have it in all the
places. Remove a few warning flags that are no longer needed.
Also, always use -Wno-unknown-pragma to (hopefully temporarily) work
around #pragma ident in debug.h in the opensolaris code. Remove some
stale warning suppression that's no longer necessary.
2015-01-03 03:35:18 +00:00
Julian Elischer
3a3f4b0b02 Submitted by: rick Maclem
MFC after:	1 week
2014-12-27 15:17:59 +00:00
Rick Macklem
c15882f091 Remove the old NFS client and server from head,
which means that the NFSCLIENT and NFSSERVER
kernel options will no longer work. This commit
only removes the kernel components. Removal of
unused code in the user utilities will be done
later. This commit does not include an addition
to UPDATING, but that will be committed in a
few minutes.

Discussed on: freebsd-fs
2014-12-23 00:47:46 +00:00
Andriy Gapon
036a8c5dac remove opensolaris cyclic code, replace with high-precision callouts
In the old days callout(9) had 1 tick precision and that was inadequate
for some uses, e.g. DTrace profile module, so we had to emulate cyclic
API and behavior.  Now we can directly use callout(9) in the very few
places where cyclic was used.

Differential Revision:	https://reviews.freebsd.org/D1161
Reviewed by:	gnn, jhb, markj
MFC after:	2 weeks
2014-12-07 11:21:41 +00:00
Julian Elischer
b8ded963c9 Allow loading of dtraceall without nfscl if what you really wnat is nfsclient
Obtained from:	Panzura tree
MFC after:	1 week
2014-10-28 04:18:09 +00:00
Mark Johnston
266b4a78c2 Factor out the common code for function boundary tracing instead of
duplicating the entire implementation for both x86 and powerpc. This makes
it easier to add support for other architectures and has no functional
impact.

Phabric:	D613
Reviewed by:	gnn, jhibbits, rpaulo
Tested by:	jhibbits (powerpc)
MFC after:	2 weeks
2014-08-16 21:42:55 +00:00
Warner Losh
009a196247 Remove unnecessary inclusions of bsd.own.mk. 2014-08-04 22:34:12 +00:00
Rui Paulo
d18aa577d5 Copy strtolctype.h to sys/cddl/contrib/opensolaris/common/util to keep
the kernel self-contained.

Requested by:	jhb
2014-07-31 08:07:23 +00:00
Marcel Moolenaar
0327754b52 Avoid using ${.CURDIR} so that the module can be built from multiple
directories.

Obtained from:	Juniper Networks, Inc.
2014-07-26 17:24:58 +00:00
Pedro F. Giffuni
c6d712caf3 Revert r268007, and re-adapt MFV r260708:
4427 pid provider rejects probes with valid UTF-8 names

Use of u8_textprep.c required -Wno-cast-qual for powerpc.

MFC after:	2 weeks
2014-07-01 15:36:05 +00:00
Pedro F. Giffuni
f34dd28f7d Revert r267869:
MFV	r260708
4427 pid provider rejects probes with valid UTF-8 names

Use of u8_textprep.c broke the build on powerpc.

Reported by:	bz, rpaulo and tinderbox.
Pointyhat:	me
2014-06-28 19:59:12 +00:00
Rui Paulo
57c2423017 Move the -I of common/util to the proper place to fix the powerpc build.
MFC after:	2 weeks
2014-06-28 18:53:02 +00:00
Rui Paulo
63083de0a6 Redefine SUNW based on SYSDIR in an attempt to fix a build problem.
MFC after:	2 weeks
2014-06-27 22:38:42 +00:00
Rui Paulo
b1f9167f94 MFV illumos
4477 DTrace should speak JSON

MFC after:	2 weeks
2014-06-26 21:45:49 +00:00
Pedro F. Giffuni
af8bd6e468 MFV r260708
4427 pid provider rejects probes with valid UTF-8 names

This make use of Solaris' u8_validate() which we happen to
use since r185029 for ZFS.

Illumos Revision:	1444d846b126463eb1059a572ff114d51f7562e5

Reference:
https://www.illumos.org/issues/4427

Obtained from:	Illumos
MFC after:	2 weeks
2014-06-25 14:23:30 +00:00
Marcel Moolenaar
50dd852b28 Use SYSDIR to reference to the top of the sys hierarchy. Define SYSDIR
relative to .CURDIR if not already defined. This makes the makefiles
more readable but also more re-usable and adaptable.

Obtained from:	Juniper Networks, Inc.
2014-03-06 01:59:13 +00:00
Mark Johnston
0339a1c2b4 Move some files that are identical on i386 and amd64 to an x86 subdirectory
rather than keeping duplicate copies.

Discussed with:	avg
MFC after:	1 week
2014-02-27 01:04:35 +00:00
Attilio Rao
54366c0bd7 - For kernel compiled only with KDTRACE_HOOKS and not any lock debugging
option, unbreak the lock tracing release semantic by embedding
  calls to LOCKSTAT_PROFILE_RELEASE_LOCK() direclty in the inlined
  version of the releasing functions for mutex, rwlock and sxlock.
  Failing to do so skips the lockstat_probe_func invokation for
  unlocking.
- As part of the LOCKSTAT support is inlined in mutex operation, for
  kernel compiled without lock debugging options, potentially every
  consumer must be compiled including opt_kdtrace.h.
  Fix this by moving KDTRACE_HOOKS into opt_global.h and remove the
  dependency by opt_kdtrace.h for all files, as now only KDTRACE_FRAMES
  is linked there and it is only used as a compile-time stub [0].

[0] immediately shows some new bug as DTRACE-derived support for debug
in sfxge is broken and it was never really tested.  As it was not
including correctly opt_kdtrace.h before it was never enabled so it
was kept broken for a while.  Fix this by using a protection stub,
leaving sfxge driver authors the responsibility for fixing it
appropriately [1].

Sponsored by:	EMC / Isilon storage division
Discussed with:	rstone
[0] Reported by:	rstone
[1] Discussed with:	philip
2013-11-25 07:38:45 +00:00
Mark Johnston
2e1ae0b3e9 Redefine the io provider using the SDT(9) macros instead of doing everything
manually. This change has no functional impact.

Discussed with:	gnn
2013-10-24 02:39:07 +00:00
Justin Hibbits
30b318b92f Add fasttrap for PowerPC. This is the last piece of the dtrace/ppc puzzle.
It's incomplete, it doesn't contain full instruction emulation, but it should be
sufficient for most cases.

MFC after:	1 month
2013-10-15 15:00:29 +00:00
Justin Hibbits
cc117e2773 Fix some ppc64 dtrace bugs, and enable systrace_freebsd32 for ppc64. 2013-08-19 05:10:46 +00:00
Ulrich Spörlein
f1fe1d39e1 Fix make depend 2013-08-14 08:03:57 +00:00
Andriy Gapon
c319ea15f4 opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG
Do this by forcing inclusion of
sys/cddl/compat/opensolaris/sys/debug_compat.h
via -include option into all source files from OpenSolaris.
Note that this -include option must always be after -include opt_global.h.

Additionally, remove forced definition of DEBUG for some modules and fix
their build without DEBUG.

Also, meaning of DEBUG was overloaded to enable WITNESS support for some
OpenSolaris (primarily ZFS) locks.  Now this overloading is removed and
that use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS.

MFC after:	17 days
2013-08-06 15:51:56 +00:00
Andriy Gapon
5d7430f0a8 dtrace: fix compilation with gcc
Cowardly taking the easiest way and using -Wno-*

MFC after:	3 days
X-MFC with:	r253772
2013-08-06 13:55:39 +00:00