There's no way currently to automatically prevent the bad .OBJDIR from being
created but it can at least be prevented from being used. Passing
WITHOUT_AUTO_OBJ=yes or MK_AUTO_OBJ=no or -DNO_OBJ in will prevent it.
Reported by: jeffr
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12989
NO_OBJ has a very specific meaning in sub-directories in that no object
directory will be made. If a user wanted to skip the 'make obj' phase then
passing -DNO_OBJ would break all sub-directories from building properly. Using
NO_OBJ internally also causes issue with NO_OBJ handling being added in
share/mk/bsd.init.mk soon.
Sponsored by: Dell EMC Isilon
Make armv7 as a new MACHINE_ARCH.
Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.
Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).
Add armv7 to the universe build.
Differential Revision: https://reviews.freebsd.org/D12010
7431 ZFS Channel Programs
illumos/illumos-gate@dfc115332cdfc115332chttps://www.illumos.org/issues/7431
ZFS channel programs (ZCP) adds support for performing compound ZFS
administrative actions via Lua scripts in a sandboxed environment (with time
and memory limits).
This initial commit includes both base support for running ZCP scripts, and a
small initial library of API calls which support getting properties and
listing, destroying, and promoting datasets.
Testing: in addition to the included unit tests, channel programs have been in
use at Delphix for several months for batch destroying filesystems. The
dsl_destroy_snaps_nvl() call has also been replaced with
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Approved by: Garrett D'Amore <garrett@damore.org>
Author: Chris Williamson <chris.williamson@delphix.com>
8552 ZFS LUA code uses floating point math
illumos/illumos-gate@916c8d8811916c8d8811https://www.illumos.org/issues/8552
In the LUA interpreter used by "zfs program", the lua format() function
accidentally includes support for '%f' and friends, which can cause compilation
problems when building on platforms that don't support floating-point math in
the kernel (e.g. sparc). Support for '%f' friends (%f %e %E %g %G) should be
removed, since there's no way to supply a floating-point value anyway (all
numbers in ZFS LUA are int64_t's).
Reviewed by: Yuri Pankov <yuripv@gmx.com>
Reviewed by: Igor Kozhukhov <igor@dilos.org>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
8590 memory leak in dsl_destroy_snapshots_nvl()
illumos/illumos-gate@e6ab4525d1e6ab4525d1https://www.illumos.org/issues/8590
In dsl_destroy_snapshots_nvl(), "snaps_normalized" is not freed after it is
added to "arg".
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Steve Gonczi <steve.gonczi@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Approved by: Dan McDonald <danmcd@joyent.com>
Author: Matthew Ahrens <mahrens@delphix.com>
FreeBSD notes:
- zfs-program.8 manual page is taken almost as is from the vendor repository,
no FreeBSD-ification done
- fixed multiple instances of NULL being used where an integer is expected
- replaced ETIME and ECHRNG with ETIMEDOUT and EDOM respectively
This commit adds a modified version of Lua 5.2.4 under
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua, mirroring the
upstream. See README.zfs in that directory for the description of Lua
customizations.
See zfs-program.8 on how to use the new feature.
MFC after: 5 weeks
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D12528
Without this change, modules will match the default compiler
configuration which may not be the same as the kernel values.
Reviewed by: imp
Obtained from: CheriBSD
MFC after: 2 weeks
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D11633
A Build-ID is an identifier generated at link time to uniquely identify
ELF binaries. It allows efficient confirmation that an executable or
shared library and a corresponding standalone debuginfo file match.
(Otherwise, a checksum of the debuginfo file must be calculated when
opening it in a debugger.)
The FreeBSD base system includes GNU bfd ld 2.17.50 as the linker for
architectures other than arm64. Build-ID support was added to bfd ld
shortly after that version, so was not previously available to us.
We can now start making use of Build-ID as we migrate to using lld or
bfd ld from ports, conditionally enabled based on the LINKER_TYPE and
LINKER_VERSION make variables added in r320244 and subsequent commits.
Reviewed by: dim
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D11314
Previously the linker emulation was only passed when building binary
objects for firmware modules. This change always passes the desired
output format for kernel modules and kernels rather than requiring the
toolchain's default output format to match the desired output format.
This in turn permits use of external toolchains whose default output
format does not match the desired output format.
Reviewed by: imp, emaste
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D10085
Recognize new MACHINE_ARCH names now as we have added hardfloat support.
Switch JZ4780 to mipselhf and remove all uses of TARGET_ARCH in kernel
.mk files.
Reviewed by: adrian
Differential Revision: https://reviews.freebsd.org/D8989
While there, make param.h guess proper MACHINE_ARCH on hardfloat targets
correctly as well, so tools like bmake can get their defaults right.
This does not help the kernel case, since we compile them with forced
-msoft-float and need to override an incorrect guess by param.h.
Reviewed by: br
Differential Revision: https://reviews.freebsd.org/D8574
build can break when different source files create the same target
files (case-insensitivity speaking). This is the case for object
files compiled with -fpic and shared libraries. The former uses
an extension of ".So", and the latter an extension ".so". Rename
shared object files from *.So to *.pico to match what NetBSD does.
See also r305855
MFC after: 1 month
Sponsored by: Bracket Computing
Differential Revision: https://reviews.freebsd.org/D7906
This is in preparation for linking with LLVM's lld, which does not have
a compiled-in default output emulation. lld requires that it is
specified via the -m option, or obtained from the object file(s) being
linked.
This will also allow all build targets to share a common linker binary.
Reviewed by: imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D7837
This breaks cross-building with WITH_META_MODE since it will rebuild
'build-tools' during the 'everything' phase.
A more proper fix is coming to bmake to implicitly require .META unless
.NOMETA (and other restrictions) are in place.
Adding .META to targets-to-build will ensure that they will rebuild if there
is no .meta file.
Adding it to all SUFFIXES and objects ensures that at least objects will
rebuild if there is no .meta file.
This will be reverted if bmake's behavior changes to rebuild on missing .meta
files.
Sponsored by: EMC / Isilon Storage Division
This allows using META_MODE directly from the kernel build directory.
This also allows removing a hack from the DIRDEPS_BUILD kernel target.
Sponsored by: EMC / Isilon Storage Division
MD_ROOT_SIZE and embed_mfs.sh were basically retired as part of
https://reviews.freebsd.org/D2903 .
However, when building a kernel with 'options MD_ROOT_SIZE' specified, this
results in a non-working MFS, as within sys/dev/md/md.c we fall within the
wrong # ifdef.
This patch implements the following:
* Allow kernels to be built without the MD_ROOT_SIZE option, which results
in a kernel built as per D2903.
* Allow kernels to be built with the MD_ROOT_SIZE option, which results
in a kernel built similarly to the pre-D2903 way, with the following
differences:
* The MFS is now put in a separate section within the kernel (oldmfs,
so it differs from the mfs section introduced by D2903).
* embed_mfs.sh is changed, so it looks up the oldmfs section within the
kernel, gets its size and offset, sees if the MFS will fit within the
allocated oldmfs section and only if all is well does a dd of the MFS
image into the kernel.
Submitted by: Stanislav Galabov <sgalabov@gmail.com>
Reviewed by: brooks, imp
Differential Revision: https://reviews.freebsd.org/D5093
This is the final step required allowing to compile and to run RISC-V
kernel and userland from HEAD.
RISC-V is a completely open ISA that is freely available to academia
and industry.
Thanks to all the people involved! Special thanks to Andrew Turner,
David Chisnall, Ed Maste, Konstantin Belousov, John Baldwin and
Arun Thomas for their help.
Thanks to Robert Watson for organizing this project.
This project sponsored by UK Higher Education Innovation Fund (HEIF5) and
DARPA CTSRD project at the University of Cambridge Computer Laboratory.
FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv
Reviewed by: andrew, emaste, kib
Relnotes: Yes
Sponsored by: DARPA, AFRL
Sponsored by: HEIF5
Differential Revision: https://reviews.freebsd.org/D4982
changes. Use the list of MFILES found by find to identify the set of
possible auto-generated files and add the intersection of this set and
SRCS to CLEANFILES.
Submitted by: imp (previous version), sbruno
Differential Revision: https://reviews.freebsd.org/D4336
ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not
used to achieve a safe pseudo-incremental build. This is explained in
more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the
cache, but with a full cache saves 30-50% in build times. When combined with
the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the
resulting dependency file, which it does not do when using mkdep(1)/'CC
-E'. Stats are provided at the end of this message.
This removes the need to modify /etc/make.conf with the CC:= and CXX:=
lines which conflicted with external compiler support [3] (causing the
bootstrap compiler to not be built which lead to obscure failures [4]),
incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid
Clang errors with parenthesis, and did not work with META_MODE.
The option name was picked to match the existing option in ports. This
feature is available for both in-src and out-of-src builds that use
/usr/share/mk.
Linking, assembly compiles, and pre-processing avoid using ccache since it is
only overhead. ccache does nothing special in these modes, although there is
no harm in calling it for them.
CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap
compiler to hash the content of the compiler binary to determine if it
should be a cache miss. For external compilers the 'mtime' option is used
as it is more efficient and likely to be correct. Future work may optimize the
'content' check using the same checks as whether a bootstrap compiler is needed
to be built.
The CCACHE_CPP2 pessimization is currently default in our devel/ccache
port due to Clang requiring it. Clang's -Wparentheses-equality,
-Wtautological-compare, and -Wself-assign warnings do not mix well with
compiling already-pre-processed code that may have expanded macros that
trigger the warnings. GCC has so far not had this issue so it is allowed to
disable the CCACHE_CPP2 default in our port.
Sharing a cache between multiple checkouts, or systems, is explained in
the ccache manual. Sharing a cache over NFS would likely not be worth
it, but syncing cache directories between systems may be useful for an
organization. There is also a memcached backend available [5]. Due to using
an object directory outside of the source directory though you will need to
ensure that both are in the same prefix and all users use the same layout. A
possible working layout is as follows:
Source: /some/prefix/src1
Source: /some/prefix/src2
Source: /some/prefix/src3
Objdir: /some/prefix/obj
Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj'
This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace
all absolute paths to be relative. Using something like this is required due
to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache
adds into its hash for the object without CCACHE_BASEDIR.
distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc.
I have not personally tested this and assume it will not mix well with
using the bootstrap compiler.
The cache from buildworld can be reused in a subdir by first running
'make buildenv' (from r290424).
Note that the cache is currently different depending on whether -j is
used or not due to ccache enabling -fdiagnostics-color automatically if
stderr is a TTY, which bmake only does if not using -j.
The system I used for testing was:
WITNESS
Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes
DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log.
The arc was fully populated with src tree files and ccache objects.
RAM: 76GiB
CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz
2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16
The WITH_FAST_DEPEND feature was used for comparison here as well to show
the dramatic time savings with a full cache.
buildworld:
x buildworld-before
+ buildworld-ccache-empty
* buildworld-ccache-full
% buildworld-ccache-full-fastdep
# buildworld-fastdep
+-------------------------------------------------------------------------------+
|% * # +|
|% * # +|
|% * # xxx +|
| |A |
| A|
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 3744.13 3794.31 3752.25 3763.5633 26.935139
+ 3 4519 4525.04 4520.73 4521.59 3.1104823
Difference at 95.0% confidence
758.027 +/- 43.4565
20.1412% +/- 1.15466%
(Student's t, pooled s = 19.1726)
* 3 1823.08 1827.2 1825.62 1825.3 2.0785572
Difference at 95.0% confidence
-1938.26 +/- 43.298
-51.5007% +/- 1.15045%
(Student's t, pooled s = 19.1026)
% 3 1266.96 1279.37 1270.47 1272.2667 6.3971113
Difference at 95.0% confidence
-2491.3 +/- 44.3704
-66.1952% +/- 1.17895%
(Student's t, pooled s = 19.5758)
# 3 3153.34 3155.16 3154.2 3154.2333 0.91045776
Difference at 95.0% confidence
-609.33 +/- 43.1943
-16.1902% +/- 1.1477%
(Student's t, pooled s = 19.0569)
buildkernel:
x buildkernel-before
+ buildkernel-ccache-empty
* buildkernel-ccache-empty-fastdep
% buildkernel-ccache-full
# buildkernel-ccache-full-fastdep
@ buildkernel-fastdep
+-------------------------------------------------------------------------------+
|# @ % * |
|# @ % * x + |
|# @ % * xx ++|
| MA |
| MA|
| A |
| A |
|A |
| A |
+-------------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 3 571.57 573.94 571.79 572.43333 1.3094401
+ 3 727.97 731.91 728.06 729.31333 2.2492295
Difference at 95.0% confidence
156.88 +/- 4.17129
27.4058% +/- 0.728695%
(Student's t, pooled s = 1.84034)
* 3 527.1 528.29 528.08 527.82333 0.63516402
Difference at 95.0% confidence
-44.61 +/- 2.33254
-7.79305% +/- 0.407478%
(Student's t, pooled s = 1.02909)
% 3 400.4 401.05 400.62 400.69 0.3306055
Difference at 95.0% confidence
-171.743 +/- 2.16453
-30.0023% +/- 0.378128%
(Student's t, pooled s = 0.954969)
# 3 201.94 203.34 202.28 202.52 0.73020545
Difference at 95.0% confidence
-369.913 +/- 2.40293
-64.6212% +/- 0.419774%
(Student's t, pooled s = 1.06015)
@ 3 369.12 370.57 369.3 369.66333 0.79033748
Difference at 95.0% confidence
-202.77 +/- 2.45131
-35.4225% +/- 0.428227%
(Student's t, pooled s = 1.0815)
[1] https://ccache.samba.org/performance.html
[2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html
[3] https://reviews.freebsd.org/D3484
[5] https://github.com/jrosdahl/ccache/pull/30
PR: 182944 [4]
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
Relnotes: yes
- Move all files related to the LinuxKPI into sys/compat/linuxkpi and
its subfolders.
- Update sys/conf/files and some Makefiles to use new file locations.
- Added description of COMPAT_LINUXKPI to sys/conf/NOTES which in turn
adds the LinuxKPI to all LINT builds.
- The LinuxKPI can be added to the kernel by setting the
COMPAT_LINUXKPI option. The OFED kernel option no longer builds the
LinuxKPI into the kernel. This was done to keep the build rules for
the LinuxKPI in sys/conf/files simple.
- Extend the LinuxKPI module to include support for USB by moving the
Linux USB compat from usb.ko to linuxkpi.ko.
- Bump the FreeBSD_version.
- A universe kernel build has been done.
Reviewed by: np @ (cxgb and cxgbe related changes only)
Sponsored by: Mellanox Technologies
root disk. The embedded image is linked into the kernel in the .mfs
section.
Add rules and variables to kern.pre.mk and kern.post.mk that handle the
linking of the image. First objcopy is used to generate an object file.
Then, the object file is linked into the kernel.
Submitted by: Steve Kiernan <stevek@juniper.net>
Reviewed by: brooks@
Obtained from: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D2903
make the find it does extremely expensive, so compute it only
once. Also make sure the 'traditional' module building method works at
the expense of a bit of duplicated code.
This will require for AArch64 as we dont have modules yet.
Sponsored by: HEIF5
Sponsored by: ARM Ltd.
Differential Revision: https://reviews.freebsd.org/D1997
$M should be the kernel machine src directory, ${MACHINE}. In most cases
${MACHINE} and ${MACHINE_CPUARCH} are the same, but this is not true for
pc98 and arm64.
It appears we previously set M=${MACHINE_CPUARCH} as a workaround to
accommodate pc98, where MACHINE_CPUARCH is pc98 but it uses
sys/i386/i386/genassym.c.
arm64 relies on this being set correctly, so update $M and add explicit
workarounds for pc98.
Differential Revision: https://reviews.freebsd.org/D2307
Reviewed by: andrew, imp
Sponsored by: The FreeBSD Foundation
discontinued by its initial authors. In FreeBSD the code was already
slightly edited during the pf(4) SMP project. It is about to be edited
more in the projects/ifnet. Moving out of contrib also allows to remove
several hacks to the make glue.
Reviewed by: net@
this option from all modules that enable it theirselves.
In C mode -fms-extensions option enables anonymous structs and unions,
allowing us to use this C11 feature in kernel. Of course, clang supports
it without any extra options.
Reviewed by: dim
I discovered this while working on llvm/lld and realized export-dynamic
only supported --. Although upstream will eventually grow to support
both - and --, switch this in our build system, because GNU ld supports
both modes, and because there's some hope lld will become the default linker
for FreeBSD in the future.
Discussed with: emaste, rdivacky
get the tail part of the path. We can now build kernels the
old-fashioned way on FreeBSD 9.x and 10.x on at least amd64 using
clang 3.3, 3.4 or gcc 4.2.1 (though with the latter you need
WITHOUT_MODULES="aesni vmm cxgbe" due to various issues with
gcc 4.2.1).
has support for the .codeXX directives). However, it is desirable, for
a time, to allow kernels to be built with clang 3.4. Historically, it
has been advantageous to allow stable X-1 to build kernels the old
way (so long as the impact of doing so is small), and this restores
that ability.
Also, centralize the addition of ${ASM_CFLAGS.${.IMPSRC}}, place it in
kern.mk rather than kern.pre.mk so that all modules can benefit, and
give the same treatment to CFLAGS in kern.mk as well.
the place where the C dialect is selected. Have a fairly long list
of newly requires warning suppression for clang 3.5.0, also
centralized in kern.mk. Survive the fallout of the removal of
bsd.sys.mk from bsd.kmod.mk.
src/sys and the rest of the tree for builds.
o eliminate including bsd.mkopts.mk for the moment in kern.opts.mk
o No need to include src.opts.mk at all anymore. The reasons for it
are now coverted in sys.mk and src.sys.mk.
