This moves the bulk of the geli support from lib386/biosdisk.c into a new
geli/gelidev.c which implements a devsw-type device whose dv_strategy()
function handles geli decryption. Support for all arches comes from moving
the taste-and-attach code to the devopen() function in libsa.
After opening any DEVT_DISK device, devopen() calls the new function
geli_probe_and_attach(), which will "attach" the geli code to the open_file
struct by creating a geli_devdesc instance to replace the disk_devdesc
instance in the open_file. That routes all IO for the device through the
geli code.
A new public geli_add_key() function is added, to allow arch/vendor-specific
code to add keys obtained from custom hardware or other sources.
With these changes, geli support will be compiled into all variations of
loader(8) on all arches because the default is WITH_LOADER_GELI.
Relnotes: yes
Sponsored by: Microchip Technology Inc
Differential Revision: https://reviews.freebsd.org/D15743
Add src.conf knob to disable the installation of /var/db/services.db
Default to leaving services.db in place, but allow the removal of the
file and its creation with a src.conf knob.
This file ends up being 2MB in size. For small systems this is a waste
of space but its a tradeoff.
Reviewed by: bdrewery
Differential Revision: https://reviews.freebsd.org/D9655
This is needed for -m32 support which is used in the kernel cloudabi32 module.
Tweak the style to make it easier to understand.
MFC after: 2 weeks
X-MFC-with: r335706
Reported by: Mark Millard
Sponsored by: Dell EMC
LLVM_TARGET_* will auto be set based on LLVM_TARGET_ALL and MK_CLANG.
If LLVM_TARGET_ALL is disabled, during a cross-build, then SYSTEM_COMPILER
and SYSTEM_LINKER are auto disabled.
This option should be used by users rather than the per-arch LLVM_TARGET
options as it is simpler to maintain for them should the supported
target list change.
MFC after: 2 weeks
Reviewed by: sbruno, dim
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D16020
This makes it possible, through src.conf(5) settings, to select which
LLVM targets you want to build during buildworld. The current list is:
* (WITH|WITHOUT)_LLVM_TARGET_AARCH64
* (WITH|WITHOUT)_LLVM_TARGET_ARM
* (WITH|WITHOUT)_LLVM_TARGET_MIPS
* (WITH|WITHOUT)_LLVM_TARGET_POWERPC
* (WITH|WITHOUT)_LLVM_TARGET_SPARC
* (WITH|WITHOUT)_LLVM_TARGET_X86
To not influence anything right now, all of these are on by default, in
situations where clang is enabled.
Selectively turning a few targets off manually should work. Turning on
only one target should work too, even if that target does not correspond
to the build architecture. (In that case, LLVM_NATIVE_ARCH will not be
defined, and you can only use the resulting clang executable for
cross-compiling.)
I performed a few measurements on one of the FreeBSD.org reference
machines, building clang from scratch, with all targets enabled, and
with only the x86 target enabled. The latter was ~12% faster in real
time (on a 32-core box), and ~14% faster in user time. For a full
buildworld the difference will probably be less pronounced, though.
Reviewed by: bdrewery
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D11077
This works similar to WITH_SYSTEM_COMPILER added in r300354. It only
supports lld via WITH_LLD_BOOTSTRAP.
When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools. If either check fails though then
libclang is built.
The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".
-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.
A new 'make test-system-linker' target is added to see the logic results.
Makefile.inc1:
CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
logic as done with SYSTEM_COMPILER. This also required moving where
bsd.linker.mk was read since XLD needs to be set before parsing it. This
creates a situation where src.opts.mk can not test LINKER_FEATURES or
add LLD_BOOTSTAP to BROKEN_OPTIONS.
Reviewed by: emaste (earlier version)
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15894
Implement MK_NVME now that the expression for where NVMe is
complicated. Default it to "yes" for x86 and powerpc64 and
no everywhere else. Use it in camcontrol to define WITH_NVME
for those platforms where we support nvme.
This should fix the newly introduced nvme files to camcontrol
which were building everywhere.
Pointy Hat To: imp
Sponsored by: Netflix
The migration to LLVM's lld linker has been in progress for quite some
time - about three years ago I opened an upstream LLVM meta-bug to track
issues using lld as FreeBSD's linker, and about 1.5 years ago requested
the first exp-run with lld as the system linker.
As of r327783 we enabled LLD_BOOTSTRAP by default on amd64, using lld as
the linker to link the kernel and world, but GNU ld was still installed
as /usr/bin/ld.
The vast majority of issues observed when building ports with lld as the
system linker have now been solved, so set LLD_IS_LD by default on amd64
and install lld as /usr/bin/ld. A small number of port failures remain
and these will be addressed in the near future.
Thanks to antoine@ for handling the exp-runs, krion@ for investigating
many port failures and adding LLD_UNSAFE or other fixes or workarounds,
and everyone who helped investigate, fix or tag ports.
PR: 214864 (exp-run)
Sponsored by: The FreeBSD Foundation
freebsd-update depends on phttpget from portsnap. We could move phttpget
out of portsnap and build it as long as WITHOUT_FREEBSD_UPDATE and
WITHOUT_PORTSNAP are not both set, but for now just make the dependency
explicit.
PR: 228220
Reported by: Dries Michiels
Sponsored by: The FreeBSD Foundation
It was an old TRE that had plenty of bugs and no performance gain over
regex(3). I disabled it by default in r323615, and there was some confusion
about what the knob does- likely due to poor naming on my part- to the tune
of "well, it sounds like it should speed things up" (mentioned by multiple
people).
To compound this, I have no intention of maintaining a second regex
implementation. If someone would like to step up and volunteer to maintain a
lean-and-mean implementation for grep, this is OK, but we have very few
volunteers to maintain even our primary regex implementation.
As with Clang, build our toolchain components by default when the host
compiler is capable of doing so, to make them available for testing and
experimentation.
Sponsored by: The FreeBSD Foundation
These have become unsorted from everything else. This is desync'd from
stable/11 due to some hand-merging that was done there, so the MFC of this
will look slightly different.
MFC after: 3 days
This is originally based on a patch from David Chisnall for soft-float
N64 but has since been updated to support O32, N32, and hard-float ABIs.
The soft-float O32, N32, and N64 support has been committed upstream.
The hard-float changes are still in review upstream.
Enable LLVM_LIBUNWIND on mips when building with a suitable (C+11-capable)
toolchain. This has been tested with external GCC for all ABIs and
O32 and N64 with clang.
Reviewed by: emaste
Obtained from: CheriBSD (original N64 patch)
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D14701
ConnectX-4/5 devices in mlx5core.
The dump is obtained by reading a predefined register map from the
non-destructive crspace, accessible by the vendor-specific PCIe
capability (VSC). The dump is stored in preallocated kernel memory and
managed by the mlx5tool(8), which communicates with the driver using a
character device node.
The utility allows to store the dump in format
<address> <value>
into a file, to reset the dump content, and to manually initiate the
dump.
A call to mlx5_fwdump() should be added at the places where a dump
must be fetched automatically. The most likely place is right before a
firmware reset request.
Submitted by: kib@
MFC after: 1 week
Sponsored by: Mellanox Technologies
LLD is a cross toolchain component. It shouldn't be built when
requesting a build without building a cross compiler.
(CROSS_COMPILER is somewhat unfortunately named; in any case, lld
should be treated as GNU binutils here.)
Submitted by: Dan McGregor <dan.mcgregor at usask.ca>
MFC after: 1 week
Makefile.${MACHINE_ARCH} and remove the now-empty files. Generate the
*32 directories on the necessary architectures (well, currently only
amd64) on the fly. Remove LOADER_EFI variable and co-locate it with
EFI.
Differential Review: https://reviews.freebsd.org/D14546
liblua glues the lua run time into the boot loader. It implements all
the runtime routines that lua expects. In addition, it has a few
standard 'C' headers that nueter various aspects of the LUA build that
are too specific to lua to be in libsa. Many refinements from the
original code to improve implementation and the number of included lua
libraries. Use int64_t for lua_Number. Have "/boot/lua" be the default
module path. Numerous cleanups from the original GSoC project,
including hacking libsa to allow lua to be built with only one change
outside luaconf.h.
Add the final bit of lua glue to bring in liblua and plug into the
multiple interpreter framework, previously committed.
Add LOADER_LUA option, currently off by default.
Presently, this is an experimental option. One must opt-in to using
this by defining WITH_LOADER_LUA and WITHOUT_FORTH. It's been
lightly tested, so keep a backup copy of your old loader handy.
The menu code, coming in the next commit, hasn't been exhaustively
tested. A LUA boot loader is 60k larger than a FORTH one, which is
80k larger than a no-interpreter one. Subtle changes in size
may tip things past some subtle limit (the binary is ~430k now
when built with LUA). A future version may offer coexistance.
Bump FreeBSD version to 1200058 to mark the milestone.
Pedro Souza's 2014 Summer of Code project. Rui Paulo, Pedro Arthur,
Zakary Nafziger and Wojciech A. Koszek also contributed. Warner Losh
reworked it extensively into its current form.
Obtained from: https://wiki.freebsd.org/SummerOfCode2014/LuaLoader
Sponsored by: Google Summer of Code
Relnotes: Yes
MFC After: 1 month
Differential Review: https://reviews.freebsd.org/D14295
Introduce WITH_/WITHOUT_LLVM_COV to match GCC's WITH_/WITHOUT_GCOV.
It is intended to provide a superset of the interface and functionality
of gcov.
It is enabled by default when building Clang, similarly to gcov and GCC.
This change moves one file in libllvm to be compiled unconditionally.
Previously it was included only when WITH_CLANG_EXTRAS was set, but the
complexity of a new special case for (CLANG_EXTRAS | LLVM_COV) is not
worth avoiding a tiny increase in build time.
Reviewed by: dim, imp
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D142645
There's a report of some regression in ports. Revert for now for an
exp run for this change in isolation (previous lld exp run also included
switching the linker used for ports to lld).
Also revert the src.conf.5 regeneration in r327824.
Reported by: antoine
Sponsored by: The FreeBSD Foundation
Akin to r327783 for amd64. lld has been usable for amd64 for quite some
time, but a couple of issues remained that affected i386. These were
recently addressed upstream in lld and merged into FreeBSD (r326831,
r326879, r326897, r326957), so we can now use ld.lld on i386 as well.
Similarly to amd64 this change enables lld only as the bootstrap linker
(used to link the kernel and userland libraries and executables), while
GNU ld.bfd is still installed as /usr/bin/ld and used for ports builds.
The ports collection is essentially ready to use lld as the system
linker for amd64, but many ports still have trouble with lld on i386,
because lld defaults to -ztext, disallowing relocations against readonly
segments. Thus switching the system linker (WITH_LLD_IS_LD) will happen
later on a per-arch basis.
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
For some time we have been planning to migrate to LLVM's lld linker.
Having a man page was the last blocking issue for using ld.lld to link
the base system kernel + userland, now addressed by r327770. Link the
kernel and userland libraries and binaries with ld.lld by default, for
additional test coverage.
This has been a long time in the making. On 2013-04-13 I submitted an
upstream tracking issue in LLVM PR 23214: [META] Using LLD as FreeBSD's
system linker. Since then 85 individual issues were identified, and
submitted as dependencies. These have been addressed along with two
and a half years of other lld development and improvement.
I'd like to express deep gratitude to upstream lld developers Rui
Ueyama, Rafael Espindola, George Rimar and Davide Italiano. They put in
substantial effort in addressing the issues we found affecting
FreeBSD/amd64.
To revert to using ld.bfd as the bootstrap linker, in /etc/src.conf set
WITHOUT_LLD_BOOTSTRAP=yes
If you need to set this, please follow up with a PR or post to the
freebsd-toolchain mailing list explaining how default WITH_LLD_BOOTSTRAP
failed for your use case.
Note that GNU ld.bfd is still installed as /usr/bin/ld, and will still
be used for linking ports. ld.lld can be installed as /usr/bin/ld by
setting in /etc/src.conf
WITH_LLD_IS_LLD=yes
A followup commit will set WITH_LLD_IS_LD by default, possibly after
Clang/LLVM/lld 6.0 is merged to FreeBSD.
Release notes: Yes
Sponsored by: The FreeBSD Foundation
temporary workaround. This fixes zfs booting generally, but breaks all
GELI booting by default. Add note to UPDATING to this effect. When the
GELI issues are resolved, this will be reverted.
Transition to WITH/WITHOUT_LOADER_GELI to flag support or not of GELI
in the boot loaders. Add HAVE_GELI so components can flag they need
support (since it's too large to include everywhere). Add temporary
warnings for the old forms to ease transition.
Also, update test script to build without GELI on x86.
Sponsored by: Netflix
Rename LOADER_FIREWIRE_SUPPORT to MK_LOADER_FIREWIRE. Only build
libfirewire when this is "yes". Add note to updating. Fix build script
to build this for x86 so the option doesn't decay. sparc64 supports
ZFS, so also build it MK_ZFS=no.
Sponsored by: Netflix
This fixes object files landing in the source tree in gnu/usr.bin/dtc
for GCC platforms.
We cannot reliably detect if an external compiler is used here, and the
default YES option does include GCC_BOOTSTRAP which implies that GCC may
be used for the build.
The problem manifests when not using an external compiler, and the host
compiler is clang. When a fresh build is done (no OBJDIR yet) the
'make obj' treewalk is done before 'make cross-tools', so
COMPILER_FEATURES at this point contains 'c++11' since the host compiler
was used for COMPILER_FEATURES. Once cross-tools builds the GCC
bootstrap compiler and then descends into 'make everything',
COMPILER_FEATURES no longer contains 'c++11' and MK_GPL_DTC defaults to
enabled. Now it builds in gnu/usr.bin/dtc without an OBJDIR preset and
drops files into the source tree.
The COMPILER_FEATURES check here is useful for knowing if we can *bootstrap*
C++11 things. Indeed we do bootstrap dtc as a build tool so it is
useful for enabling the BSD dtc for the build, but we end up needing the
GPL dtc for installation anyway.
Reviewed by: manu, emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12817
If they are still needed, you can find them in the net/bsdrcmds port.
This was proposed June, 20th and approved by various committers [1].
They have been marked as deprecated on CURRENT in r320644 [2] on July, 4th.
Both stable/11 and release/11.1 contain the deprecation notice (thanks to
allanjude@).
Note that ruptime(1)/rwho(1)/rwhod(8) were initially thought to be part of
rcmds but this was a mistake and those are therefore NOT removed.
[1] https://lists.freebsd.org/pipermail/freebsd-arch/2017-June/018239.html
[2] https://svnweb.freebsd.org/base?view=revision&revision=320644
Reviewed by: bapt, brooks
Differential Revision: https://reviews.freebsd.org/D12573
Make armv7 as a new MACHINE_ARCH.
Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.
Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).
Add armv7 to the universe build.
Differential Revision: https://reviews.freebsd.org/D12010
Add flag to the makefile to allow loader compilation as
Little-Endian 32-bit executable.
Usage:
make WITH_LOADER_FORCE_LE=yes -C sys/boot all
Submitted by: Wojciech Macek <wma@freebsd.org>
Reviewed by: imp, nwhitehorn
Obtained from: Semihalf
Sponsored by: QCM Technologies
Differential revision: https://reviews.freebsd.org/D12421
Start the phasing out of TRE by disabling it by default. r317254 introduced
a BSD_GREP_FASTMATCH knob (defaulting to on) for testing of bsdgrep with and
without TRE enabled. More bugs have cropped up since then, and
WITHOUT_BSD_GREP_FASTMATCH has shown in testing to be more stable than its
counterpart.
Approved by: emaste (mentor)
Differential Revision: https://reviews.freebsd.org/D12381
== Rationale ==
r295380 introduced "make check" and consolidated means for running
test code in an attempt to simplify running tests. One could either
install files/libraries/programs and run "make check", or run "make check"
with an explicit CHECKDIR, e.g., `make check CHECKDIR=$(make -V.OBJDIR)``.
One criticism that was received is that "make check" should be run with
the intent of making dev->test->commit easier, which means that the target
audience's workflow should be developers. One developer pattern available
in other opensource projects is to run test code from a developer sandbox,
instead of installing to a system.
== Method ==
This approach is slightly different from the standard approach, in the sense
that it builds and installs into a deterministic directory under .OBJDIR (as I call it,
the "sandbox"), then runs "make check" against that. In the event the test
run is successful, the deterministic directory is removed to save space.
== Approach ==
bsd.lib.mk, bsd.prog.mk:
To support this functionality, a new variable `HAS_TESTS` is being added.
HAS_TESTS enables appropriate behavior with bsd.lib.mk and bsd.prog.mk, as
follows:
- Add "make check" as an available target from the directory.
- Pass down appropriate variables via ${TESTS_ENV}, i.e.,
${TESTS_LD_LIBRARY_PATH} and ${TESTS_PATH}.
One should add "HAS_TESTS" to directories containing tests in them, e.g. from
bin/sh/Makefile,
HAS_TESTS=
SUBDIR.${MK_TESTS}+= tests
HAS_TESTS doesn't automatically add the tests subdirectory for flexibility
reasons.
bsd.opts.mk, src.opts.mk:
- The knob ${MK_MAKE_CHECK_USE_SANDBOX} has been added, both to explicitly
direct (internally) when to set a deterministic ${DESTDIR} and to also allow
users to disable this behavior globally, i.e., via src.conf.
- MK_TESTS has been promoted from src.opts.mk to bsd.opts.mk to leverage
syntactic sugar for having MK_TESTS be a dependency for
MK_MAKE_CHECK_USE_SANDBOX, but to also ensure that src.opts.mk isn't required
to use suite.test.mk (which is a dependency of bsd.test.mk).
suite.test.mk:
- beforecheck behavior (when MK_MAKE_CHECK_USE_SANDBOX is enabled) is modified
from a no-op to:
-- Build.
-- Run "make hierarchy" on the sandbox dir.
-- Install the tests/files to the sandbox dir.
- aftercheck behavior (when MK_MAKE_CHECK_USE_SANDBOX is enabled) is modified
from a no-op to:
-- Remove the sandbox dir.
Again, because the dependency order set in bsd.test.mk is
beforecheck -> check -> aftercheck, "make check" will not be run unless
"beforecheck" completes successfully, and "aftercheck" will not be run unless
"beforecheck" and "check" complete successfully.
== Caveats ==
- This target must either be run with MK_INSTALL_AS_USER or as root. Otherwise
it will fail when running "make install" as the default user/group for many
makefiles when calling INSTALL is root/wheel.
- This target must be run from a suitable top-level directory. For example,
running tests from `tests/sys/fs/tmpfs` won't work, but `tests/sys/fs` will,
because `tests/sys/fs/tmpfs` relies on files installed by `tests/sys/fs`.
- Running MK_INSTALL_AS_USER may introduce determinism issues. However, using
it could identify deficiences in tests in terms of needing to be run as
root, which are not properly articulated in the test requirements.
- The doesn't negate the need for running "make installworld" and
"make checkworld", etc. Again, this just is intended to simplify the
dev->test->commit workflow.
== Cleanup done ==
- CHECKDIR is removed; one can use "MK_MAKE_CHECK_USE_SANDBOX=no" to enable
"legacy" (r295380) behavior.
MFC after: 2 months
Relnotes: yes (CHECKDIR removed; "make check" behavior changed)
Requested by: jhb
Reviewed by: arch (silence), testing (silence)
Differential Revision: D11905
The assembly in sys/mips/include/profile.h will only work for o32 ABI.
Submitted by: Alexander Richardson
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D11950
`MK_ZONEINFO_LEAPSECONDS_SUPPORT == yes` and
`MK_ZONEINFO_OLD_TIMEZONES_SUPPORT == yes`.
Keep `LEAPSECONDS` and `OLDTIMEZONES` for backwards compatibility,
but print out a warning notifying users that they should use the new
variables, in an effort to migrate them to the variables. This is being
done mostly for automated build tools, etc, that might rely on these
variables being set. The variables will be removed in the future on
^/head, e.g., after ^/stable/12 is cut.
MFC after: 1 month
Relnotes: yes
Reviewed by: bdrewery
Differential Revision: D11376
The BSDL dtc has grown the needed features (overlays mostly) and is able to
compile all of our base DTS.
You can use WITH_GPL_DTC is you need the GPL one or DTC= in make.conf(5)
to specify an alternate location for the compiler to use.
Discussed with: emaste, imp
