This is required to make use of KERN_TLS
Reviewed by: jhb
Sponsored by: Ampere Computing
Submitted by: Klara, Inc.
Differential Revision: https://reviews.freebsd.org/D28405
This merges upstream patches from OpenSSL's master branch to add
KTLS infrastructure for TLS 1.0-1.3 including both RX and TX
offload and SSL_sendfile support on both Linux and FreeBSD.
Note that TLS 1.3 only supports TX offload.
A new WITH/WITHOUT_OPENSSL_KTLS determines if OpenSSL is built with
KTLS support. It defaults to enabled on amd64 and disabled on all
other architectures.
Reviewed by: jkim (earlier version)
Approved by: secteam
Obtained from: OpenSSL (patches from master)
MFC after: 1 week
Relnotes: yes
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D28273
WITHOUT_LIBTHR has been broken for a little over five years now, since the
xz 5.2.0 update introduced a hard liblzma dependency on libthr, and building
a useful system without threading support is becoming increasingly more
difficult.
Additionally, in the five plus years that it's been broken more reverse
dependencies have cropped up in libzstd, libsqlite3, and libcrypto (among
others) that make it more and more difficult to reconcile the effort needed
to fix these options.
Remove the broken options.
PR: 252760
Reviewed by: brooks, emaste, kib
Differential Revision: https://reviews.freebsd.org/D28263
This reverts commit bd72252aace382921840ddbceea712b96f4ad242.
The commit at hand breaks the build for all mips targets and does not
have a one-liner fix.
make[5]: "/usr/src/share/mk/sys.mk" line 169: Malformed conditional (${MACHINE_CPUARCH} == "mips" && ${COMPILER_TYPE} == "gcc")
Now that I have -head fitting in 8MB of flash again, I can test
out freebsd-head on my home AP test setup. Unfortunately,
the introduction of -O2 in r366664 causes the following infinite
loop shortly after boot:
------
MAP: No valid partition found at map/rootfs.uzip
Warning: no time-of-day clock registered, system time will not be set accurately
start_init: trying /sbin/init
BAD_PAGE_FAULT: pid 1 tid 100001 (init), uid 0: pc 0x4042c320 got a read fault (type 0x2) at 0x2e3a0
Trapframe Register Dump:
zero: 0 at: 0 v0: 0 v1: 0
a0: 0x1af34 a1: 0 a2: 0 a3: 0x7fffeff0
t0: 0 t1: 0 t2: 0 t3: 0
t4: 0 t5: 0 t6: 0 t7: 0
t8: 0 t9: 0x152e8 s0: 0x7fffee84 s1: 0
s2: 0 s3: 0 s4: 0 s5: 0
s6: 0 s7: 0 k0: 0 k1: 0
gp: 0x362c0 sp: 0x7fffedf0 s8: 0 ra: 0x40417df0
sr: 0xf413 mullo: 0 mulhi: 0 badvaddr: 0x2e3a0
cause: 0xffffffff80000008 pc: 0x4042c31c
Page table info for pc address 0x4042c320: pde = 0x80712000, pte = 0xa002065a
Dumping 4 words starting at pc address 0x4042c320:
8f9980e0 80820000 10400067 00809825
Page table info for bad address 0x2e3a0: pde = 0, pte = 0
------
I'm not yet sure why, but until I figure it out with the mips64/cheri
folk this should be reverted.
This should only use -O on GCC generated code for MIPS platforms.
Tested:
* QCA934x (mips74k) - WDR-3600/WDR-4300 APs
Differential Revision: https://reviews.freebsd.org/D28122
which(1) accepts both relative/absolute paths as well as lone binary
names. Set KYUA to kyua and use which(1) to confirm that it can find one;
if it cannot, just advise the user to set KYUA directly to the kyua binary
rather than assuming a relative location from LOCALBASE.
This allows `make check` to be operated with the version of kyua in base
without losing the flexibility of specifying another one.
ngie@ notes that the original intention was to avoid redundant $PATH lookups
and improve the determinism of the target. A future change will likely push
us back to this state, perhaps in the form of reverting this entirely and
just switching to using kyua in base. Accepting any in $PATH should be
considered a transitional move, at least until it's declared otherwise,
since kyua was only semi-recently added to base.
Reviewed-by: brooks, emaste, lwhsu, ngie
Differential-Revision: https://reviews.freebsd.org/D28045
is enabled.
This builds wpa_supplicant / hostpad using internal encryption routines
rather than using libcrypt.
This has been supported in wpa for years now, however since we use
local makefiles for this, we bitrotted dependencies and configuration
options.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D27958
Currently only libexec/rtld-elf32 uses internal LIBC_NOSSP_PIC during
the build but it gets it directly from the objdir rather than a sysroot.
For example, /usr/obj/usr/src/amd64.amd64/obj-lib32/lib/libc/libc_nossp_pic.a.
We don't stage lib32 libraries in WORLDTMP/usr/lib32 and doing so doesn't
buy much. If we want to use a staged lib32 library then we need to look in
LIBCOMPATTMP where they were staged. For example if LIBC_PIC were wanted then
look for /usr/obj/usr/src/amd64.amd64/obj-lib32/tmp/usr/lib32/libc_pic.a.
Reported by: rlibby
Reviewed by: rlibby
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D27648
As suggested in D27598. This also supports MK_WERROR.clang=no and
MK_WERROR.gcc=no to support the existing NO_WERROR.<compiler> uses.
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D27601
Only keep the widechar version of ncurses as libncursesw.so.9
Keep the old name to avoid breaking the ABI compatibility (the non
widechar version libncurses.so.9 is not binary compatible with
libncursesw.so.9) since all ports and base are already only linking
against the widechar version we can simply remove libncurses.so.9
Since the .9 version only lived in the dev branch and never ended in a
release, it is simply removed and not added to any binary compat
package.
Add symlinks to keep build time compatibility for anyone linking against
-lncurses
With this patch if a Makefile is using the INCSGROUPS mechanisms it can
override the default package for specific includes files using
GROUPPACKAGE= mynewpackage
While here add a few comments after endif/endfor so it's easier to read.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D27611
Otherwise we loose the info as we use bsd.dirs.mk for creating directories.
Reviewed by: bapt
Differential Revision: https://reviews.freebsd.org/D27712
Unconditionally install bsdgrep as grep, bootstrap or not. Remove all
build glue and stop installing both gnugrep and libgnuregex now that
all consumers of the latter are gone.
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D27732
As discussed on -current, -stable, -toolchain, and with jhb@ and imp@,
disable the obsolete in-tree GDB 6.1.1 by default. This was kept only
to provide kgdb for the crashinfo tool, but is long-obsolete, does not
support all architectures that FreeBSD does, and held back other work
(such as forcing the use of DWARF2 for kernel debug).
Crashinfo will use kgdb from the gdb package or devel/gdb port, and will
privde a message referencing those if no kgdb is found.
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
This has been years in the making, and we all knew it was bound to happen
some day. Switch to the BSDL grep implementation now that it's been a
little more thoroughly tested and theoretically supports all of the
extensions that gnugrep in base had with our libregex(3).
Folks shouldn't really notice much from this update; bsdgrep is slower than
gnugrep, but this is currently the price to pay for fewer bugs. Those
dissatisfied with the speed of grep and in need of a faster implementation
should check out what textproc/ripgrep and textproc/the_silver_searcher
can do for them.
I have some WIP to make bsdgrep faster, but do not consider it a blocker
when compared to the pros of switching now (aforementioned bugs, licensing).
PR: 228798 (exp-run)
PR: 128645, 156704, 166842, 166862, 180937, 193835, 201650
PR: 232565, 242308, 246000, 251081, 191086, 194397
Relnotes: yes, please
This was introduced and then disabled by default primarily to avoid dealing
with bugs in libgnuregex. rS363823 switched to using libregex for it, so
let's just rip the option out now so we can make sure we're getting tested
with libregex via bsdgrep.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D27476
We are seeing regular build failures due to libc.so being installed again and
another parallel make job tries to read the partially written libc.so at the
same time. When building with -j32 or higher this almost always happens on
the first clean build (subsequent incremental builds always work fine).
Using -S should "fix" the "section header table goes past the end of the
file: e_shoff = 0x..." errors that have started to plague our builds.
We originally thought this only affected CheriBSD, but I just got the same
error while building the latest upstream FreeBSD.
The real fix should be to not install libraries twice, but until then this
workaround is needed.
Original patch by jrtc27@, I only made some minor changes to the comment.
Obtained from: CheriBSD (49837edd3e)
Reviewed By: markj, bdrewery
Differential Revision: https://reviews.freebsd.org/D27102
Check for the variable SUBDIR. and error as it usually means someone
forgot to include src.opts.mk.
This guard from CheriBSD found the bugs in r367655 and r367728.
Reviewed by: bdrewery, arichardson
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D27211
WITH_INIT_ALL_ZERO and WITH_INIT_ALL_PATTERN are mutually exclusive.
The .error when they were both set broke makeman so demote it to a
warning (and presumably the compiler will fail on an error later on).
We could improve this to make one take precedence but this is sufficient
for now.
MFC with: r367577
Sponsored by: The FreeBSD Foundation
There are two options:
- WITH_INIT_ALL_ZERO: Zero all variables on the stack.
- WITH_INIT_ALL_PATTERN: Initialize variables with well-defined patterns.
The exact pattern are a compiler implementation detail and vary by type.
They are somewhat documented in the LLVM commit message:
https://reviews.llvm.org/rL349442
I've used WITH_INIT_ALL_* to match Microsoft's InitAll feature rather
than naming them after the LLVM specific compiler flags.
In a range of consumer products, options like these are used in
both debug and production builds with debugs builds using patterns
(intended to provoke crashes on use of uninitialized values) and
production using zeros (deemed more likely to lead to harmless
misbehavior or NULL-pointer dereferences).
Reviewed by: emaste
Obtained from: CheriBSD
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D27131
LLVM's demangler supports more modern C++ constructs such as lambdas and
unnamed types, and is actively maintained. The command line tool is
usable as a drop-in replacement for GNU c++filt, or elftoolchain's
cxxfilt. The latter is still available by using WITHOUT_LLVM_CXXFILT, if
needed.
PR: 250702
MFC after: 2 weeks
Since elftoolchain's cxxfilt is rather far behind on features, and we
ran into several bugs, add an option to use llvm-cxxfilt as an drop-in
replacement.
It supports the same options as elftoolchain cxxfilt, though it doesn't
have support for old ARM (C++ Annotated Reference Manual, not the CPU)
and GNU v2 manglings. But these are irrelevant in 2020.
Note: as we already compile the required libraries as part of libllvm,
this will not add any significant build time either.
PR: 250702
Reviewed by: emaste, yuri
Differential Revision: https://reviews.freebsd.org/D27071
MFC after: 2 weeks
Until clang 11 that was equivalent to -O2, but clang changed it to -O1 so
generated MIPS code will now be unnecessarily slow. It also removes a weird
special case from sys.mk.
This is similar to the D26471 change for debug kernels and should not change
anything since everything was previously building MIPS code at -O2 until the
clang 11 update.
Reviewed By: trasz
Differential Revision: https://reviews.freebsd.org/D26749
It appears this was changed from ln to use install in rS245752. I noticed
this because my buildenv was setting INSTALL=install -U -M //METALOG
and then these links fail to be created with the following error:
install: open //METALOG: Permission denied
Reviewed By: brooks
Differential Revision: https://reviews.freebsd.org/D26618
Add a wrapping script to use ATF to run tests written with Googletest
one by one. This helps locating and tracking the failing case in CI easier.
This is a temporarily solution while Googletest support in Kyua is developing.
We will revert this once Kyua+Googletest integration is ready.
Reviewed by: emaste
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D25896
VirtFS allows sharing an arbitrary directory tree between bhyve virtual
machine and the host. Current implementation has a fairly complete support
for 9P2000.L protocol, except for the extended attribute support. It has
been verified to work with the qemu-kvm hypervisor.
Reviewed by: rgrimes, emaste, jhb, trasz
Approved by: trasz (mentor)
MFC after: 1 month
Relnotes: yes
Sponsored by: Conclusive Engineering (development), vStack.com (funding)
Differential Revision: https://reviews.freebsd.org/D10335
Clang 12 warns about passing a path to -fuse-ld and -Werror makes that
an error preventing building world without this change.
Reviewed by: arichardson, emaste
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D26591
This is the initial set up for PowerPC64LE.
The current plan is for this arch to remain experimental for FreeBSD 13.
This started as a weekend learning project for me and kinda snowballed from
there.
(More to follow momentarily.)
Reviewed by: imp (earlier version), emaste
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D26399
bootonce feature is temporary, one time boot, activated by
"bectl activate -t BE", "bectl activate -T BE" will reset the bootonce flag.
By default, the bootonce setting is reset on attempt to boot and the next
boot will use previously active BE.
By setting zfs_bootonce_activate="YES" in rc.conf, the bootonce BE will
be set permanently active.
bootonce dataset name is recorded in boot pool labels, bootenv area.
in case of nextboot, the nextboot_enable boolean variable is recorded in
freebsd:nvstore nvlist, also stored in boot pool label bootenv area.
On boot, the loader will process /boot/nextboot.conf if nextboot_enable
is "YES", and will set nextboot_enable to "NO", preventing /boot/nextboot.conf
processing on next boot.
bootonce and nextboot features are usable in both UEFI and BIOS boot.
To use bootonce/nextboot features, the boot loader needs to be updated on disk;
if loader.efi is stored on ESP, then ESP needs to be updated and
for BIOS boot, stage2 (zfsboot or gptzfsboot) needs to be updated
(gpart or other tools).
At this time, only lua loader is updated.
Sponsored by: Netflix, Klara Inc.
Differential Revision: https://reviews.freebsd.org/D25512
As we do for shared library binaries, pass -S to install(1) when
installing symlinks. Doing so helps avoid transient failures when
libraries are being reinstalled, which seems to be the root cause of
spurious libgcc_s.so link failures during CI builds.
PR: 233769
Reviewed by: emaste
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D26453
The current default is provided in various Makefile.inc in some top-level
directories and covers a good portion of the tree, but doesn't cover parts
of the build a little deeper (e.g. libcasper).
Provide a default in src.sys.mk and set WARNS to it in bsd.sys.mk if that
variable is defined. This lets us relatively cleanly provide a default WARNS
no matter where you're building in the src tree without breaking things
outside of the tree.
Crunchgen has been updated as a bootstrap tool to work on this change
because it needs r365605 at a minimum to succeed. The cleanup necessary to
successfully walk over this change on WITHOUT_CLEAN builds has been added.
There is a supplemental project to this to list all of the warnings that are
encountered when the environment has WARNS=6 NO_WERROR=yes:
https://warns.kevans.dev -- this project will hopefully eventually go away
in favor of CI doing a much better job than it.
Reviewed by: emaste, brooks, ngie (all earlier version)
Reviewed by: emaste, arichardson (depend-cleanup.sh change)
Differential Revision: https://reviews.freebsd.org/D26455
Use of ranlib or lorder is no longer necessary with current linkers
(probably anything newer than ~1990) and ar's ability to create an object
index and symbol table in the archive.
Currently the build system uses lorder+tsort to sort the .o files in
dependency order so that a single-pass linker can use them. However,
we can use the -s flag to ar to add an index to the .a file which makes
lorder unnecessary.
Running ar -s is equivalent to running ranlib afterwards, so we can also
skip the ranlib invocation.
Similarly, we don't have to pass the .o files for shared libraries in
dependency order since both ld.bfd and ld.lld will correctly resolve
references between the .o files.
This removes many fork()+execve calls for each library so should speed up
builds a bit. Additionally lorder.sh uses a regular expression that is not
supported by the macOS libc or glibc and results in many warnings when
cross-building (see D25989).
There is one functional change: lorder.sh removed duplicated .o files
from the linker command line which now no longer happens. I fixed the duplicates
in the base system in r364649. I also checked the ports tree for uses of
bsd.lib.mk and found one duplicate source file which I fixed in r548168.
Most ports use CMake/autotools rather than bsd.lib.mk but if this breaks any
ports that I missed in my search please let me know.
Avoiding the shell script actually speeds up the linking step noticeably: I
measured how long it takes to rebuild the .a and .so files for lib/libc using a
basic benchmark: `rm $LIBC_OBJDIR/*.so* $LIBC_OBJDIR/*.a* && /usr/bin/time make -DWITHOUT_TESTS -s > /dev/null`
Without this change ~4.5 seconds and afterwards ~3.1 seconds.
Looking at truss -cf output we can see that the number fork() system
calls goes down from 27 to 12 (and the speedup while tracing is more
noticeable: 81 seconds -> 65 seconds).
See also https://www.gnu.org/software/coreutils/manual/html_node/tsort-background.html
for some more background:
This whole procedure has been obsolete since about 1980, because Unix
archives now contain a symbol table (traditionally built by ranlib, now
generally built by ar itself), and the Unix linker uses the symbol table
to effectively make multiple passes over an archive file.
Or alternatively https://www.unix.com/man-page/osf1/1/lorder/:
The lorder command is essentially obsolete. Use the following command in
its place: % ar -ts file.a
Reviewed By: emaste, imp, dim
Differential Revision: https://reviews.freebsd.org/D26044
In D12421, the ability to compile stand/ in little-endian was added, with the
intention to extend loader.kboot to run in Petitboot.
However, no further work was done, as the kernel then gained self-execution
capabilities as Petitboot was taught to load FreeBSD kernels directly.
The FreeBSD installer on powerpc64 (on POWER8 and POWER9) uses
/boot/etc/kboot.conf instead of loader.
As this option does nothing but cause stand/ to be miscompiled and actively
causes confusion, remove it.
(I have a functioning petitboot loader in my local tree, however, it turned
out to be quite inconvient to use due to the current petitboot plugin design
so I put it on hold.)
Reviewed by: emaste, imp, jhibbits
Sponsored by: Tag1 Consulting, Inc.
Differential Revision: https://reviews.freebsd.org/D26430
Use MACHINE_CPUARCH with arm64 (aarch64) when we build code that could run
on any 64-bit Arm instruction set. This will simplify checks in downstream
consumers targeting prototype instruction sets.
The only place we check for MACHINE_ARCH == aarch64 is when building the
device tree blobs. As these are targeting current generation ISAs.
Sponsored by: Innovate UK
Differential Revision: https://reviews.freebsd.org/D26370
This allows use of the standard src.conf configuration for controlling
whether the tree is cleaned before build or not. The default is still
to clean.
Setting either NOCLEAN or NO_CLEAN will mention the new src.conf option.
NOCLEAN remains a .warning, while for now NO_CLEAN is .info.
Reviewed by: bdrewery (earlier version)
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D22762
For historical reasons, defining MALLOC_PRODUCTION in /etc/make.conf has
been used to turn off potentially expensive debug checks and statistics
gathering in the implementation of malloc(3).
It seems more consistent to turn this into a regular src.conf(5) option,
e.g. WITH_MALLOC_PRODUCTION / WITHOUT_MALLOC_PRODUCTION. This can then
be toggled similar to any other source build option, and turned on or
off by default for e.g. stable branches.
Reviewed by: imp, #manpages
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D26337
ld.bfd in particular requires -lm to come after libifconfig on the
command line when linking rescue.
Reviewed by: freqlabs, adrian
Differential Revision: https://reviews.freebsd.org/D26258
Allow architecture names to be passed in to the build system via CPUTYPE.
This allows the user to use values such as armv8.1-a or armv8-a+crc as
the CPUTYPE.
Sponsored by: Innovate UK
This changeset introduces the new libnetmap library for writing
netmap applications.
Before libnetmap, applications could either use the kernel API
directly (e.g. NIOCREGIF/NIOCCTRL) or the simple header-only-library
netmap_user.h (e.g. nm_open(), nm_close(), nm_mmap() etc.)
The new library offers more functionalities than netmap_user.h:
- Support for complex netmap options, such as external memory
allocators or per-buffer offsets. This opens the way to future
extensions.
- More flexibility in the netmap port bind options, such as
non-numeric names for pipes, or the ability to specify the netmap
allocator that must be used for a given port.
- Automatic tracking of the netmap memory regions in use across the
open ports.
At the moment there is no man page, but the libnetmap.h header file
has in-depth documentation.
Reviewed by: hrs
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D26171
