WITHOUT_BINUTILS and WITHOUT_BINUTILS_BOOTSTRAP previously included
claims about being unable to build if set. Those cautions are no longer
universally true, and most FreeBSD targets can function more or less
without enabling GNU Binutils. Just remove the cautions.
Sponsored by: The FreeBSD Foundation
GNU as 2.17.50 is currently required by amd64 and i386 for at least one
file that cannot be assembled by Clang's integrated assembler (IAS).
Other supported CPU architectures either use Clang IAS for all assembly
files, or rely on external toolchain.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23180
In the WITHOUT_ descriptions we don't need to mention that ld.bfd is
limited to powerpc. When WITHOUT_BINUTILS is specified ld.bfd is not
installed on any CPU architecture.
All archs except powerpc either use lld or require external toolchain.
powerpc still needs binutils ld to link 32-bit binaries.
Reviewed by: jhibbits
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D23107
Describe /usr/bin/cc etc. as links to the compiler, and don't conflate
WITHOUT_CLANG_IS_CC with installing GCC. Leave a reference to WITH_GCC
and WITHOUT_CLANG_IS_CC installing links to GCC, although this will be
removed in ~1.5 months when GCC 4.2.1 is removed from the tree.
Sponsored by: The FreeBSD Foundation
${WORLDTMP}/legacy/usr/libexec will only have libexec/ bits that we've
pushed as bootstrap tools, so this is generally safe to include prior to
PATH. The following are the ramifications of this change:
- BPATH addition gets us at least bootstrap flua in WMAKEENV path for
buildenv, for those earlier systems where it's bootstrapped still
- Reworked the sysent target to just set PATH and let it get worked out in
src.lua.mk or individual sysent makefiles -- this gives us back the
ability to overwrite LUA_CMD and use a different/external lua for these
targets. sysent can also now work cleanly in buildenv.
- tools/build/Makefile will now symlink the host flua into build's host
tools so that the above can work without needing to add the host's
/usr/libexec explicitly into TMPPATH.
Reviewed by: arichardson, brooks, imp (all slightly earlier version)
Differential Revision: https://reviews.freebsd.org/D22464
- Rename 'blkcipher' to 'cipher'. Some of the ciphers being tested
are stream ciphers.
- Rename 'authenc' to 'eta' as it is only testing ETA chained
operations and not other combination modes.
- Add a notion of an OCF session and some helper routines to try to
reduce duplicated code. This also uses a single session for both
encrypt and decrypt operations during a single test.
- Add tests to ensure that AEAD algorithms fail decryption with
EBADMSG when given a corrupted tag.
- Remove the transitional hack for COP_F_CIPHER_FIRST.
- Update block comment to mention plain hashes.
Reviewed by: cem
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D22940
For libssp.so, rebuild stack_protector.c with FORTIFY_SOURCE stubs that just
abort built into it.
For libssp_nonshared.a, steal stack_protector_compat.c from
^/lib/libc/secure and massage it to maintain that __stack_chk_fail_local
is a hidden symbol.
libssp is now built unconditionally regardless of {WITH,WITHOUT}_SSP in the
build environment, and the gcclibs version has been disconnected from the
build in favor of this one.
PR: 242950 (exp-run)
Reviewed by: kib, emaste, pfg, Oliver Pinter (earlier version)
Also discussed with: kan
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D22943
This should help people examining src.conf(5) draw the connection between
the HTTPD knobs and the particular implementation we're installing,
simple_httpd.
Reported by: saken658 via GitHub
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
All Rights Reserved on same line as other copyright holders (but not
me). Other such holders are also listed last where it's clear.
Summary:
Enable on powerpc64 and in lib/libclang_rt/Makefile change
MACHINE_CPUARCH to MACHINE_ARCH because on powerpc64
MACHINE_ARCH==MACHINE_CPUARCH so the 32-bit library overwrites 64-bit
library during installworld.
This patch doesn't enable any other libclang_rt libraries because they
need to be separately ported.
I have verified that games/julius (which fails on powerpc64 elfv2
without this change because of no libclang_rt profiling library) builds.
Test Plan: Ship it, test on powerpc and powerpcspe
Submitted by: pkubaj
Reviewed by: dim, jhibbits
Differential Revision: https://reviews.freebsd.org/D22425
MFC after: 1 month
X-MFC-With: r353358
flua is bootstrapped as part of the build for those on older
versions/revisions that don't yet have flua installed. Once upgraded past
r354833, "make sysent" will again naturally work as expected.
Reviewed by: brooks
Differential Revision: https://reviews.freebsd.org/D21894
The valectl(4) program is used to manage vale(4) switches.
Add it to the system commands so that it can be used right away.
This program was previously called vale-ctl, and stored in
tools/tools/netmap
Reviewed by: hrs, bcr, lwhsu, kevans
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D22146
Add lib/libzstd to _elftoolchain_libs
tools/build/Makefile needs to create the install dir for libzstd
Since this would make the line too long, rework to use a list
in one per line format (easier to add in future)
and dispense with the .for loop
Reviewed by: emaste bapt
Differential Revision: https://reviews.freebsd.org/D220134
rename the WITH_LOADER_VERIEXEC_PASS_MANFIEST description to its correct
name. Also correct a bunch of spelling errors in that description.
MFC after: 3 days
This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof). Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs. The stats(3) framework can be used in
both userspace and the kernel.
See the stats(3) manual page for details.
This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.
The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.
Reviewed by: sef (earlier version)
Obtained from: Netflix
Relnotes: yes
Sponsored by: Klara Inc, Netflix
Differential Revision: https://reviews.freebsd.org/D20477
At least some of the characters in E000-F8FF range are used by Powerline
fonts, and having no attributes for these ranges in UnicodeData.txt
other than "Other, Private Use" it should be safe to mark all of them as
printable. Some actually were before r340491, so this fixes the
regression introduced there as well.
PR: 240911
Reviewed by: bapt
Tested by: Daniel Ponte <amigan@gmail.com>
Differential Revision: https://reviews.freebsd.org/D21850
The one used previously was missing the characters in 0-127 range,
making various tools try to escape them in output.
PR: 235100
Reviewed by: bapt
Tested by: Ting-Wei Lan <lantw44@gmail.com>
Differential Revision: https://reviews.freebsd.org/D21794
This directory is actually needed during make installworld and will prevent
to reinstall a system after make delete-old is done.
PR: 226137
Reported by: rakuco
MFC after: 3 days
picobsd/tinyware has had this compact HTTPD server for a long time, and some
people do use it. Move it out into usr.sbin well in advance of any action
being taken on picobsd.
This has been gated behind an HTTPD option defaulted to *off*, primarily for
two reasons:
1.) This code likely needs a good audit, as it's been living off in picobsd
land for a long time, and
2.) We don't currently ship an httpd and this may not be a welcome surprise.
Reviewed by: eugen
Differential Revision: https://reviews.freebsd.org/D21724
run_early_customize is run as a shell list, not as a subshell, so that the side
effects of setting variables can affect later stages of the build (for better or
worse, it's been like this since it was introduced). It therefore has the side
effect of turning off xtrace always, which limits the usefulness of sh -x
nanobsd.sh. Remember the old setting and only turn off tracing after the command
if tracing was off before. All the other places where we do similar things we use
a subshell, so we don't need to do this.
rm -x was introduced in the FreeBSD 10 time frame. 4 years ago I added a
function to cope with building nanobsd images on hosts as old FreeBSD 7 that
lacked rm -x. The workaround is no longer needed as FreeBSD 9 hasn't been
supported for almost 3 years. Eliminate the wrapper and use rm -x directly
again.
This will allow feature control bits (e.g. for ASLR, PROT_MAX) to be
inspected or modified.
Some clean-up and additional work is likely still required, but we can
iterate on this in the tree.
Submitted by: Bora Özarslan <borako.ozarslan@gmail.com>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19290
per thread, so that instead of repeating the same info for all threads
in proc, it would print thread specific info. Also includes thread number
that would match 'info threads' info and can be used as argument for
thread swithcing with 'thread' command.
Those scripts are without copyright and license assignement since their creation
After grabbing information from The various authors and contributors assign
proper license header and copyrights.
This has been reported by yuripv in his work on integrating those in Illumos!
Reported by: yuripv
Discussed with: marino, edwin
MFC after: 3 days
With this last piece in place, make -C /usr/src/release release.iso is
finally able to run in a jail. This was not possible before because
msdosfs cannot be mounted inside a jail.
Submitted by: ryan@ixsystems.com
Reviewed by: emaste@, imp@, gjb@
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21385
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
PowerPC64 still needs ld.bfd for 32-bit binaries/libraries.
This will be needed when ELFv2 becomes default, but there is no harm in
committing it already.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D21136
For quite some time kgdb has been internally handling FreeBSD kernel
module state; add-on scripts and tools are not needed. asf(8) served
a similar purpose to this script and was removed in r335222.
PR: 229046
Reported by: jhb
Sponsored by: The FreeBSD Foundation
Save the last callout function pointer (and its argument) executed
on each CPU for inspection by a debugger. Add a ddb `show callout_last`
command to show these pointers. Add a kernel module that I used
for testing that command.
Relocate `ce_migration_cpu` to reduce padding and therefore preserve
the size of `struct callout_cpu` (320 bytes on amd64) despite the
added members.
This should help diagnose reference-after-free bugs where the
callout's mutex has already been freed when `softclock_call_cc`
tries to unlock it.
You might hope that the pointer would still be available, but it
isn't. The argument to that function is on the stack (because
`softclock_call_cc` uses it later), and that might be enough in
some cases, but even then, it's very laborious. A pointer to the
callout is saved right before these newly added fields, but that
callout might have been freed. We still have the pointer to its
associated mutex, and the name within might be enough, but it might
also have been freed.
Reviewed by: markj jhb
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D20794
PR/238816 initially addressed updates to usage() however the PR has
morphed into a shopping list of updates to usage() and man pages.
PR: 238816 (I added to the list during discussion)
MFC after: 1 week
pkg now uses /dev/null for some of its operations. NanoBSD's packaging
stuff didn't mount that for the chroot it ran in, so any config that
added packages would see the error:
pkg: Cannot open /dev/null:No such file or directory
when trying to actually add those packages. It's easy enough for
nanobsd to mount /dev and it won't hurt anything that was already
working and may help things that weren't (like this). I moved the
mount/unmount pair to be in the right push/pop order from the
submitted patch.
PR: 238727
Submitted by: mike tancsa
Tested by: Karl Denninger
libunwind and openmp to the upstream release_80 branch r363030
(effectively, 8.0.1 rc2). The 8.0.1 release should follow this within a
week or so.
MFC after: 2 weeks
Note llvm-ar is linked to llvm-ranlib since r311565. r348677 fixed
"make delete-old" issue with llvm-ar but missed it somehow.
Discussed with: emaste, jhb
r348504 moved llvm-symbolizer from the CLANG_EXTRAS knob to CLANG, but
the man page was still in the CLANG_EXTRAS section in
OptionalObsoleteFiles.inc.
Reported by: jhb
MFC after: 3 days
MFC with: r348504
After r348610 `make delete-old` was still removing llvm-ar and llvm-nm
(and associated man pages).
Reported by: jhb
Sponsored by: The FreeBSD Foundation
ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20484
Allow support of other VCSes. Note that two other nanobsd files already
had a similar case, excluding .git and .hg in addition to CVS and .svn.
Sponsored by: The FreeBSD Foundation
does not ship a -lomp symlink. Also update OptionalObsoleteFiles for
this, and add 32-bit variants while here.
Submitted by: jbeich
PR: 237975
MFC after: 3 days
ed(4) and ep(4) have been removed. fxp(4) remains popular in older
systems, but isn't as future proof as em(4).
Reviewed by: bz, jhb
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D20311
It seems to be incompatible with the OVMF.fd (of unknown provenance)
in use by the Cirrus-CI config. We will soon have a known OVMF build
via a port/package (see review D19869) and we can switch back to q35
once packages are available.
Discussed with: bcran
When using -S0, seed the PRNG with the current time in nanoseconds, not
seconds, so consecutive runs don't accidentally use the same seed.
Also, rename some variables for clarity.
Reviewed by: ngie
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20078
Update ci-qemu-test.sh
o Update the path to the OVMF file, which is now in /usr/local/share/uefi-edk2-qemu.
o Use the more modern q35, pc-q35-3.0 (Q35 + ICH9, 2009) QEMU machine
instead of the default, obsolete pc, pc-i440fx-3.0 (i440FX + PIIX, 1996).
For example this adds ACPI support.
o Specify the system firmware using the newer pflash drive syntax instead
of bios.
o Remove extra, unneeded devices by passing -nodefaults.
o Change text to talk about 'firmware' instead of 'bios', since UEFI
isn't a BIOS.
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D20074
ufs partition as p2, and put the zfs partition at p3, to test the ability
of the zfs probe code to find a zfs pool on something other than the first
partition.
tools/boot/install-boot.sh was assuming that if a device was passed in,
it should operate on the current system and run efibootmgr etc. to
update the boot manager. However, rootgen.sh passes a md(4) device and
not a fixed disk.
Add a -u option to install-boot.sh to tell it to update the system
in-place and run efibootmgr etc.
Also, source install-boot.sh in rootgen.sh to allow it to find and
call make_esp_file etc. And pass the loader file to make_esp_file instead
of a directory name.
Reported by: ian
Reviewed by: ian,imp,tsoome
Differential Revision: https://reviews.freebsd.org/D19992
See r346250 and followup commits and mailing list discussion.
We currently fail to boot properly in the absense of boot-time entropy.
Sponsored by: The FreeBSD Foundation
the temporary image in $TMPDIR.
Allow the script to be run from the src/tools/boot directory by using make
-V SRCTOP to find the top of the tree, because this script is handy for
quick smoke-testing of loader changes, as well as being useful in CI testing.
Also, use a temp directory in $TMPDIR to assemble the boot image, and write
the boot log file to $TMPDIR. Arrange to have the temporary image clean
itself up, but leave the log file in $TMPDIR for post-mortem analysis of
failures when the script is run interactively.
Differential Revision: https://reviews.freebsd.org/D19876
