This provides a framework to define a template describing
a set of "variables of interest" and the intended way for
the framework to maintain them (for example the maximum, sum,
t-digest, or a combination thereof). Afterwards the user
code feeds in the raw data, and the framework maintains
these variables inside a user-provided, opaque stats blobs.
The framework also provides a way to selectively extract the
stats from the blobs. The stats(3) framework can be used in
both userspace and the kernel.
See the stats(3) manual page for details.
This will be used by the upcoming TCP statistics gathering code,
https://reviews.freebsd.org/D20655.
The stats(3) framework is disabled by default for now, except
in the NOTES kernel (for QA); it is expected to be enabled
in amd64 GENERIC after a cool down period.
Reviewed by: sef (earlier version)
Obtained from: Netflix
Relnotes: yes
Sponsored by: Klara Inc, Netflix
Differential Revision: https://reviews.freebsd.org/D20477
At least some of the characters in E000-F8FF range are used by Powerline
fonts, and having no attributes for these ranges in UnicodeData.txt
other than "Other, Private Use" it should be safe to mark all of them as
printable. Some actually were before r340491, so this fixes the
regression introduced there as well.
PR: 240911
Reviewed by: bapt
Tested by: Daniel Ponte <amigan@gmail.com>
Differential Revision: https://reviews.freebsd.org/D21850
The one used previously was missing the characters in 0-127 range,
making various tools try to escape them in output.
PR: 235100
Reviewed by: bapt
Tested by: Ting-Wei Lan <lantw44@gmail.com>
Differential Revision: https://reviews.freebsd.org/D21794
This directory is actually needed during make installworld and will prevent
to reinstall a system after make delete-old is done.
PR: 226137
Reported by: rakuco
MFC after: 3 days
picobsd/tinyware has had this compact HTTPD server for a long time, and some
people do use it. Move it out into usr.sbin well in advance of any action
being taken on picobsd.
This has been gated behind an HTTPD option defaulted to *off*, primarily for
two reasons:
1.) This code likely needs a good audit, as it's been living off in picobsd
land for a long time, and
2.) We don't currently ship an httpd and this may not be a welcome surprise.
Reviewed by: eugen
Differential Revision: https://reviews.freebsd.org/D21724
run_early_customize is run as a shell list, not as a subshell, so that the side
effects of setting variables can affect later stages of the build (for better or
worse, it's been like this since it was introduced). It therefore has the side
effect of turning off xtrace always, which limits the usefulness of sh -x
nanobsd.sh. Remember the old setting and only turn off tracing after the command
if tracing was off before. All the other places where we do similar things we use
a subshell, so we don't need to do this.
rm -x was introduced in the FreeBSD 10 time frame. 4 years ago I added a
function to cope with building nanobsd images on hosts as old FreeBSD 7 that
lacked rm -x. The workaround is no longer needed as FreeBSD 9 hasn't been
supported for almost 3 years. Eliminate the wrapper and use rm -x directly
again.
This will allow feature control bits (e.g. for ASLR, PROT_MAX) to be
inspected or modified.
Some clean-up and additional work is likely still required, but we can
iterate on this in the tree.
Submitted by: Bora Özarslan <borako.ozarslan@gmail.com>
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D19290
per thread, so that instead of repeating the same info for all threads
in proc, it would print thread specific info. Also includes thread number
that would match 'info threads' info and can be used as argument for
thread swithcing with 'thread' command.
Those scripts are without copyright and license assignement since their creation
After grabbing information from The various authors and contributors assign
proper license header and copyrights.
This has been reported by yuripv in his work on integrating those in Illumos!
Reported by: yuripv
Discussed with: marino, edwin
MFC after: 3 days
With this last piece in place, make -C /usr/src/release release.iso is
finally able to run in a jail. This was not possible before because
msdosfs cannot be mounted inside a jail.
Submitted by: ryan@ixsystems.com
Reviewed by: emaste@, imp@, gjb@
MFC after: 1 week
Sponsored by: iXsystems, Inc.
Differential Revision: https://reviews.freebsd.org/D21385
KTLS adds support for in-kernel framing and encryption of Transport
Layer Security (1.0-1.2) data on TCP sockets. KTLS only supports
offload of TLS for transmitted data. Key negotation must still be
performed in userland. Once completed, transmit session keys for a
connection are provided to the kernel via a new TCP_TXTLS_ENABLE
socket option. All subsequent data transmitted on the socket is
placed into TLS frames and encrypted using the supplied keys.
Any data written to a KTLS-enabled socket via write(2), aio_write(2),
or sendfile(2) is assumed to be application data and is encoded in TLS
frames with an application data type. Individual records can be sent
with a custom type (e.g. handshake messages) via sendmsg(2) with a new
control message (TLS_SET_RECORD_TYPE) specifying the record type.
At present, rekeying is not supported though the in-kernel framework
should support rekeying.
KTLS makes use of the recently added unmapped mbufs to store TLS
frames in the socket buffer. Each TLS frame is described by a single
ext_pgs mbuf. The ext_pgs structure contains the header of the TLS
record (and trailer for encrypted records) as well as references to
the associated TLS session.
KTLS supports two primary methods of encrypting TLS frames: software
TLS and ifnet TLS.
Software TLS marks mbufs holding socket data as not ready via
M_NOTREADY similar to sendfile(2) when TLS framing information is
added to an unmapped mbuf in ktls_frame(). ktls_enqueue() is then
called to schedule TLS frames for encryption. In the case of
sendfile_iodone() calls ktls_enqueue() instead of pru_ready() leaving
the mbufs marked M_NOTREADY until encryption is completed. For other
writes (vn_sendfile when pages are available, write(2), etc.), the
PRUS_NOTREADY is set when invoking pru_send() along with invoking
ktls_enqueue().
A pool of worker threads (the "KTLS" kernel process) encrypts TLS
frames queued via ktls_enqueue(). Each TLS frame is temporarily
mapped using the direct map and passed to a software encryption
backend to perform the actual encryption.
(Note: The use of PHYS_TO_DMAP could be replaced with sf_bufs if
someone wished to make this work on architectures without a direct
map.)
KTLS supports pluggable software encryption backends. Internally,
Netflix uses proprietary pure-software backends. This commit includes
a simple backend in a new ktls_ocf.ko module that uses the kernel's
OpenCrypto framework to provide AES-GCM encryption of TLS frames. As
a result, software TLS is now a bit of a misnomer as it can make use
of hardware crypto accelerators.
Once software encryption has finished, the TLS frame mbufs are marked
ready via pru_ready(). At this point, the encrypted data appears as
regular payload to the TCP stack stored in unmapped mbufs.
ifnet TLS permits a NIC to offload the TLS encryption and TCP
segmentation. In this mode, a new send tag type (IF_SND_TAG_TYPE_TLS)
is allocated on the interface a socket is routed over and associated
with a TLS session. TLS records for a TLS session using ifnet TLS are
not marked M_NOTREADY but are passed down the stack unencrypted. The
ip_output_send() and ip6_output_send() helper functions that apply
send tags to outbound IP packets verify that the send tag of the TLS
record matches the outbound interface. If so, the packet is tagged
with the TLS send tag and sent to the interface. The NIC device
driver must recognize packets with the TLS send tag and schedule them
for TLS encryption and TCP segmentation. If the the outbound
interface does not match the interface in the TLS send tag, the packet
is dropped. In addition, a task is scheduled to refresh the TLS send
tag for the TLS session. If a new TLS send tag cannot be allocated,
the connection is dropped. If a new TLS send tag is allocated,
however, subsequent packets will be tagged with the correct TLS send
tag. (This latter case has been tested by configuring both ports of a
Chelsio T6 in a lagg and failing over from one port to another. As
the connections migrated to the new port, new TLS send tags were
allocated for the new port and connections resumed without being
dropped.)
ifnet TLS can be enabled and disabled on supported network interfaces
via new '[-]txtls[46]' options to ifconfig(8). ifnet TLS is supported
across both vlan devices and lagg interfaces using failover, lacp with
flowid enabled, or lacp with flowid enabled.
Applications may request the current KTLS mode of a connection via a
new TCP_TXTLS_MODE socket option. They can also use this socket
option to toggle between software and ifnet TLS modes.
In addition, a testing tool is available in tools/tools/switch_tls.
This is modeled on tcpdrop and uses similar syntax. However, instead
of dropping connections, -s is used to force KTLS connections to
switch to software TLS and -i is used to switch to ifnet TLS.
Various sysctls and counters are available under the kern.ipc.tls
sysctl node. The kern.ipc.tls.enable node must be set to true to
enable KTLS (it is off by default). The use of unmapped mbufs must
also be enabled via kern.ipc.mb_use_ext_pgs to enable KTLS.
KTLS is enabled via the KERN_TLS kernel option.
This patch is the culmination of years of work by several folks
including Scott Long and Randall Stewart for the original design and
implementation; Drew Gallatin for several optimizations including the
use of ext_pgs mbufs, the M_NOTREADY mechanism for TLS records
awaiting software encryption, and pluggable software crypto backends;
and John Baldwin for modifications to support hardware TLS offload.
Reviewed by: gallatin, hselasky, rrs
Obtained from: Netflix
Sponsored by: Netflix, Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D21277
PowerPC64 still needs ld.bfd for 32-bit binaries/libraries.
This will be needed when ELFv2 becomes default, but there is no harm in
committing it already.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D21136
For quite some time kgdb has been internally handling FreeBSD kernel
module state; add-on scripts and tools are not needed. asf(8) served
a similar purpose to this script and was removed in r335222.
PR: 229046
Reported by: jhb
Sponsored by: The FreeBSD Foundation
Save the last callout function pointer (and its argument) executed
on each CPU for inspection by a debugger. Add a ddb `show callout_last`
command to show these pointers. Add a kernel module that I used
for testing that command.
Relocate `ce_migration_cpu` to reduce padding and therefore preserve
the size of `struct callout_cpu` (320 bytes on amd64) despite the
added members.
This should help diagnose reference-after-free bugs where the
callout's mutex has already been freed when `softclock_call_cc`
tries to unlock it.
You might hope that the pointer would still be available, but it
isn't. The argument to that function is on the stack (because
`softclock_call_cc` uses it later), and that might be enough in
some cases, but even then, it's very laborious. A pointer to the
callout is saved right before these newly added fields, but that
callout might have been freed. We still have the pointer to its
associated mutex, and the name within might be enough, but it might
also have been freed.
Reviewed by: markj jhb
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D20794
PR/238816 initially addressed updates to usage() however the PR has
morphed into a shopping list of updates to usage() and man pages.
PR: 238816 (I added to the list during discussion)
MFC after: 1 week
pkg now uses /dev/null for some of its operations. NanoBSD's packaging
stuff didn't mount that for the chroot it ran in, so any config that
added packages would see the error:
pkg: Cannot open /dev/null:No such file or directory
when trying to actually add those packages. It's easy enough for
nanobsd to mount /dev and it won't hurt anything that was already
working and may help things that weren't (like this). I moved the
mount/unmount pair to be in the right push/pop order from the
submitted patch.
PR: 238727
Submitted by: mike tancsa
Tested by: Karl Denninger
libunwind and openmp to the upstream release_80 branch r363030
(effectively, 8.0.1 rc2). The 8.0.1 release should follow this within a
week or so.
MFC after: 2 weeks
Note llvm-ar is linked to llvm-ranlib since r311565. r348677 fixed
"make delete-old" issue with llvm-ar but missed it somehow.
Discussed with: emaste, jhb
r348504 moved llvm-symbolizer from the CLANG_EXTRAS knob to CLANG, but
the man page was still in the CLANG_EXTRAS section in
OptionalObsoleteFiles.inc.
Reported by: jhb
MFC after: 3 days
MFC with: r348504
After r348610 `make delete-old` was still removing llvm-ar and llvm-nm
(and associated man pages).
Reported by: jhb
Sponsored by: The FreeBSD Foundation
ASAN reports become a lot more useful with llvm-symbolizer in $PATH, and the
build is not much more time-consuming. The added benefit is that the
resulting reports will actually include symbol information; without, thread
trace information includes a bunch of addresses that immediately resolve to
an inline function in
^/contrib/compiler-rt/lib/sanitizer_common/sanitizer_common.h and take a
little more effort to examine.
Reviewed by: emaste
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D20484
Allow support of other VCSes. Note that two other nanobsd files already
had a similar case, excluding .git and .hg in addition to CVS and .svn.
Sponsored by: The FreeBSD Foundation
does not ship a -lomp symlink. Also update OptionalObsoleteFiles for
this, and add 32-bit variants while here.
Submitted by: jbeich
PR: 237975
MFC after: 3 days
ed(4) and ep(4) have been removed. fxp(4) remains popular in older
systems, but isn't as future proof as em(4).
Reviewed by: bz, jhb
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D20311
It seems to be incompatible with the OVMF.fd (of unknown provenance)
in use by the Cirrus-CI config. We will soon have a known OVMF build
via a port/package (see review D19869) and we can switch back to q35
once packages are available.
Discussed with: bcran
When using -S0, seed the PRNG with the current time in nanoseconds, not
seconds, so consecutive runs don't accidentally use the same seed.
Also, rename some variables for clarity.
Reviewed by: ngie
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D20078
Update ci-qemu-test.sh
o Update the path to the OVMF file, which is now in /usr/local/share/uefi-edk2-qemu.
o Use the more modern q35, pc-q35-3.0 (Q35 + ICH9, 2009) QEMU machine
instead of the default, obsolete pc, pc-i440fx-3.0 (i440FX + PIIX, 1996).
For example this adds ACPI support.
o Specify the system firmware using the newer pflash drive syntax instead
of bios.
o Remove extra, unneeded devices by passing -nodefaults.
o Change text to talk about 'firmware' instead of 'bios', since UEFI
isn't a BIOS.
Reviewed by: imp, emaste
Differential Revision: https://reviews.freebsd.org/D20074
ufs partition as p2, and put the zfs partition at p3, to test the ability
of the zfs probe code to find a zfs pool on something other than the first
partition.
tools/boot/install-boot.sh was assuming that if a device was passed in,
it should operate on the current system and run efibootmgr etc. to
update the boot manager. However, rootgen.sh passes a md(4) device and
not a fixed disk.
Add a -u option to install-boot.sh to tell it to update the system
in-place and run efibootmgr etc.
Also, source install-boot.sh in rootgen.sh to allow it to find and
call make_esp_file etc. And pass the loader file to make_esp_file instead
of a directory name.
Reported by: ian
Reviewed by: ian,imp,tsoome
Differential Revision: https://reviews.freebsd.org/D19992
See r346250 and followup commits and mailing list discussion.
We currently fail to boot properly in the absense of boot-time entropy.
Sponsored by: The FreeBSD Foundation
the temporary image in $TMPDIR.
Allow the script to be run from the src/tools/boot directory by using make
-V SRCTOP to find the top of the tree, because this script is handy for
quick smoke-testing of loader changes, as well as being useful in CI testing.
Also, use a temp directory in $TMPDIR to assemble the boot image, and write
the boot log file to $TMPDIR. Arrange to have the temporary image clean
itself up, but leave the log file in $TMPDIR for post-mortem analysis of
failures when the script is run interactively.
Differential Revision: https://reviews.freebsd.org/D19876
The current approach of injecting manifest into mac_veriexec is to
verify the integrity of it in userspace (veriexec (8)) and pass its
entries into kernel using a char device (/dev/veriexec).
This requires verifying root partition integrity in loader,
for example by using memory disk and checking its hash.
Otherwise if rootfs is compromised an attacker could inject their own data.
This patch introduces an option to parse manifest in kernel based on envs.
The loader sets manifest path and digest.
EVENTHANDLER is used to launch the module right after the rootfs is mounted.
It has to be done this way, since one might want to verify integrity of the init file.
This means that manifest is required to be present on the root partition.
Note that the envs have to be set right before boot to make sure that no one can spoof them.
Submitted by: Kornel Duleba <mindal@semihalf.com>
Reviewed by: sjg
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D19281
The WITH_PORT_BASE_{BINUTILS,GCC} options are used to prevent 'make check-old'
and 'make delete-old' from deleting files installed by the base/binutils
and base/gcc packages as normally one disables the in-tree variants
(e.g. WITHOUT_BINUTILS) when using these packages.
Reviewed by: bapt (earlier version)
Differential Revision: https://reviews.freebsd.org/D19483
* Set MK_OPENMP to yes by default only on amd64, for now.
* Bump __FreeBSD_version to signal this addition.
* Ensure gcc's conflicting omp.h is not installed if MK_OPENMP is yes.
* Update OptionalObsoleteFiles.inc to cope with the conflicting omp.h.
* Regenerate src.conf(5) with new WITH/WITHOUT fragments.
Relnotes: yes
PR: 236062
MFC after: 1 month
X-MFC-With: r344779
This initial integration takes googlemock/googletest release 1.8.1, integrates
the library, tests, and sample unit tests into the build.
googlemock/googletest's inclusion is optionally available via `MK_GOOGLETEST`.
`MK_GOOGLETEST` is dependent on `MK_TESTS` and is enabled by default when
built with a C++11 capable toolchain.
Google tests can be specified via the `GTESTS` variable, which, in comparison
with the other test drivers, is more simplified/streamlined, as Googletest only
supports C++ tests; not raw C or shell tests (C tests can be written in C++
using the standard embedding methods).
No dependent libraries are assumed for the tests. One must specify `gmock`,
`gmock_main`, `gtest`, or `gtest_main`, via `LIBADD` for the program.
More information about googlemock and googletest can be found on the
Googletest [project page](https://github.com/google/googletest), and the
[GoogleMock](https://github.com/google/googletest/blob/v1.8.x/googlemock/docs/Documentation.md)
and
[GoogleTest](https://github.com/google/googletest/tree/v1.8.x/googletest/docs)
docs.
These tests are originally integrated into the build as plain driver tests, but
will be natively integrated into Kyua in a later version.
Known issues/Errata:
* [WhenDynamicCastToTest.AmbiguousCast fails on FreeBSD](https://github.com/google/googletest/issues/2172)
Reviewed by: asomers
Approved by: emaste (mentor)
MFC after: 2 months
Differential Revision: https://reviews.freebsd.org/D19551
is specified.
When WITHOUT_IPFILTER is specified, delete-old-files fails to delete
the optional rc.d files from above. Fix this.
WITHOUT_IPFILTER fails to delete the ipfilter.5 optional file during
delete-old-files. Fix this.
Reported by: Dmitry Luhtionov <dmitryluhtionov@gmail.com>
MFC after: 1 week
This is a WIP tool I'm using to figure out why ANI is weirdly busted in my
home FreeBSD AP/STA setup. Although athstats (mostly) gets the ANI statistics
correct, ANI is making the radio deaf it doesn't recover without being disabled.
It's very WIP.
Tested:
* Carambola 2, (AR9331), AP/STA mode.
UEFI related headers were copied from edk2.
A new build option "MK_LOADER_EFI_SECUREBOOT" was added to allow
loading of trusted anchors from UEFI.
Certificate revocation support is also introduced.
The forbidden certificates are loaded from dbx variable.
Verification fails in two cases:
There is a direct match between cert in dbx and the one in the chain.
The CA used to sign the chain is found in dbx.
One can also insert a hash of TBS section of a certificate into dbx.
In this case verifications fails only if a direct match with a
certificate in chain is found.
Submitted by: Kornel Duleba <mindal@semihalf.com>
Reviewed by: sjg
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D19093
The send_packets() function was using ring->cur as index to scan
the transmit ring. This function may also set ring->cur ahead of
ring->head, in case no more slots are available. However, the function
also uses nm_ring_space() which looks at ring->head to check how many
slots are available. If ring->head and ring->cur are different, this
results in pkt-gen advancing ring->cur beyond ring->tail.
This patch fixes send_packets() (and similar source locations) to
use ring->head as a index, rather than using ring->cur.
MFC after: 1 week
It is protected by MK_NLS. If it should really be optional then
it needs to be documented as such in share/mk/bsd.README and
.sinclude used where needed.
This fixes a regression from r335011.
PR: 232527
Submitted by: jarrod@downtools.com.au
Reported by: ktullavik@gmail.com
MFC after: 3 days
These libraries don't compile on non-C++-11 capable compilers, e.g., g++ 4.2.1
and its corresponding implementation of the c++ library, i.e., libstdc++.
Blacklist compilation on all non-C++-11 capable compilers and give others the
option of opting out of building/installing gmock/gtest via MK_GOOGLETEST.
This option is controlled by MK_CXX and MK_TESTS, as ATF compilation is.
- Drop profile libraries; MK_PROFILE=no is set in all Makefile's.
- Correct library path to libmlx5.so.1 and libibverbs.so.1
MFC after: 5 days
MFC with: 344207
- Add missing /usr/sbin/pmc, pmcformat.h, libpmcstat.h and pmc.haswellxeon.3
to the list.
- Correct man page section for pmcstudy.8.
- Include recently added libipt and libopencsd for corresponding TARGET_ARCH
MFC after: 5 days
OptionalObsoleteFiles.inc
Note: only files with conditional installation logic were
included from the PR.
PR: 233046
Submitted by: <rozhuk.im@gmail.com>
MFC after: 5 days
Building binaries as PIE allows the executable itself to be loaded at a
random address when ASLR is enabled (not just its shared libraries).
With this change PIE objects have a .pieo extension and INTERNALLIB
libraries libXXX_pie.a.
MK_PIE is disabled for some kerberos5 tools, Clang, and Subversion, as
they explicitly reference .a libraries in their Makefiles. These can
be addressed on an individual basis later. MK_PIE is also disabled for
rtld-elf because it is already position-independent using bespoke
Makefile rules.
Currently only dynamically linked binaries will be built as PIE.
Discussed with: dim
Reviewed by: kib
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18423
This commit essentially has three parts:
* Add the AES-CCM encryption hooks. This is in and of itself fairly small,
as there is only a small difference between CCM and the other ICM-based
algorithms.
* Hook the code into the OpenCrypto framework. This is the bulk of the
changes, as the algorithm type has to be checked for, and the differences
between it and GCM dealt with.
* Update the cryptocheck tool to be aware of it. This is invaluable for
confirming that the code works.
This is a software-only implementation, meaning that the performance is very
low.
Sponsored by: iXsystems Inc.
Differential Revision: https://reviews.freebsd.org/D19090
It's installed to /usr/sbin, not to /usr/bin (and was always here).
While here, add missing manpages and /var/yp directory to the list.
MFC after: 1 week
tinybsd offers two choices when prompting user for MFSROOT: 'YES'
and 'NO'. Script logic only handles 'yes'. Change offered values
to lower case.
PR: 131059
Submitted by: Brock Williams <brock@cotcomsol.com>
MFC after: 1 week
Add all the files under /usr/share/examples to the MK_EXAMPLES
section. OLD_DIRS entries are not removed if they're not empty so
prior to this change WITHOUT_EXAMPLES didn't have significant effect
on the updated system.
PR: 228484
Submitted by: Dmitry Wagin <dmitry.wagin@ya.ru> (original patch)
MFC after: 1 week
Add two more entries for WITHOUT_SENDMAIL install. The /var/spool/clientmqueue
entry would be deleted only if there are no files/dirs in it, so the
content generated during previous lifecycle of the system is safe
PR: 228484
Submitted by: Dmitry Wagin <dmitry.wagin@ya.ru>
MFC after: 1 week
Add asn1_compile, make-roken, kcc, and slc to the OptionalObsoleteFiles.inc
so they would be removed during delete-old stage if the new world is built
without Kerberos support.
PR: 230725
Submitted by: Dmitry Wagin <dmitry.wagin@ya.ru>
MFC after: 1 week
devices like mice, keyboards, bt-audio, ...
This script currently allows scanning for nearby devices, adds one to
/etc/bluetooth/hosts, adds an entry to hcsecd's conf and if it is a HID, add an
entry to bthidd's configs, as well.
Submitted by: erdgeist <erdgeist@erdgeist.org>
Approved by: bapt
MFC after: 2 weeks
Differential Revision: D3778
Reviewers: bapt, emax
usr/share/dict/eign is removed unconditionally as of r319664, so it no
longer needs to be in OptionalObsoleteFiles.
PR: 230353
Reported by: philippe.michel7 sfr.fr
This is primarily aimed at failed updates due to package conflicts, and
affects treatment of failed updates. Whereas before potentially a large
number of packages would need to be synced for each attempt, they can now
be persisted. Requires rsync. There may be better ways to implement this,
e.g. using secondary cache path that is only used on followup attempts and
then wiped on success, which avoids polluting current cache.
Small modifications to the nmreplay man page.
Used igor and mandoc tools to fix warnings and errors.
Reviewed by: bcr
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D18629
Changelist:
- General reformatting
- Fix packet duplication in cons(). Whenever cons() reached the
burst limit it would send all pending packets without advancing
head. This caused the last injected packet to be sent again in
the next round.
- Fix full-speed transmissions after first loop.
MFC after: 3 days
Currently, the installer uses pre-created 800KB FAT12 filesystems that
it dd's onto the ESP partition.
This changeset improves that by having the installer generate a FAT32
filesystem directly onto the ESP using newfs_msdos and then copying
loader.efi into /EFI/freebsd.
For live installs it then runs efibootmgr to add a FreeBSD boot entry
in the BIOS.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D17947
MK_AUDIT already controls auditd(8), praudit(1), etc. It should also control
the audit test suite.
Submitted by: ngie
MFC after: 2 weeks
Pull Request: https://github.com/freebsd/freebsd/pull/240
/usr/sbin/audit(dist)?d are only installed if ${MK_AUDIT} == yes. Their
supporting scripts should only be installed in those instances as well.
Submitted by: ngie
Reviewed by: emaste
MFC after: 2 weeks
Pull Request: https://github.com/freebsd/freebsd/pull/242
The removal (and creation of a port) has been pre-announced in UPDATING
1 month ago. Packages are available for all supported FreeBSD vesions.
I did not think that another entry in UPDATING is required to note the
actual removal.
No MFC is planned - CTM shall be kept in base for all releases up to 12.x.
Reviewed by: rgrimes
Approved by: imp, bcr (manpages)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D17935
Validate the value of the -l argument (packet length) against the MTU of the netmap port.
In case the netmap port does not refer to a physical interface (e.g. VALE port or pipe), then
the netmap buffer size is used as MTU.
This change also sets a better default value for the -M option, so that pkt-gen uses
the largest possible fragments in case of multi-slot packets.
Differential Revision: https://reviews.freebsd.org/D18436
GNU binutils ld.bfd 2.17.50 does not support ifuncs and produces broken
binaries when ifuncs are in use. When LLD_IS_LD is default we have an
ifunc-capable system linker and can just avoid installing ld.bfd.
Reported by: theraven
Reviewed by: bz
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D18340
Some WITH_/WITHOUT_ defaults will likey change in the future (e.g. as we
migrate to copyfree base system components).
Add non-default descriptions for the benefit of WIP branches.
This should provide more complete coverage of currently defined Unicode
characters as compared to manually assembled one we use currently.
Comparison of original and new UTF-8 ctype maps by character class:
TYPE ORIG NEW
alnum 94229 126029
alpha 93557 125419
blank 4 2
cntrl 73 137685
digit 469 622
graph 109615 137203
lower 1478 2145
print 109641 137222
punct 3428 797
rune 110481 274907
space 33 24
upper 983 1781
xdigit 469 622
Large number of added cntrl definitions is due to the fact that private-use
planes are currently defined as such, this can change in the future.
Discussed with: bapt
Approved by: kib (mentor, implicit)
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D17842
document the libufs interface for fetching and storing inodes.
The undocumented getino / putino interface has been replaced
with a new getinode / putinode interface.
Convert the utilities that had been using the undocumented
interface to use the new documented interface.
No functional change (as for now the libufs library does not
do inode check-hashes).
Reviewed by: kib
Tested by: Peter Holm
Sponsored by: Netflix
Various improvements to the netmap pkt-gen program:
- indentation fixes
- support for IPV6
- fixes to checksum computation
- support for NS_MOREFRAG
- rate limiting in ping mode
Reviewed by: bcr, 0mp
Approved by: gnn (mentor)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D17698
llvm-profdata is used with llvm-cov for code coverage (although llvm-cov
can also operate independently in a gcov-compatible mode).
Although llvm-profdata can be used independently of llvm-cov it makes
sense to group these under one option.
Also handle these in OptionalObsoleteFiles.inc while here.
Sponsored by: The FreeBSD Foundation
llvm-cov provides a gcov-compatible interface when invoked as gcov.
Reviewed by: dim, markj
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17923
Add the lb program, which is able to load-balance input traffic
received from a netmap port over M groups, with N netmap pipes in
each group. Each received packet is forwarded to one of the pipes
chosen from each group (using an L3/L4 connection-consistent hash function).
This also adds a man page for lb and some cross-references in related
man pages.
Reviewed by: bcr, 0mp
Approved by: gnn (mentor)
Differential Revision: https://reviews.freebsd.org/D17735
- Simplify the source dir specification, and update README
appropriately
- Drop the LC (doonly) processing, it's broken, and even if fixed, not
really useful
- Don't remove the target directories while installing new data as it
removes Makefile.depend which we don't manage; only rm the files we
are going to add/replace/delete instead
- Restrict adding bsd.endian.mk to colldef and ctypedef Makefiles, it's
not needed in other (text-only) categories
- GC unused scripts; they don't seem to be particularly helpful standalone
as well
Reviewed by: bapt
Approved by: kib (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D17858
The linker's -z now flag sets the DF_BIND_NOW flag, which signals to the
runtime loader that all relocation processing should be performed at
process startup rather than on demand. In combination with lld's
default of enabling relro this causes the GOT to be made read-only when
the process starts, preventing straightforward GOT overwrite attacks.
Shawn Webb discovered a failure on HardenedBSD with BIND_NOW and ifunc
use, which resulted in my rtld fix in r340137. Add a BIND_NOW knob as
it is trivial to do so and is a useful ELF hardening feature. This
change is equivalent to HardenedBSD's but not identical as there are
other diffs/conflicts nearby.
Note that our ELF Tool Chain readelf does not currently decode the
DF_BIND_NOW flag - see PR232983.
Reviewed by: brooks
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D17846
Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.
All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.
Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
that it uses the same ctype maps and functions as other UTF-8 locales.
Reviewed by: bapt, cem, eadler
Approved by: kib (mentor, implicit)
Differential Revision: https://reviews.freebsd.org/D17833
You should not be using DES. You should not have been using DES for the
past 30 years.
The ed DES-CBC scheme lacked several desirable properties of a sealed
document system, even ignoring DES itself. In particular, it did not
provide the "integrity" cryptographic property (detection of tampering), and
it treated ASCII passwords as 64-bit keys (instead of using a KDF like
scrypt or PBKDF2).
Some general approaches ed(1) users might consider to replace the removed
DES mode:
1. Full disk encryption with something like AES-XTS. This is easy to
conceptualize, design, and implement, and it provides confidentiality for
data at rest. Like CBC, it lacks tampering protection. Examples include
GELI, LUKS, FileVault2.
2. Encrypted overlay ("stackable") filesystems (EncFS, PEFS?, CryptoFS,
others).
3. Native encryption at the filesystem layer. Ext4/F2FS, ZFS, APFS, and
NTFS all have some flavor of this.
4. Storing your files unencrypted. It's not like DES was doing you much
good.
If you have DES-CBC scrambled files produced by ed(1) prior to this change,
you may decrypt them with:
openssl des-cbc -d -iv 0 -K <key in hex> -in <inputfile> -out <plaintext>
Reviewed by: allanjude, bapt, emaste
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D17829
r235274 added a sort regression test (it operates by comparing output
against GNU sort). The commit included a number of 0-byte files, one
of which ends in a trailing . which reportedly breaks svn/git checkouts
on Windows.
It appears these were added accidentally, so just remove them.
PR: 232479
MFC after: 1 month
In the last decade(s) we have seen both short term or long term projects
committed to the tree which were considered or even marked "experimental".
While out-of-tree development has become easier than it used to be in
CVS times, there still is a need to have the code shipping with HEAD but
not enabled by default.
While people may think about VIMAGE as one of the recent larger, long term
projects, early protocol implementations (before they are standardised)
are others. (Free)BSD historically was one of the operating systems
which would have running code at early stages and help develop and
influence standardisation and the industry.
Give developers an opportunity to be more pro-active for early adoption
or running large scale code changes stumbling over each others but not
the user's feet. I have not added the option to NOTES in order to avoid
breaking supported option builds, which require constant compile testing.
Discussed with: people in the corridor
This driver was marked as gone in 12. We're at 13 now. Remove it.
Data from nycbug's dmesg cache shows only one potential user,
suggesting it never was used much. However, even though this device
has been obsolete for 15 years at least, sys/joystick.h is included in
a number of graphics packages still, so that remains. A full exprun
is needed before that can be removed.
RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D17629
I held the mistaken belief this was completely unused. While the
driver is unused and likely not relevant for a long time,
sys/joystick.h lives on in maybe half a dozen ports, even though
hardware to use it hasn't been widely used in maybe 15 years.
These are needed for .ctors/.dtors and .jcr handling. The former needs
all the function pointers to be called in the correct order from the
.init/.fini section. The latter just needs to call a gcj specific function
if it exists with a pointer to the start of the .jcr section.
This is currently disabled until __dso_handle support is added.
Reviewed by: emaste
MFC after: 1 month
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17587
Use userboot.so from the test directory if possible, fall back to .OBJDIR.
This avoids a problem that we've had since userboot coexistence was added,
where userboot.so alone no longer exists in the .OBJDIR but is instead just
a link installed later.
Added man page for vale-ctl program.
Small fixes to vale-ctl, including the support for -m option
(to specify the netmap memory allocator id).
Reviewed by: 0mp
Approved by: gnn (mentor)
Differential Revision: https://reviews.freebsd.org/D17683
Remove mse and all support for bus and inport devices from the tree.
Data from nycbug's dmesg database shows the last sighting of this
driver was in 4.10 on only one machine.
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D17628
This driver was marked as gone in 12. We're at 13 now. Remove it.
Data from nycbug's dmesg cache shows only one potential user,
suggesting it never was used much.
RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D17629
WITHOUT_LOADER_LUA is only needed since we turned it off by default on
powerpc and sparc64 in r338203. Same with
WITHOUT_LOADER_GEIL. WITH_NVME, WITHOUT_NVME, WITH_LOADER_FORCE_LE
have been needed since they were added.
it appropriately when building share/ctypedef and share/colldef.
This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
build and in the native EB build match. Revert the changes done to libc
in r308170 as they are no longer needed.
PR: 231965
Reviewed by: bapt, emaste, sbruno, 0mp
Approved by: kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17603
data, namely 0xE000-0xF8FF private use area, and 0xFF00-0xFFF half- and
fullwidth punctuation.
While here, update tools/tools/locale/README based on my experience
rebuilding the locale data.
PR: 225692
Reviewed by: bapt, cem (previous version)
Approved by: re (gjb), kib (mentor)
Differential Revision: https://reviews.freebsd.org/D17471
Without this we get spurious output during boot as we try to run
nonexistant HyperV scripts on non-x86 models.
Reviewed by: kib
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17211
Since r326030 strings is installed unconditionally so should not be
removed when WITHOUT_TOOLCHAIN is set.
Reported by: Dan McGregor
Approved by: re (kib)
Sponsored by: The FreeBSD Foundation
Make the building of drm dependent on MK_MODULE_DRM and the building
of module drm2 on MK_MODULE_DRM2. The defaults are unchanged.
Approved by: re@ (gjb)
Differential Review: https://reviews.freebsd.org/D16894
hostapd requires libpcap, which links against libmlx5 and libibverbs when
building WITH_OFED. These were not pulled in to bsdbox and most bsdbox
builds were WITHOUT_OFED up until recently, so it was not noticed.
Approved by: re (gjb)
given in random(4).
This includes updating of the relevant man pages, and no-longer-used
harvesting parameters.
Ensure that the pseudo-unit-test still does something useful, now also
with the "other" algorithm instead of Yarrow.
PR: 230870
Reviewed by: cem
Approved by: so(delphij,gtetlow)
Approved by: re(marius)
Differential Revision: https://reviews.freebsd.org/D16898
vermaden (maintainer of beadm) points out the following inconsistencies:
- "missing command" is not printed prior to usage if the error is simply a
missing command; this should be obvious from the context
- "bectl rename" isn't using the "don't unmount" flag (zfs rename -u), so
the active BE can't be renamed. It doesn't make sense in our context to
*not* use -u, so use it.
Documentation updates reflect the above and note an inconsistency with the
'destroy' command that is consistent with other parts of the base system.
A fix for libbe(3) not properly being installed to /lib is included.
SHLIBDIR should have been added when it was moved in r337995.
Approved by: re (kib)
Revert r338177, r338176, r338175, r338174, r338172
After long consultations with re@, core members and mmacy, revert
these changes. Followup changes will be made to mark them as
deprecated and prent a message about where to find the up-to-date
driver. Followup commits will be made to make this clear in the
installer. Followup commits to reduce POLA in ways we're still
exploring.
It's anticipated that after the freeze, this will be removed in
13-current (with the residual of the drm2 code copied to
sys/arm/dev/drm2 for the TEGRA port's use w/o the intel or
radeon drivers).
Due to the impending freeze, there was no formal core vote for
this. I've been talking to different core members all day, as well as
Matt Macey and Glen Barber. Nobody is completely happy, all are
grudgingly going along with this. Work is in progress to mitigate
the negative effects as much as possible.
Requested by: re@ (gjb, rgrimes)
Checking for any include below ${SRCTOP}/sys is too strict and breaks
e.g. mkimg which includes sys/sys/disk. ABI issues will only be caused
by including headers in sys/sys since they might not match the host.
Approved By: jhb (mentor)
Suggested By: imp
to fit in only direct blocks whose size is exactly a multiple of the
filesystem block size.
Reported by: Peter Holm
Tested by: Peter Holm
Sponsored by: Netflix
This has two advantages:
1) We no longer create lots of empty directories that are not needed
2) This is a requirement for building on non-FreeBSD hosts since mtree will
only exist after the bootstrap-tools phase there.
Aproved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16773
This can cause surprising errors if the build tools is built against
headers that don't match the host system. It is also required in order
to allow building on non-FreeBSD systems where the headers in
/usr/include/sys are usually completely incompatible with those in the
source tree.
I added an error to Makefile.boot if this is done and found this was
only the case in libnv. With this error in the Makefile ABI breakages
such as r336019 should no longer be possible.
Reviewed By: bdrewery, kevans
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D16186
BPF (eBPF) is an independent instruction set architecture which is
introduced in Linux a few years ago. Originally, eBPF execute
environment was only inside Linux kernel. However, recent years there
are some user space implementation (https://github.com/iovisor/ubpf,
https://doc.dpdk.org/guides/prog_guide/bpf_lib.html) and kernel space
implementation for FreeBSD is going on
(https://github.com/YutaroHayakawa/generic-ebpf).
The BPF target support can be enabled using WITH_LLVM_TARGET_BPF, as it
is not built by default.
Submitted by: Yutaro Hayakawa <yhayakawa3720@gmail.com>
Reviewed by: dim, bdrewery
Differential Revision: https://reviews.freebsd.org/D16033
The timespecadd(3) family of macros were imported from NetBSD back in
r35029. However, they were initially guarded by #ifdef _KERNEL. In the
meantime, we have grown at least 28 syscalls that use timespecs in some
way, leading many programs both inside and outside of the base system to
redefine those macros. It's better just to make the definitions public.
Our kernel currently defines two-argument versions of timespecadd and
timespecsub. NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define
three-argument versions. Solaris also defines a three-argument version, but
only in its kernel. This revision changes our definition to match the
common three-argument version.
Bump _FreeBSD_version due to the breaking KPI change.
Discussed with: cem, jilles, ian, bde
Differential Revision: https://reviews.freebsd.org/D14725
Since all post-installkernel steps are assumed to operate in the updated
installation, it's necessary to chroot all of the followup steps in the new
boot environment. Set up and mount the source and object directories at the
same paths inside the BE root, and clean up to the extent changes were made.
This commit fixes upgrading using beinstall past the new ntpd user change.
Improve testability of changes to this script while I'm here.
Reported by: rpokala (earlier patch)
