Update libarchive to 3.2.0
New features:
- new bsdcat command-line utility
- LZ4 compression (in src only via external utility from ports)
- Warc format support
- 'Raw' format writer
- Zip: Support archives >4GB, entries >4GB
- Zip: Support encrypting and decrypting entries
- Zip: Support experimental streaming extension
- Identify encrypted entries in several formats
- New --clear-nochange-flags option to bsdtar tries to remove noschg and
similar flags before deleting files
- New --ignore-zeros option to bsdtar to handle concatenated tar archives
- Use multi-threaded LZMA decompression if liblzma supports it
- Expose version info for libraries used by libarchive
Patched files (fixed compiler warnings):
contrib/libarchive/cat/bsdcat.c (vendor PR #702)
contrib/libarchive/cat/bsdcat.h (vendor PR #702)
contrib/libarchive/libarchive/archive_read_support_format_mtree.c (PR #701)
contrib/libarchive/libarchive_fe/err.c (vendor PR #703)
MFC after: 1 month
Relnotes: yes
Two new functions are provided, bit_ffs_at() and bit_ffc_at(), which allow
for efficient searching of set or cleared bits starting from any bit offset
within the bit string.
Performance is improved by operating on longs instead of bytes and using
ffsl() for searches within a long. ffsl() is a compiler builtin in both
clang and gcc for most architectures, converting what was a brute force
while loop search into a couple of instructions.
All of the bitstring(3) API continues to be contained in the header file.
Some of the functions are large enough that perhaps they should be uninlined
and moved to a library, but that is beyond the scope of this commit.
sys/sys/bitstring.h:
Convert the majority of the existing bit string implementation from
macros to inline functions.
Properly protect the implementation from inadvertant macro expansion
when included in a user's program by prefixing all private
macros/functions and local variables with '_'.
Add bit_ffs_at() and bit_ffc_at(). Implement bit_ffs() and
bit_ffc() in terms of their "at" counterparts.
Provide a kernel implementation of bit_alloc(), making the full API
usable in the kernel.
Improve code documenation.
share/man/man3/bitstring.3:
Add pre-exisiting API bit_ffc() to the synopsis.
Document new APIs.
Document the initialization state of the bit strings
allocated/declared by bit_alloc() and bit_decl().
Correct documentation for bitstr_size(). The original code comments
indicate the size is in bytes, not "elements of bitstr_t". The new
implementation follows this lead. Only hastd assumed "elements"
rather than bytes and it has been corrected.
etc/mtree/BSD.tests.dist:
tests/sys/Makefile:
tests/sys/sys/Makefile:
tests/sys/sys/bitstring.c:
Add tests for all existing and new functionality.
include/bitstring.h
Include all headers needed by sys/bitstring.h
lib/libbluetooth/bluetooth.h:
usr.sbin/bluetooth/hccontrol/le.c:
Include bitstring.h instead of sys/bitstring.h.
sbin/hastd/activemap.c:
Correct usage of bitstr_size().
sys/dev/xen/blkback/blkback.c
Use new bit_alloc.
sys/kern/subr_unit.c:
Remove hard-coded assumption that sizeof(bitstr_t) is 1. Get rid of
unrb.busy, which caches the number of bits set in unrb.map. When
INVARIANTS are disabled, nothing needs to know that information.
callapse_unr can be adapted to use bit_ffs and bit_ffc instead.
Eliminating unrb.busy saves memory, simplifies the code, and
provides a slight speedup when INVARIANTS are disabled.
sys/net/flowtable.c:
Use the new kernel implementation of bit-alloc, instead of hacking
the old libc-dependent macro.
sys/sys/param.h
Update __FreeBSD_version to indicate availability of new API
Submitted by: gibbs, asomers
Reviewed by: gibbs, ngie
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D6004
After calling the cap_init(3) function Casper will fork from it's original
process, using pdfork(2). Forking from a process has a lot of advantages:
1. We have the same cwd as the original process.
2. The same uid, gid and groups.
3. The same MAC labels.
4. The same descriptor table.
5. The same routing table.
6. The same umask.
7. The same cpuset(1).
From now services are also in form of libraries.
We also removed libcapsicum at all and converts existing program using Casper
to new architecture.
Discussed with: pjd, jonathan, ed, drysdale@google.com, emaste
Partially reviewed by: drysdale@google.com, bdrewery
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D4277
control algorithm options. The argument is variable length and is opaque
to TCP, forwarded directly to the algorithm's ctl_output method.
Provide new includes directory netinet/cc, where algorithm specific
headers can be installed.
The new API doesn't yet have any in tree consumers.
The original code written by lstewart.
Reviewed by: rrs, emax
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D711
tools/regression/geom_{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}
in to the FreeBSD test suite as
tests/sys/geom/class/{concat,eli,gate,mirror,nop,raid3,shsec,stripe,uzip}
The tools/regression/geom and tools/regression/geom_part testcases are being
left alone because both test sets are both currently broken.
The majority of this work was done on ^/user/ngie/more-tests2 . The differences
are as follows:
- tests/sys/geom/class/Makefile.inc is not present; it was
inlined into the class's Makefiles for explicitness.
- The testcases officially require root via kyua
- The geom_gate(4) tests don't use the pidfile changes proposed in
https://reviews.freebsd.org/D4836 .
MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division
Fix a bug in rpcbind for multihomed hosts. If the server had interfaces on
two separate subnets, and a client on the first subnet contacted rpcbind at
the address on the second subnet, rpcbind would advertise addresses on the
first subnet. This is a bug, because it should prefer to advertise the
address where it was contacted. The requested service might be firewalled
off from the address on the first subnet, for example.
usr.sbin/rpcbind/check_bound.c
If the address on which a request was received is known, pass that
to addrmerge as the clnt_uaddr parameter. That is what addrmerge's
comment indicates the parameter is supposed to mean. The previous
behavior is that clnt_uaddr would contain the address from which the
client sent the request.
usr.sbin/rpcbind/util.c
Modify addrmerge to prefer to use an IP that is equal to clnt_uaddr,
if one is found. Refactor the relevant portion of the function for
clarity, and to reduce the number of ifdefs.
etc/mtree/BSD.tests.dist
usr.sbin/rpcbind/tests/Makefile
usr.sbin/rpcbind/tests/addrmerge_test.c
Add unit tests for usr.sbin/rpcbind/util.c:addrmerge.
usr.sbin/rpcbind/check_bound.c
usr.sbin/rpcbind/rpcbind.h
usr.sbin/rpcbind/util.c
Constify some function arguments
Reviewed by: imp
MFC after: 4 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D4690
This is inspired by parallel efforts being done on projects/zfsd (and makes
more sense because these are functional tests of the subsystems, not the
geom(8) commands
bugfix-only release, with no new features.
Please note that from 3.5.0 onwards, clang and llvm require C++11
support to build; see UPDATING for more information.
tools/regression/mac/mac_portacl into the FreeBSD test suite as
tests/sys/mac/bsdextended and tests/sys/mac/portacl, respectively
MFC after: 1 month
Sponsored by: EMC / Isilon Storage Division
lib/libc/tests/nss
- Convert the testcases to ATF
- Do some style(9) cleanups:
-- Sort headers
-- Apply indentation fixes
-- Remove superfluous parentheses
- Explicitly print out debug printfs for use with `kyua {debug,report}`; for
items that were overly noisy, they've been put behind #ifdef DEBUG
conditionals
- Fix some format strings
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
tests/sys/kern/pipe
- Fix style(9) bugs
- Fix compiler warnings
- Use `nitems(x)` instead of `sizeof(x) / sizeof(*x)` pattern
The testcases will be converted over to ATF eventually, but for now will be
integrated in as plain C tests
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
The :encode_tv_random_million testcase fails the epsilon tests few thousand
times out of one million, so expect the testcase to fail
MFC after: 1 week
Submitted by: keramida
Sponsored by: EMC / Isilon Storage Division
The goal here is to make the upgrade seamless for users
Add aliases for zh_HK
Remove bad symlinks created by previous bad upgrade procedure.
Complete ObsoleteFiles.inc with more locales that have been removed
as lib/libc/rpc
This testcase requires rpcbind be up in running; otherwise the testcases
will time out and be skipped
MFC after: 1 week
Sponsored by: EMC / Isilon Storage Division
Extend OptionalObsoleteFiles.inc to delete all lib32 files when MK_LIB32 is
set to no on a system that previously had lib32 libraries installed.
Also, to prevent "make delete-old-dirs" from always deleting lib32 directories
after an installworld, move the lib32 subtree to its own mtree file that only
gets applied when MK_LIB32=yes.
Test: Ran "make delete-old" and "make delete-old-libs" on a system that never
had MK_LIB32 enabled, and on a system where MK_LIB32 was enabled and later
disabled. Did this both on amd64 and powerpc64.
Test: Ran "make tinderbox" without errors.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D3923
- Make the requirements more complete for the testcases
- Detect prerequisites so the tests won't fail (zfs.ko is loaded, zpool(1)
is available, ACL support is enabled with UFS, etc).
- Work with temporary files/directories/mountpoints that work with atf/kyua
- Limit the testcases to work on temporary filesystems to reduce tainting the
test host
MFC after: 2 weeks
Reviewed by: trasz (earlier version)
Differential Revision: https://reviews.freebsd.org/D3810
test suite as tests/sys/posixshm
Some other highlights:
- Convert the testcases over to ATF
- Don't use hardcoded paths to /tmp (which violate the ATF/kyua samdbox); use
mkstemp to generate temporary paths for non-SHM_ANON shm objects.
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
leap second date correct
Updates to the file can be obtained from ftp://time.nist.gov/pub/ or
ftp://tycho.usno.navy.mil/pub/ntp/.
Suggested by: dwmalone
Reviewed by: roberto, dwmalone, delphij
Approved by: roberto
MFC after: 1 week
bug of installing 'realtek' and 'intel_iwn' as files rather then as
a 'LICENSE' file in their directories.
Also add obsolete entries for the older names and names that existed in head
for a period of time.
Suggested by: jmg
X-MFC-With: r289391
MFC after: 3 weeks
Sponsored by: EMC / Isilon Storage Division
The functional_test.sh harness for each test subdir was inspired
by the version in bin/sh/tests/functional_test.sh
Some gymnastics were required to deal with implicit rules for
.c / .o -> .out as the suffix transformation rules were
incorrectly trying to create the test outputs from some of the
source files
Sponsored by: EMC / Isilon Storage Division
This avoids needing a large boot partition / file system in order to
accommodate multiple kernels, and provides consistency with userland
debug. This also simplifies the process of moving kernel debug files
to a separate package and installing them on demand.
In addition, change kernel debug file extension to .debug, to match
userland debug files.
When using the supported kernel installation method the
/usr/lib/debug/boot/kernel directory will be renamed (to kernel.old)
as is done with /boot/kernel.
Developers wishing to maintain the historical behavior of installing
debug files in /boot/kernel/ can set KERN_DEBUGDIR="" in src.conf(5).
Reviewed by: bdrewery, brooks, imp, markj
Relnotes: yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D1006
Rationale: ident(1) is useful out of RCS, lot of scripts are using ident(1) and
failing when base is built WITHOUT_RCS.
This version is:
- fully compatible with RCS 5.7 ident.
- fully compatible with RCS 5.9 ident.
- passes all ident test from GNU RCS 5.9 test suite
This version has support for: svn extension for the Keyword id (double colon and
# before last $)
Différences with GNU RCS ident:
- no long options as found in GNU RCS 5.9 (but not commented there).
- '-V' reports nothing but has been added for compatibility.
Differential Revision: https://reviews.freebsd.org/D3200
Reviewed by: pfg
Now when a lib is marked as PRIVATELIB it is renamed into libprivate$foo instead
of being installed in /usr/lib/private and playing with rpath.
Also allow to install headers for PRIVATELIBS in that case the headers will be
installed in /usr/include/private/$foo
Keep the headers under a private namespace to prevent third party build system
to easily find them to ensure they are only used on purpose.
This allows for non base applications to statically link against a library in
base which is linked to a privatelib
Treating PRIVATELIBS as regular libraries allows to push them into our current
compatX packages if needed.
While here finish promotion of libevent as PRIVATELIB
Install header for bsdstat and libucl
Differential Revision: https://reviews.freebsd.org/D2365
Reviewed by: brooks, des
Discussed with: imp
As jhb noted, the actual mmap(2) implementation is under sys/vm, not
sys/kern/, so the correct logical place is tests/sys/vm/, not
tests/sys/kern/
X-MFC with: r282076
MFC after: 6 days
discontinued by its initial authors. In FreeBSD the code was already
slightly edited during the pf(4) SMP project. It is about to be edited
more in the projects/ifnet. Moving out of contrib also allows to remove
several hacks to the make glue.
Reviewed by: net@
For each test category, we generate a script containing ATF test cases for
the tests under that category. Each test case simply runs dtest.pl (the
upstream test harness) with the corresponding test files. The exclude.sh
script is used to record info about tests which should be skipped or are
expected to fail; it is used to generate atf_skip and atf_expect_fail calls.
The genmakefiles.sh script can be used to regenerate the test makefiles when
new tests are brought it from upstream.
The test suite is currently not connected to the build as there is a small
number of lingering test issues which still need to be worked out. In the
meantime however, the test suite can be easily built and installed
manually from cddl/usr.sbin/dtrace/tests.
Reviewed by: ngie
Sponsored by: EMC / Isilon Storage Division
update paths; and include everything in the "base" distribution.
The "games" distribution being optional made sense when there were more
games and we had small disks; but the "games-like" games were moved into
the ports tree a dozen years ago and the remaining "utility-like" games
occupy less than 0.001% of my laptop's small hard drive. Meanwhile every
new user is confronted by the question "do you want games installed" when
they they try to install FreeBSD.
The next steps will be:
2. Removing punch card (bcd, ppt), phase-of-moon (pom), clock (grdc), and
caesar cipher (caesar, rot13) utilities. I intend to keep fortune, factor,
morse, number, primes, and random, since there is evidence that those are
still being used.
3. Merging src/games into src/usr.bin.
This change will not be MFCed.
Reviewed by: jmg
Discussed at: EuroBSDCon
Approved by: gjb (release-affecting changes)
contains the libraries for Address Sanitizer (asan), Undefined Behavior
Sanitizer (ubsan) and Profile Guided Optimization.
ASan is a fast memory error detector. It can detect the following types
of bugs:
Out-of-bounds accesses to heap, stack and globals
Use-after-free
Use-after-return (to some extent)
Double-free, invalid free
Memory leaks (experimental)
Typical slowdown introduced by AddressSanitizer is 2x.
UBSan is a fast and compatible undefined behavior checker. It enables a
number of undefined behavior checks that have small runtime cost and no
impact on address space layout or ABI.
PLEASE NOTE: the sanitizers still have some rough edges on FreeBSD,
particularly on i386. These will hopefully be smoothed out in the
coming time.
Differential Revision: https://reviews.freebsd.org/D1505
- Fix depend target by removing a space after an "-I" inclusion option.
- Fix some minor compile issues in the "osmtest" utility.
MFC after: 3 days
PR: 196580
Sponsored by: Mellanox Technologies
for counter mode), and AES-GCM. Both of these modes have been added to
the aesni module.
Included is a set of tests to validate that the software and aesni
module calculate the correct values. These use the NIST KAT test
vectors. To run the test, you will need to install a soon to be
committed port, nist-kat that will install the vectors. Using a port
is necessary as the test vectors are around 25MB.
All the man pages were updated. I have added a new man page, crypto.7,
which includes a description of how to use each mode. All the new modes
and some other AES modes are present. It would be good for someone
else to go through and document the other modes.
A new ioctl was added to support AEAD modes which AES-GCM is one of them.
Without this ioctl, it is not possible to test AEAD modes from userland.
Add a timing safe bcmp for use to compare MACs. Previously we were using
bcmp which could leak timing info and result in the ability to forge
messages.
Add a minor optimization to the aesni module so that single segment
mbufs don't get copied and instead are updated in place. The aesni
module needs to be updated to support blocked IO so segmented mbufs
don't have to be copied.
We require that the IV be specified for all calls for both GCM and ICM.
This is to ensure proper use of these functions.
Obtained from: p4: //depot/projects/opencrypto
Relnotes: yes
Sponsored by: FreeBSD Foundation
Sponsored by: NetGate
A variant of this code has been tested on amd64/i386 for some time by
EMC/Isilon on 10-STABLE/11-CURRENT. It builds on other architectures, but the
code will remain off until it's proven it works on virtual hardware or real
hardware on other architectures
Sponsored by: EMC / Isilon Storage Division
A variant of this code has been tested on amd64/i386 for some time by
EMC/Isilon on 10-STABLE/11-CURRENT. It builds on other architectures, but the
code will remain off until it's proven it works on virtual hardware or real
hardware on other architectures
Sponsored by: EMC / Isilon Storage Division
lib/libthr/tests
A variant of this code has been tested on amd64/i386 for some time by
EMC/Isilon on 10-STABLE/11-CURRENT. It builds on other architectures, but the
code will remain off until it's proven it works on virtual hardware or real
hardware on other architectures
Original work by: pho
Sponsored by: EMC / Isilon Storage Division
500 new testcases
Various TODOs have been sprinkled around the Makefiles for items that even need
to be ported (missing features), testcases have issues with building/linking, or
issues at runtime.
A variant of this code has been tested extensively on amd64 and i386
10-STABLE/11-CURRENT for several months without issue. It builds on other
architectures, but the code will remain off until I have prove it works on
virtual hardware or real hardware on other architectures
In collaboration with: pho, Casey Peel <casey.peel@isilon.com>
Sponsored by: EMC / Isilon Storage Division
pjdfstest execution is opt-in and must be done as root due to some of the
assumptions made by the test suite and lack of error checking in the non-root
case
A description of how to execute pjdfstest with kyua is provided in
share/pjdfstest/README
Phabric: D824 (an earlier prototype patch)
MFC after: 1 month
Relnotes: yes
Sponsored by: EMC / Isilon Storage Division
Directories for /usr/lib{,32}/{i18n,private} were missing from the mtree
file, which caused installworld to install the files that should be in
the directory as the name of the directory.
This will prevent vim users from accidentally checking in buggy mtree files
(mixed tabs/spaces).
MFC after: 2 weeks
Sponsored by: EMC / Isilon Storage Division
many thanks for their continued support of FreeBSD.
While I'm there, also implement a new build knob, WITHOUT_HYPERV to
disable building and installing of the HyperV utilities when necessary.
The HyperV utilities are only built for i386 and amd64 targets.
This is a stable/10 candidate for inclusion with 10.1-RELEASE.
Submitted by: Wei Hu <weh microsoft com>
MFC after: 1 week
tree support includes a device tree source compiler dtc(8) which
converts .dts files into .dtb files. /boot/loader will load dtb files
from this directory by default, allowing for fewer differences between
images for different SoCs. Compiled dts files will wind up here
eventually as an alternative to embedding them into the kernel.
Document this in hier(7), as well as add missing entries for
/boot/firmware and /boot/zfs, though the latter two should only be
considered place holders if someone wants to make them better.
particularly useful for services such as "network" (netif) where each
interface can now have its own separate configuration file.
Add /etc/rc.conf.d to the mtree file so it is always present.
MFC after: 3 days
UNIX systems, eg. MacOS X and Solaris. It uses Sun-compatible map format,
has proper kernel support, and LDAP integration.
There are still a few outstanding problems; they will be fixed shortly.
Reviewed by: allanjude@, emaste@, kib@, wblock@ (earlier versions)
Phabric: D523
MFC after: 2 weeks
Relnotes: yes
Sponsored by: The FreeBSD Foundation
This change consists of two merges from projects/zfsd/head along with the
addition of an ATF test case for the new functionality.
sbin/devd/tests/Makefile
sbin/devd/tests/client_test.c
Add ATF test cases for reading events from both devd socket types.
r266519:
sbin/devd/devd.8
sbin/devd/devd.cc
Create a new socket, of type SOCK_SEQPACKET, for communicating with
clients. SOCK_SEQPACKET sockets preserve record boundaries,
simplying code in the client. The old SOCK_STREAM socket is retained
for backwards-compatibility with existing clients.
r269993:
sbin/devd/devd.8
Fix grammar bug.
CR: https://reviews.freebsd.org/rS266519
MFC after: 5 days
Sponsored by: Spectra Logic
Rename all of the TAP test applications from <test> to <test>_test
to match the convention described in the TestSuite wiki page
Phabric: D538
Approved by: jmmv (mentor)
Sponsored by: EMC / Isilon Storage Division
