This should be effectively a nop for all archs, but for some reason the codegen
difference on the PowerPC 970 is such that the struct assignment doesn't work
(unless a printf() using one of the elements in the copied struct follows it),
while the memcpy() succeeds. On all archs the memcpy() should be expanded to an
inline copy, since the copy is bounded to ~16 bytes.
MFC after: 3 weeks
domains can be done by the _domain() API variants. UMA also supports a
first-touch policy via the NUMA zone flag.
The slab layer is now segregated by VM domains and is precise. It handles
iteration for round-robin directly. The per-cpu cache layer remains
a mix of domains according to where memory is allocated and freed. Well
behaved clients can achieve perfect locality with no performance penalty.
The direct domain allocation functions have to visit the slab layer and
so require per-zone locks which come at some expense.
Reviewed by: Attilio (a slightly older version)
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
userspace to control NUMA policy administratively and programmatically.
Implement domainset based iterators in the page layer.
Remove the now legacy numa_* syscalls.
Cleanup some header polution created by having seq.h in proc.h.
Reviewed by: markj, kib
Discussed with: alc
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D13403
would be filesystem type dependent, but that's difficult to accomplish
and it's unclear how the UEFI firmware will cope. Be conservative and
make boot loaders cope instead.
Sponsored by: Netflix
condition. This should prevent a double free. In addition, prevent a
leak by freeing dp each loop and when we're done.
CID: 1383577
Sponsored by: Netflix
Fix for remainder overflow, when in rare cases adding remainder to divider
exceeded 1 and turned the total to 1000 in final formatting, taking up
the space for the unit character.
The fix continues the division of the original number if the above case
happens -- added the appropriate check to the for loop performing
the division. This lowers the value shown, to make it fit into the buffer
space provided (1.0M for 4+1 character buffer, as used by ls).
Add test case for the reported bug and extend test program to support
providing buffer length (ls -lh uses 5, tests hard-coded 4).
PR: 224498
Submitted by: Pawel Biernacki <pawel.biernacki@gmail.com>
Reported by: Masachika Ishizuka <ish@amail.plala.or.jp>
Reviewed by: cem, kib
Approved by: cem, kib
MFC after: 1 week
Sponsored by: Mysterious Code Ltd.
Differential Revision: D13578
The daemonfd function is equivalent to the daemon(3) function expect that
arguments are descriptors. For example dhclient(8) which is sandboxed is
unable to open /dev/null to close stdio instead it's allows to fail
daemon(3) function to close the descriptors and then do it explicit in code.
Instead of such hacks we can use now daemonfd.
This API can be also helpful to migrate system to platforms like CheriBSD.
Reviewed by: brooks@, bcr@, jilles@ (earlier version)
Differential Revision: https://reviews.freebsd.org/D13433
have access to machines that are pushing 400 devices. When 1,000 was
selected, it was rare to get even 40 or 50 devices. Bump the limit by
10x to keep up with the times.
Sponsored by: Netflix
In order to let truss(8) support tracing of 32-bit CloudABI
applications, we need to add a new ABI type to libsysdecode. We can
reuse the existing errno mapping table. Also link in the cloudabi32
system call table to translate system call names.
While there, remove all of the architecture ifdefs. There are not
needed, as the CloudABI data types and system call tables build fine on
any architecture. Building this unconditionally will make it easier to
do tracing for different compat modes, emulation, etc.
Reviewed by: jhb
Differential Revision: https://reviews.freebsd.org/D13516
There are two versions of variant I of TLS
- ARM and aarch64 uses original version of variant I here TP points to
start of TCB followed by aligned TLS segment. Both TCB and TLS must
be aligned to alignment of TLS section. The TCB[0] points to DTV vector
and DTV values are real addresses (without bias).
- MIPS, PowerPC and RISC-V use modified version of variant I,
where TP points (with bias) to TLS and TCB immediately precedes TLS
without any alignment gap. Only TLS should be aligned. The TCB[0]
points to DTV vector and DTV values are biased by constant value (0x8000)
from real addresses.
Take all this in account when allocating memory for TLS structures.
MFC after: 1 month
Reviewed by: kib, mizhka
Tested by: mizhka(on mips)
Differential Revision: https://reviews.freebsd.org/D13378
Now that the POSIX working group is going to require that basename(3)
and dirname(3) are thread-safe in future revisions of the standard,
there is even less of a need to provide basename_r(3). Remove this
function to prevent people from writing code that only builds on
FreeBSD and Bionic.
Removing this function seems to break exactly one port: sbruno@'s
qemu-user-static. I will send him a pull request on GitHub in a bit.
__FreeBSD_version will not be bumped, as any value from 2017 can be used
to test for the presence of a thread-safe basename(3)/dirname(3).
PR: https://bugs.freebsd.org/224016
Currently each call to telldir() requires a malloc and adds an entry to a
linked list which must be traversed on future telldir(), seekdir(),
closedir(), and readdir() calls. Applications that call telldir() for every
directory entry incur O(n^2) behavior in readdir() and O(n) in telldir() and
closedir().
This optimization eliminates the malloc() and linked list in most cases by
packing the relevant information into a single long. On 64-bit architectures
msdosfs, NFS, tmpfs, UFS, and ZFS can all use the packed representation. On
32-bit architectures msdosfs, NFS, and UFS can use the packed
representation, but ZFS and tmpfs can only use it for about the first 128
files per directory. Memory savings is about 50 bytes per telldir(3) call.
Speedup for telldir()-heavy directory traversals is about 20-30x for one
million files per directory.
Reviewed by: kib, mav, mckusick
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13385
matching failure.
According to the Open Group documentation for fwscanf:
"Upon successful completion, these functions shall return the number of
successfully matched and assigned input items; this number can be zero in
the event of an early matching failure."
Without this change, fwscanf would return EOF in the case of an early
matching failure, instead of the proper return value of 0.
This change aligns fwscanf(3) with the implementation in fscanf(3).
PR: 202240
Submitted by: rajendra.sy@gmail.com
Reviewed by: jhb, cem
Approved by: sjg (mentor)
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D13288
Using
.symver foo,foo@@VER
causes foo and foo@@VER to be output to the .o file. This requires foo
to be weak since the linker handles foo@@VER as foo.
Using
.symver foo,foo@@@VER
causes just foo@@ver to be output and avoid the need for making foo
weak. It also reduces the constraint on how exactly a linker has to
handle foo and foo@@VER being present.
Submitted by: Rafael Espíndola
Reviewed by: dim, kib
Differential Revision: https://reviews.freebsd.org/D11653
efivar_device_path_to_unix_path translates from UEFI to Unix
efivar_unix_path_to_device_path translates from Unix to UEFI
At present, only HD() device types are supported (both GPT and
MBR). CdRom and floppy devices aren't supported. ZFS isn't supported
because there's no way in the UEFI standard to specify a ZFS datastore.
Network devices aren't supported either.
Three forms of Unix path are accepted: /path/to/file (for a mounted
filesystem), //path/to/file (uses the EFI partition on the same disk
as /), and dev:/path/to/file (for unmounted filesystem). Two forms are
produced (the first and last).
Sponsored by: Netflix
system calls. Man pages are missing for v2 and v5, so any entries for
those versions were inferred by new implementations of these functions
in libc.
Obtained from: http://www.tuhs.org/cgi-bin/utree.pl
It would previously return negative zero for -0.0 since -0.0 does not
compare less than 0. The issue was discovered when running the libc++
test suite on softfloat MIPS64.
I have verified that both clang and GCC generate sensible code for the
builtin. For soft float they clear the sign bit using integer operations
and in hard float mode they use abs.d.
Reviewed by: #mips, jhb, brooks, imp, emaste
Approved by: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D13135
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using mis-identified many licenses so this was mostly a manual - error
prone - task.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
- Add a new KTR_STRUCT_ARRAY ktrace record type which dumps an array of
structures.
The structure name in the record payload is preceded by a size_t
containing the size of the individual structures. Use this to
replace the previous code that dumped the kevent arrays dumped for
kevent(). kdump is now able to decode the kevent structures rather
than dumping their contents via a hexdump.
One change from before is that the 'changes' and 'events' arrays are
not marked with separate 'read' and 'write' annotations in kdump
output. Instead, the first array is the 'changes' array, and the
second array (only present if kevent doesn't fail with an error) is
the 'events' array. For kevent(), empty arrays are denoted by an
entry with an array containing zero entries rather than no record.
- Move kevent decoding tables from truss to libsysdecode.
This adds three new functions to decode members of struct kevent:
sysdecode_kevent_filter, sysdecode_kevent_flags, and
sysdecode_kevent_fflags.
kdump uses these helper functions to pretty-print kevent fields.
- Move structure definitions for freebsd11 and freebsd32 kevent
structures to <sys/event.h> so that they can be shared with userland.
The 32-bit structures are only exposed if _WANT_KEVENT32 is defined.
The freebsd11 structures are only exposed if _WANT_FREEBSD11_KEVENT is
defined. The 32-bit freebsd11 structure requires both.
- Decode freebsd11 kevent structures in truss for the compat11.kevent()
system call.
- Log 32-bit kevent structures via ktrace for 32-bit compat kevent()
system calls.
- While here, constify the 'void *data' argument to ktrstruct().
Reviewed by: kib (earlier version)
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D12470
sa(4) has some unique behavior that is special-cased in cam_get_device. The
existing tests don't provide coverage for this special case.
Reviewed by: ken
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13185
Add some rules to more closely match what illumos does when an address
resolves to multiple symbols:
- prefer non-local symbols
- prefer symbols with fewer leading underscores and no leading '$'
Add some regression tests to verify these rules.
* Wrongly matches strings that are shorter than the pattern
* Fails to match negative character sets
* Fails to match character sets that aren't at the end of the pattern
* Fails to match character ranges
Reviewed by: imp
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D13173
Previously it was enabled by WITH_/WITHOUT_TOOLCHAIN, but it is commonly
expected to be available and may have non-toolchain consumers. As it
is now taken from the BSD-licensed ELF Tool Chain project, just install
it unconditionally.
PR: 213665, 223725
Reviewed by: bdrewery
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D8398
Mainly focus on files that use BSD 3-Clause license.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
Initially, only tag files that use BSD 4-Clause "Original" license.
RelNotes: yes
Differential Revision: https://reviews.freebsd.org/D13133
Do not use macros in the -width of a .Bl, since mandoc does not support them.
Fix issues reported by igor and mandoc -Tlint.
Use a .Bl for list of clock IDs instead of a comma list.
MFC after: 3 days
Sponsored by: Dell EMC
Add notes to each of these that specifically state that results are
undefined if the strings overlap. In the case of memcpy, we document
the overlapping behavior on FreeBSD (pre-existing). For str*, it is
left unspecified, however, since the default (and x86) implementations
do not handle overlapping strings properly.
PR: 223653
Sponsored by: Netflix
syslog in libc secretly reconnects to the daemon.
Another issue is that we don't have any information from openlog(3) if we
succeeded to open log or not so we don't know if we are ready
to enter cabability mode.
Because all of that we decided we need a syslog service for Caspser.
Reviewed by: bapt@
Differential Revision: https://reviews.freebsd.org/D12824
function, so check if cap_chanel_t is NULL is not enough.
Casper with a normal libc will still fail in capability mote so let's not
enter capability mode without casper support when we need to resolve DNS.
Reviewed by: cem
Differential Revision: https://reviews.freebsd.org/D12823
This API allows callers to enumerate all known pages, including any
direct map & kernel map virtual addresses, physical addresses, size,
offset into the core, & protection configured.
For architectures that support direct map addresses, also generate pages
for any direct map only addresses that are not associated with kernel
map addresses.
Fix page size portability issue left behind from previous kvm page table
lookup interface.
Reviewed by: jhb
Sponsored by: Backtrace I/O
Differential Revision: https://reviews.freebsd.org/D12279
We started out having Linux compatible libefivar interfaces. This was
in anticipation of porting the GPL'd efibootmgr to FreeBSD via a
port. However, since we need that functionality in the base, that port
isn't going to happened. It also appears that efivar is a private
library that's not used much outside a command line util and
efibootmgr. Reduce compatibility with the Linux version a little by
removing the mode parameter to efi_set_variable (which was unused on
FreeBSD, and not set to something useful in the code we'd
written). Also remove some efi error routines that were never
implemented and existed only to placate early GPL efibootmgr porting
experiments.
Suggested by: Matt Williams
Sponsored by: Netflix
This allows the _SKIP_DEPEND optimization to work, avoiding reading
the files when not needed. It also fixes META_MODE incorrectly
reading these files when not needed.
Sponsored by: Dell EMC Isilon
* Don't test MACHINE, it's irrelevant to userland and should never be
used in userland Makefiles.
* If we match armv[67] and CPUTYPE is undefined OR it doesn't have
'soft' in it, choose armhf.
* Add a note that the soft float on armv[67] may be broken.
Sponsored by: Netflix
:U:Mfoo expands to :Mfoo, apparently. Explicit check for CPUTYPE being
defined, and test for it's value not containing *soft* before calling CRTARCH
armhf.
Tested, somewhat. Unfortunately recent changes appear to have affected
cross-builds where it no longer works, per my tests after universe12a being
upgraded from 07/2017 to 11/2017 sources (DESTDIR isn't being used in WORLDTMP;
MK_SYSTEM_COMPILER might be causing issues right now).
MFC after: 1 week
MFC with: r325502
Reported by: imp
CPUTYPE (apparently) isn't defined in non-cross-builds, which caused
arm/armv[67] hosted/targeted builds to fail when evaluating CPUTYPE.
Add the :U modifier to CPUTYPE so it evaluates to "". This allows armv[67] to
get past the conditional successfully.
MFC after: 1 week
Reported by: bob prohaska <fbsd@www.zefox.net>
- MK_PROFILE is controlled in bsd.opts.mk, which is pulled in via bsd.own.mk,
which is pulled in via bsd.init.mk . All upstream Makefiles which build off
of this one use bsd.init.mk.
- COMPILER_{TYPE,VERSION} is set via bsd.compiler.mk .
This reduces the namespace pollution/complexity somewhat.
MFC after: 1 week
Despite the fact that it's a working solution, it doesn't follow the design
philosophy of only doing TARGET_* in Makefile.inc1 and special locations in
the source tree.
PR: 222925
Requested by: imp
- Define TARGET_CPUARCH and use in libclang_rt as the basis for CRTARCH
When cross-compiling, the wrong architecture was being embedded in the
libclang_rt binary filenames. It should be based on TARGET_ARCH (target), not
MACHINE_ARCH (host).
If TARGET_ARCH isn't defined (host-builds), fallback to MACHINE_ARCH.
- Define CRTARCH to armhf when TARGET/TARGET_ARCH are set to arm/armv[67]
TARGET_ABI/TARGET_CPU in Makefile.inc1 sets the ABI to gnueabihf, which
affects the clang lookup path per `getArchNameForCompilerRTLib(..)` in
contrib/llvm/tools/clang/lib/Driver/ToolChain.cpp, so chase clang and
Linux's assumed naming convention for hard-float arm architectures.
CROSSENV (in Makefile.inc1) sets CPUTYPE/MACHINE(_ARCH)? to the
TARGET*-relevant values when building the `libraries` target, so test
those variables instead.
- Add OLD_FILES/OLD_LIBS entries for TARGET/TARGET_ARCH == arm/armv[67]. This
impacts only arm/armv6 and arm/armv7.
PR: 222925
Several of the flags were being treated as CFLAGS, when they were actually
technically CXXFLAGS. Move them there.
Only apply -fno-sanitize=safe-stack with clang.
PR: 223179
TARGET_ABI/TARGET_CPU in Makefile.inc1 sets the ABI to gnueabihf, which affects
the clang lookup path per getArchNameForCompilerRTLib(..) in
contrib/llvm/tools/clang/lib/Driver/ToolChain.cpp .
This is a follow up to r324873.
PR: 222925
As of r325320 posix_fallocate returns EINVAL on ZFS to indicate that
the underlying filesystem does not support this operation, per
POSIX.1-2008. Document this case in the man page.
MFC after: 20 days
MFC with: r325320
Sponsored by: The FreeBSD Foundation
respected.
Please notice that libcasper is already in ObsoleteFiles so we don't add it
again.
Reported by: Herbert J. Skuhra <herbert@mailbox.org>
Reviewed by: bdrewery@
Differential Revision: https://reviews.freebsd.org/D12918
mt(1) man page.
LTO-8 Type M (also known as M8) is a pristine LTO-7 cartridge
formatted in a LTO-8 drive in a new, higher density format. It
has a separate density code, and is only readable in an LTO-8
drive.
lib/libmt/mtlib.c:
Add the LTO-8 Type M density code to the density table
in libmt.
usr.bin/mt/mt.1:
Add the LTO-8 Type M density code to the density
table in the mt(1) man page.
MFC after: 3 days
Sponsored by: Spectra Logic
For statically linked binaries, where all relocation are solved by static
linker, the linker expect that offset to TLS section is aligned. Additionaly,
to maintain absolute alignment, TLS TCB should by also aligned.
Obtained from: CheriBSD (initial version)
MFC after: 1 month
Reviewed by: brooks (previous version), kib
Differential Revision: https://reviews.freebsd.org/D12907
{}'s around the if (c == EOF) block to prevent potential 'trailing else'
issues from being introduced when refactoring. As my gets_s() code
is based on this, it makes sense to fix the same issue here first
here and now, then do an svn copy again to capture this history).
Suggested by: ed@ in D12785
When trying to build world for MIPS64 with clang I was getting
linker errors because of a missing reference to std::get_new_handler().
It turns out std::get_new_handler() was not listed in Version.map so it was
marked as a local symbol in libcxxrt.so.
Reviewed by: theraven, brooks (mentor), emaste
Approved by: trasz
Obtained from: CheriBSD
Differential Revision: https://reviews.freebsd.org/D11925
The bug is an out-of-bounds read detected with address sanitizer that
happens when 'sp' in p_b_coll_elems() includes NUL byte[s], e.g. if it's
equal to "GS\x00". In that case len will be equal to 4, and the
strncmp(cp->name, sp, len) call will succeed when cp->name is "GS" but the
cp->name[len] == '\0' comparison will cause the read to go out-of-bounds.
Checking the length using strlen() instead eliminates the issue.
The bug was found in LLVM with oss-fuzz:
https://reviews.llvm.org/D39380
MFC after: 1 week
Obtained from: Vlad Tsyrklevich through posting on openbsd-tech
The idea behinds mocks is that we don't need to ifdef a lot of code in
tools itself but those defines are hidden in the casper library.
Right now the mocks are implemented as define/inlines functions.
There was a very long discussion how this should be implemented.
This approach has some advantages like we don't need to link to any additional
libraries. Unfortunately there are also some disadvantages for example it is
easy to get library out of sync between two versions of functions or that we
need extra define to compile program with casper support.
This isn't an ideal solution but it's good enough for now and should simplify
capsicumizing programs. This also doesn't close us any other ways to do those
mocks and this should evolve in time.
Discussed with: pjd, emaste, ed, rwatson, bapt, cem, bdrewery
Differential Revision: https://reviews.freebsd.org/D8753
Previously, this didn't work because L2ARC devices' labels don't contain
pool GUIDs. Modify zfsd so that the pool GUID won't be required:
lib/libdevdctl/guid.h
Change INVALID_GUID from a uint64_t constant to a function that
returns an invalid Guid object. Remove the void constructor.
Nothing uses it, and it violates RAII.
cddl/usr.sbin/zfsd/case_file.h
cddl/usr.sbin/zfsd/case_file.cc
Allow CaseFile::Find to match a CaseFile based on Vdev GUID alone.
In CaseFile::ReEvaluate, attempt to online devices even if the newly
arrived device has no pool GUID.
cddl/usr.sbin/zfsd/vdev_iterator.cc
Iterate through a pool's cache devices as well as its regular
devices.
Reported by: avg
Reviewed by: avg
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D12791
bother passing it to crypt(). It won't succeed and may allow an attacker
to confirm that the user exists.
Reported by: jkim@
MFC after: 1 week
Security: CVE-2016-6210
The Makefile.inc1 TARGET_TRIPLE is for specifying which -target is used
during the build of world.
MFC after: 2 weeks
Reviewed by: dim, imp
Sponsored by: Dell EMC Isilon
Differential Revision: https://reviews.freebsd.org/D12792
RB_POWERCYCLE instructs the platform to power off and then power back
on a short time later, if that's possible. Otherwise, degrade to the
RB_POWEROFF behavior.
Sponsored by: Netflix
library -- libpmcstat.
This includes PMC logging module, symbols lookup functions,
ELF parsing, process management, PMC attachment, etc.
This allows to reuse code while building new hwpmc(4)-based applications.
Also add pmcstat_symbol_search_by_name() function that allows to find
mapped IP range for a given function name.
Reviewed by: kib
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D12718
In r322258 I made p1003_1b.aio_listio_max a tunable. However, further
investigation shows that there was never any good reason for that limit to
exist in the first place. It's used in two completely different ways:
* To size a UMA zone, which globally limits the number of concurrent
aio_suspend calls.
* To artifically limit the number of operations in a single lio_listio call.
There doesn't seem to be any memory allocation associated with this limit.
This change does two things:
* Properly names aio_suspend's UMA zone, and sizes it based on a new constant.
* Eliminates the artifical restriction on lio_listio. Instead, lio_listio
calls will now be limited by the more generous max_aio_queue_per_proc. The
old p1003_1b.aio_listio_max is now an alias for
vfs.aio.max_aio_queue_per_proc, so sysconf(3) will still work with
_SC_AIO_LISTIO_MAX.
Reported by: bde
Reviewed by: jhb
MFC after: 3 weeks
Sponsored by: Spectra Logic Corp
Differential Revision: https://reviews.freebsd.org/D12120
Several of the flags were being treated as CFLAGS, when they were actually
technically CXXFLAGS. Move them there.
Also, only apply -fno-sanitize=safe-stack with clang.
This is a draft diff.
PR: 223179
