Clang emits SSE instructions on amd64 in the common path of
pthread_mutex_unlock. If the thread does not otherwise use SSE,
this usage incurs a context-switch of the FPU/SSE state, which
reduces the performance of multiple real-world applications by a
non-trivial amount (3-5% in one application).
Instead of this change, I experimented with eagerly switching the
FPU state at context-switch time. This did not help. Most of the
cost seems to be in the read/write of memory--as kib@ stated--and
not in the #NM handling. I tested on machines with and without
XSAVEOPT.
One counter-argument to this change is that most applications already
use SIMD, and the number of applications and amount of SIMD usage
are only increasing. This is absolutely true. I agree that--in
general and in principle--this change is in the wrong direction.
However, there are applications that do not use enough SSE to offset
the extra context-switch cost. SSE does not provide a clear benefit
in the current libthr code with the current compiler, but it does
provide a clear loss in some cases. Therefore, disabling SSE in
libthr is a non-loss for most, and a gain for some.
I refrained from disabling SSE in libc--as was suggested--because
I can't make the above argument for libc. It provides a wide variety
of code; each case should be analyzed separately.
https://lists.freebsd.org/pipermail/freebsd-current/2015-March/055193.html
Suggestions from: dim, jmg, rpaulo
Approved by: kib (mentor)
MFC after: 2 weeks
Sponsored by: Dell Inc.
It looks like EVFILT_READ and EVFILT_WRITE trigger under the same
conditions as poll()'s POLLRDNORM and POLLWRNORM as described by POSIX.
The only difference is that POLLRDNORM has to be triggered on regular
files unconditionally, whereas EVFILT_READ only triggers when not EOF.
Introduce a new flag, NOTE_FILE_POLL, that can be used to make
EVFILT_READ and EVFILT_WRITE behave identically to poll(). This flag
will be used by cloudlibc's poll() function.
Reviewed by: jmg
Differential Revision: https://reviews.freebsd.org/D3303
of the timehands, from the kern_tc.c implementation to vdso. Add
comments giving hints where to look for the algorithm explanation.
To compensate the removal of rmb() in userspace binuptime(), add
explicit lfence instruction before rdtsc. On i386, add usual
complications to detect SSE2 presence; assume that old CPUs which do
not implement SSE2 also execute rdtsc almost in order.
Reviewed by: alc, bde (previous version)
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Summary:
Back in 2005, maxim@ attempted to fix shutdown() to return ENOTCONN in case the socket was not connected (r150152). This had to be rolled back (r150155), as it broke some of the existing programs that depend on this behavior. I reapplied this change on my system and indeed, syslogd failed to start up. I fixed this back in February (279016) and MFC'ed it to the supported stable branches. Apart from that, things seem to work out all right.
Since at least Linux and Mac OS X do the right thing, I'd like to go ahead and give this another try. To keep old copies of syslogd working, only start returning ENOTCONN for recent binaries.
I took a look at the XNU sources and they seem to test against both SS_ISCONNECTED, SS_ISCONNECTING and SS_ISDISCONNECTING, instead of just SS_ISCONNECTED. That seams reasonable, so let's do the same.
Test Plan:
This issue was uncovered while writing tests for shutdown() in CloudABI:
https://github.com/NuxiNL/cloudlibc/blob/master/src/libc/sys/socket/shutdown_test.c#L26
Reviewers: glebius, rwatson, #manpages, gnn, #network
Reviewed By: gnn, #network
Subscribers: bms, mjg, imp
Differential Revision: https://reviews.freebsd.org/D3039
This is required by our FORTIFY_SOURCE implementation as it
does more inlining. As a rule of thumb, FORTIFY_SOURCE doubles
the number of inlines except that in grep inlining
blows up for some reason.
to no longer claim they are experimental.
Reviewed by: rwatson@, wblock@
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D2985
Fix a warning triggered by the gcc + FORTIFY_SOURCE patches:
In function 'libusb20_parse_config_desc': lib/libusb/libusb20_desc.c:141:
warning: passing argument 1 of 'memcpy' discards qualifiers from pointer
target type
Submitted by: hselansky
We want to ensure we always use libarchive from ports in the ports tree.
It simplifies ports maintainance and anyway libarchive.pc was not reflecting the
different way libarchive can be built in base
SIGCHLD signal, should keep full 32 bits of the status passed to the
_exit(2).
Split the combined p_xstat of the struct proc into the separate exit
status p_xexit for normal process exit, and signalled termination
information p_xsig. Kernel-visible macro KW_EXITCODE() reconstructs
old p_xstat from p_xexit and p_xsig. p_xexit contains complete status
and copied out into si_status.
Requested by: Joerg Schilling
Reviewed by: jilles (previous version), pho
Tested by: pho
Sponsored by: The FreeBSD Foundation
This is based on work done by jeff@ and jhb@, as well as the numa.diff
patch that has been circulating when someone asks for first-touch NUMA
on -10 or -11.
* Introduce a simple set of VM policy and iterator types.
* tie the policy types into the vm_phys path for now, mirroring how
the initial first-touch allocation work was enabled.
* add syscalls to control changing thread and process defaults.
* add a global NUMA VM domain policy.
* implement a simple cascade policy order - if a thread policy exists, use it;
if a process policy exists, use it; use the default policy.
* processes inherit policies from their parent processes, threads inherit
policies from their parent threads.
* add a simple tool (numactl) to query and modify default thread/process
policities.
* add documentation for the new syscalls, for numa and for numactl.
* re-enable first touch NUMA again by default, as now policies can be
set in a variety of methods.
This is only relevant for very specific workloads.
This doesn't pretend to be a final NUMA solution.
The previous defaults in -HEAD (with MAXMEMDOM set) can be achieved by
'sysctl vm.default_policy=rr'.
This is only relevant if MAXMEMDOM is set to something other than 1.
Ie, if you're using GENERIC or a modified kernel with non-NUMA, then
this is a glorified no-op for you.
Thank you to Norse Corp for giving me access to rather large
(for FreeBSD!) NUMA machines in order to develop and verify this.
Thank you to Dell for providing me with dual socket sandybridge
and westmere v3 hardware to do NUMA development with.
Thank you to Scott Long at Netflix for providing me with access
to the two-socket, four-domain haswell v3 hardware.
Thank you to Peter Holm for running the stress testing suite
against the NUMA branch during various stages of development!
Tested:
* MIPS (regression testing; non-NUMA)
* i386 (regression testing; non-NUMA GENERIC)
* amd64 (regression testing; non-NUMA GENERIC)
* westmere, 2 socket (thankyou norse!)
* sandy bridge, 2 socket (thankyou dell!)
* ivy bridge, 2 socket (thankyou norse!)
* westmere-EX, 4 socket / 1TB RAM (thankyou norse!)
* haswell, 2 socket (thankyou norse!)
* haswell v3, 2 socket (thankyou dell)
* haswell v3, 2x18 core (thankyou scott long / netflix!)
* Peter Holm ran a stress test suite on this work and found one
issue, but has not been able to verify it (it doesn't look NUMA
related, and he only saw it once over many testing runs.)
* I've tested bhyve instances running in fixed NUMA domains and cpusets;
all seems to work correctly.
Verified:
* intel-pcm - pcm-numa.x and pcm-memory.x, whilst selecting different
NUMA policies for processes under test.
Review:
This was reviewed through phabricator (https://reviews.freebsd.org/D2559)
as well as privately and via emails to freebsd-arch@. The git history
with specific attributes is available at https://github.com/erikarn/freebsd/
in the NUMA branch (https://github.com/erikarn/freebsd/compare/local/adrian_numa_policy).
This has been reviewed by a number of people (stas, rpaulo, kib, ngie,
wblock) but not achieved a clear consensus. My hope is that with further
exposure and testing more functionality can be implemented and evaluated.
Notes:
* The VM doesn't handle unbalanced domains very well, and if you have an overly
unbalanced memory setup whilst under high memory pressure, VM page allocation
may fail leading to a kernel panic. This was a problem in the past, but it's
much more easily triggered now with these tools.
* This work only controls the path through vm_phys; it doesn't yet strongly/predictably
affect contigmalloc, KVA placement, UMA, etc. So, driver placement of memory
isn't really guaranteed in any way. That's next on my plate.
Sponsored by: Norse Corp, Inc.; Dell
Some external tools just do a 'ls /dev/vmm' to figure out the bhyve virtual
machines on the host. These tools break if the devmem device nodes also
appear in /dev/vmm.
Requested by: grehan
the 'user' sysctl tree, which have all been coming back 0 or empty
since r240176.
Differential Revision: https://reviews.freebsd.org/D2945
Reviewed by: sbruno
Approved by: jmallett (mentor)
MFC after: 3 days
This function is equivalent to fclose(3) function except that it
does not close the underlying file descriptor.
fdclose(3) is step forward to make FILE structure private.
Reviewed by: wblock, jilles, jhb, pjd
Approved by: pjd (mentor)
Differential Revision: https://reviews.freebsd.org/D2697
directory sys/contrib/libnv.
The goal of this operation is to NOT install header files which shouldn't
be used outside the nvlist library.
Approved by: pjd (mentor)
Update setkey and libipsec to understand aes-gcm-16 as an
encryption method.
A partial commit of the work in review D2936.
Submitted by: eri
Reviewed by: jmg
MFC after: 2 weeks
Sponsored by: Rubicon Communications (Netgate)
temporary file is created and then a rename() call move it to official file.
This operation didn't have any check to make sure data was written to disk
and if a power cycle happens system could end up with a 0 length passwd
or group database.
There is a pfSense bug with more infor about it:
https://redmine.pfsense.org/issues/4523
The following changes were made to protect passwd and group operations:
* lib/libutil/gr_util.c:
- Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file
- After rename(), fsync() call on directory for faster result
* lib/libutil/pw_util.c
- Replace mkstemp() by mkostemp() with O_SYNC flag to create temp file
* usr.sbin/pwd_mkdb/pwd_mkdb.c
- Added O_SYNC flag on dbopen() calls
- After rename(), fsync() call on directory for faster result
* lib/libutil/pw_util.3
- pw_lock() returns a file descriptor to master password file on success
Differential Revision: https://reviews.freebsd.org/D2978
Approved by: bapt
Sponsored by: Netgate
point support. The fenv-vfp.c file overrides -mfloat-abi so it can use
floating point instructions if it detects support at runtime. Make it also
override -mfpu in case the user has set -mfpu=none.
to be before the lavel, otherwise an extra word may be added between the
label and the data.
Obtained from: ABT Systems Ltd
Sponsored by: The FReeBSD Foundation
CLANG WARNINGS.
BUMP SHLIB_MAJOR version as the ABI potentially changed.
Also run an 'exp run' to double check if any external project are using
those functions. Thanks antoine@.
PR: 200807
Differential Revision: D2775
Reviewed by: kib, ngie
This symbol can sometimes be emitted by clang++, and was not yet
exported from libcxxrt. Attempt to be compatible with libsupc++ by
using the same CXXABI_1.3.6 symbol version.
Reported by: yuri@rawbw.com
PR: 200863
Reviewed by: emaste
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D2850
When enough time has passed for users to update their userland the kernel
fix will be applied. This will change the ABI to have x0 point to the args
and sp be correctly aligned.
It is expected this compatibility code can be removed when the kernel and
qemu usermode emulation have both been updated for the new ABI.
This fixes clang failures, and most likely other crashes.
Obtained from: ABT Systems Ltd
Sponsored by: The FreeBSD Foundation
due to a change in behavior of the 'vm_map_gpa()'.
Prior to r284539 if 'vm_map_gpa()' was called to map an address range in the
guest MMIO region then it would return NULL. This was used by the "movs"
emulation to detect if the 'src' or 'dst' operand was in MMIO space.
Post r284539 'vm_map_gpa()' started returning a non-NULL pointer even when
mapping the guest MMIO region.
Fix this by returning non-NULL only if [gaddr, gaddr+len) is entirely
within the 'lowmem' or 'highmem' regions and NULL otherwise.
Pointy hat to: neel
Reviewed by: grehan
Reported by: tychon, Ben Perrault (ben.perrault@gmail.com)
MFC after: 1 week
devmem is used to represent MMIO devices like the boot ROM or a VESA framebuffer
where doing a trap-and-emulate for every access is impractical. devmem is a
hybrid of system memory (sysmem) and emulated device models.
devmem is mapped in the guest address space via nested page tables similar
to sysmem. However the address range where devmem is mapped may be changed
by the guest at runtime (e.g. by reprogramming a PCI BAR). Also devmem is
usually mapped RO or RW as compared to RWX mappings for sysmem.
Each devmem segment is named (e.g. "bootrom") and this name is used to
create a device node for the devmem segment (e.g. /dev/vmm/testvm.bootrom).
The device node supports mmap(2) and this decouples the host mapping of
devmem from its mapping in the guest address space (which can change).
Reviewed by: tychon
Discussed with: grehan
Differential Revision: https://reviews.freebsd.org/D2762
MFC after: 4 weeks
- Fix a case where NaNs were not mixed correctly and signalling NaNs were
not converted to quiet NaNs.
- Eliminate two negations from ctan(z).
In collaboration with: bde
but must still satisfy csinh(conj(z)) == conj(csinh(z)) and csinh(-z) ==
-csinh(z). This allows eliminating two negations from csin(z).
In collaboration with: bde
Since METAMODE has been added, sys.mk loads bsd.mkopt.mk which ends load loading
bsd.own.mk which then defines SHLIBDIR before all the Makefile.inc everywhere.
This makes /lib being populated again.
Reported by: many
Use a constant array for the MIB. Newer LLVM decided that mib[] warranted
stack protections, with the obvious crash after the setup was done.
As a positive side effect, code size shrinks a bit.
I'm not sure why this hasn't bitten us yes, but it is certainly possible and
there are no real drawbacks to this change anyway.
Submitted by: pfg
Obtained from: NetBSD
MFC after: 1 week
lib/libfetch/http.c:1628:26: error: address of array 'purl->user'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.user = purl->user ?
~~~~~~^~~~ ~
lib/libfetch/http.c:1630:30: error: address of array 'purl->pwd'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.password = purl->pwd?
~~~~~~^~~~
lib/libfetch/http.c:1657:25: error: address of array 'url->user'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.user = url->user ?
~~~~~^~~~ ~
lib/libfetch/http.c:1659:29: error: address of array 'url->pwd'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.password = url->pwd ?
~~~~~^~~ ~
lib/libfetch/http.c:1669:25: error: address of array 'url->user'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.user = url->user ?
~~~~~^~~~ ~
lib/libfetch/http.c:1671:29: error: address of array 'url->pwd'
will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion]
aparams.password = url->pwd ?
~~~~~^~~ ~
Since url->user and url->pwd are arrays, they can never be NULL, so the
checks can be removed.
Reviewed by: bapt
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D2673
Off by default, build behaves normally.
WITH_META_MODE we get auto objdir creation, the ability to
start build from anywhere in the tree.
Still need to add real targets under targets/ to build packages.
Differential Revision: D2796
Reviewed by: brooks imp
MAM is Medium Auxiliary Memory and is most commonly found as flash
chips on tapes.
This includes support for reading attributes and decoding most
known attributes, but does not yet include support for writing
attributes or reporting attributes in XML format.
libsbuf/Makefile:
Add subr_prf.c for the new sbuf_hexdump() function. This
function is essentially the same function.
libsbuf/Symbol.map:
Add a new shared library minor version, and include the
sbuf_hexdump() function.
libsbuf/Version.def:
Add version 1.4 of the libsbuf library.
libutil/hexdump.3:
Document sbuf_hexdump() alongside hexdump(3), since it is
essentially the same function.
camcontrol/Makefile:
Add attrib.c.
camcontrol/attrib.c:
Implementation of READ ATTRIBUTE support for camcontrol(8).
camcontrol/camcontrol.8:
Document the new 'camcontrol attrib' subcommand.
camcontrol/camcontrol.c:
Add the new 'camcontrol attrib' subcommand.
camcontrol/camcontrol.h:
Add a function prototype for scsiattrib().
share/man/man9/sbuf.9:
Document the existence of sbuf_hexdump() and point users to
the hexdump(3) man page for more details.
sys/cam/scsi/scsi_all.c:
Add a table of known attributes, text descriptions and
handler functions.
Add a new scsi_attrib_sbuf() function along with a number
of other related functions that help decode attributes.
scsi_attrib_ascii_sbuf() decodes ASCII format attributes.
scsi_attrib_int_sbuf() decodes binary format attributes, and
will pass them off to scsi_attrib_hexdump_sbuf() if they're
bigger than 8 bytes.
scsi_attrib_vendser_sbuf() decodes the vendor and drive
serial number attribute.
scsi_attrib_volcoh_sbuf() decodes the Volume Coherency
Information attribute that LTFS writes out.
sys/cam/scsi/scsi_all.h:
Add a number of attribute-related structure definitions and
other defines.
Add function prototypes for all of the functions added in
scsi_all.c.
sys/kern/subr_prf.c:
Add a new function, sbuf_hexdump(). This is the same as
the existing hexdump(9) function, except that it puts the
result in an sbuf.
This also changes subr_prf.c so that it can be compiled in
userland for includsion in libsbuf.
We should work to change this so that the kernel hexdump
implementation is a wrapper around sbuf_hexdump() with a
statically allocated sbuf with a drain. That will require
a drain function that goes to the kernel printf() buffer
that can take a non-NUL terminated string as input.
That is because an sbuf isn't NUL-terminated until it is
finished, and we don't want to finish it while we're still
using it.
We should also work to consolidate the userland hexdump and
kernel hexdump implemenatations, which are currently
separate. This would also mean making applications that
currently link in libutil link in libsbuf.
sys/sys/sbuf.h:
Add the prototype for sbuf_hexdump(), and add another copy
of the hexdump flag values if they aren't already defined.
Ideally the flags should be defined in one place but the
implemenation makes it difficult to do properly. (See
above.)
Sponsored by: Spectra Logic Corporation
MFC after: 1 week
- fix types of rl_completion_entry_function and rl_add_defun
- call update pos before completion to refresh the screen
From Thomas Eriksson
Adjust API to a more modern readline (Ryo Onodera)
remove duplicate declaration
it takes a new argument allowing to specify the endianness of the database
to generate
Differential Revision: https://reviews.freebsd.org/D2730
Reviewed by: ian
