This patch adds a new socket option, SO_REUSEPORT_LB, which allow multiple
programs or threads to bind to the same port and incoming connections will be
load balanced using a hash function.
Most of the code was copied from a similar patch for DragonflyBSD.
However, in DragonflyBSD, load balancing is a global on/off setting and can not
be set per socket. This patch allows for simultaneous use of both the current
SO_REUSEPORT and the new SO_REUSEPORT_LB options on the same system.
Required changes to structures
Globally change so_options from 16 to 32 bit value to allow for more options.
Add hashtable in pcbinfo to hold all SO_REUSEPORT_LB sockets.
Limitations
As DragonflyBSD, a load balance group is limited to 256 pcbs
(256 programs or threads sharing the same socket).
Submitted by: Johannes Lundberg <johanlun0@gmail.com>
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D11003
- Add an implementation of atomic_fcmpset_32() using RAS for armv4/v5.
This fixes recent world breakage due to use of atomic_fcmpset() in
userland.
- While here, be more careful to not expose wrapper macros for 64-bit
atomic_*cmpset to userland for armv4/v5 as only 32-bit cmpset is
implemented.
This has been reviewed, but not runtime-tested, but should fix the arm.arm
and arm.armeb worlds that have been broken for a while.
Reviewed by: imp
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D15147
The return value of atomic_cmpset() and atomic_fcmpset() is an int (which
is really a bool) that has the values 0 or 1. Some of the inlines were
using the type being operated on (e.g. uint32_t) as either the return type
of the function, or the type of a local 'ret' variable used to hold the
return value. Fix all of these to just use plain 'int'. Due to C promotion
rules and the fact that the value can only be 0 or 1, these should all be
harmless.
Reviewed by: imp (only the v4 ones)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D15147
considered as originated by our host packet. And thus rcvif should be
NULL, since it is used by ipfw(4) to determine that packet was originated
from this host. Some of icmp6_reflect() consumers reuse mbuf and m_pkthdr
without resetting rcvif pointer. To avoid this always reset m_pkthdr.rcvif
pointer to NULL in icmp6_reflect(). Also remove such line and comment
describing this from icmp6_error(), since it does not longer matters.
PR: 227674
Reported by: eugen
MFC after: 1 week
Intel® Arria® 10 SoC.
Cadence Quad SPI Flash is not generic SPI controller, but SPI flash
controller, so don't use spibus here, instead provide quad spi flash
interface.
Since it is not on spibus, then mx25l flash device driver is not usable
here, so provide new n25q flash device driver with quad spi flash
interface.
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D10245
This combined with previous changes significantly depessimizes the behaviour
under contentnion.
In particular the lock1_processes test (locking/unlocking separate files)
from the will-it-scale suite was executed with 128 concurrency on a
4-socket Broadwell with 128 hardware threads.
Operations/second (lock+unlock) go from ~750000 to ~45000000 (6000%)
For reference single-process is ~1680000 (i.e. on stock kernel the resulting
perf is less than *half* of the single-threaded run),
Note this still does not really scale all that well as the locks were just
bolted on top of the current implementation. Significant room for improvement
is still here. In particular the top performance fluctuates depending on the
extent of false sharing in given run (which extends beyond the file).
Added chain+lock pairs were not padded w.r.t. cacheline size.
One big ticket item is the hash used for spreading threads: it used to be the
process pid (which basically serialized all threaded ops). Temporarily the
vnode addr was slapped in instead.
Tested by: pho
I have changed my given name from Bruce to Rebecca, and my FreeBSD account
from brucec to bcran.
Update committers-src.dot and calendar.freebsd to show these changes.
Reviewed by: rrs
Differential Revision: https://reviews.freebsd.org/D15125
kproc_suspend_check. In r329612 bufspacedaemon was turned into a thread
of the bufdaemon process causing both to call kproc_suspend_check with the
same proc argument and that function contains the following while loop:
while (SIGISMEMBER(p->p_siglist, SIGSTOP)) {
wakeup(&p->p_siglist);
msleep(&p->p_siglist, &p->p_mtx, PPAUSE, "kpsusp", 0);
}
So one thread wakes up the other and the other wakes up the first again,
locking up UP machines on shutdown.
Also register the shutdown handlers with SHUTDOWN_PRI_LAST + 100 so they
run after the syncer has shutdown, because the syncer can cause a
situation where bufdaemon help is needed to proceed.
PR: 227404
Reviewed by: kib
Tested by: cy, rmacklem
1. check if P_ADVLOCK is already set and if so, don't lock to set it
(stolen from DragonFly)
2. when trying for fast path unlock, check that we are doing unlock
first instead of taking the interlock for no reason (e.g. if we want
to *lock*). whilere make it more likely that falling fast path will
not take the interlock either by checking for state
Note the code is severely pessimized both single- and multithreaded.
sysentvec::sv_hwcap/sv_hwcap2 are pointers to u_long, so cpu_features* need
to be u_long to use the pointers. This also requires a temporary cast in
printing the bitfields, which is fine because the feature flag fields are
only 32 bits anyway.
In recently added sendrecv profile, there was a line purposefully
added to introduce a compilation error in which `-dev' is used to
debug the entry. Removing the entry.
Sponsored by: Smule, Inc.
The options `-d' (debug), `-e' (exit after compile), and `-v' (verbose)
when combined in any order (though best remembered as `-dev') will run
the conflated script through dtrace(1), test for error conditions, and
show the line that dtrace(1) failed at (with context).
If no errors are found, the output is the same as `-e[v]'.
When writing a new profile for dwatch(1), you can quickly test to
make sure it compiles by running `dwatch -devX profile_name' where
profiles live in /usr/libexec/dwatch or /usr/local/libexec/dwatch
(the latter being where profiles installed via ports should go).
Sponsored by: Smule, Inc.
these assumptions may not hold true once we've panic'd. Therefore, the
checks hold less value after a panic. Additionally, if one of the checks
fails while we are already panic'd, this creates a double-panic which can
interfere with debugging the original panic.
Therefore, this commit allows an administrator to suppress a response to
KASSERT checks after a panic by setting a tunable/sysctl. The
tunable/sysctl (debug.kassert.suppress_in_panic) defaults to being
enabled.
Reviewed by: kib
Sponsored by: Netflix, Inc.
Differential Revision: https://reviews.freebsd.org/D12920
FreeBSD exports the AT_HWCAP* auxvec items if provided by the ELF sysentvec
structure. Add the CPU features to be exported, so user space can more
easily check for them without using the hw.cpu_features and hw.cpu_features2
sysctls.
Prior to r332851:
* --exclude always win out over --include
* --exclude-dir always wins out over --include-dir
r332851 broke that behavior, resulting in:
* First of --exclude, --include wins
* First of --exclude-dir, --include-dir wins
As it turns out, both behaviors are wrong by modern grep standards- the
latest rule wins. e.g.:
`grep --exclude foo --include foo 'thing' foo`
foo is included
`grep --include foo --exclude foo 'thing' foo`
foo is excluded
As tested with GNU grep 3.1.
This commit makes bsdgrep follow this behavior.
Reported by: se
Not all feature flags are synced. Those for processors we don't currently
support are ignored currently. Those that are supported are synced best I
can tell. One flag was renamed to match the Linux flag name
(PPC_FEATURE2_VCRYPTO -> PPC_FEATURE2_VEC_CRYPTO).
There's no point checking for a bunch of file modes if we're not a
practicing believer of DIR_SKIP or DEV_SKIP.
This also reduces some style violations that were particularly ugly looking
when browsing through.
- cd9660 relies on an #include "iso.h" but does not build any .c files
out of source, so remove reach-over .PATH
- ffs does not rely on any sys/ headers, so remove -I from CFLAGS.
- ffs_tables from sys/ is used by ffs; move the SRCS entry from the top-
level Makefile to ffs' Makefile.inc.
Sponsored by: The FreeBSD Foundation
When disabling regulator when they are unused, check before is they are
enabled.
While here don't check the enable_cnt on the regulator entry as it is
checked by regnode_stop.
This solve the panic on any board using a fixed regulator that is driven
by a gpio when the regulator is unused.
Tested On: OrangePi One
Pointy Hat to: myself
Reported by: kevans, Milan Obuch (freebsd-arm@dino.sk)
EFLAGS copy that lives out of a basic block!" errors on i386.
Pull in r325446 from upstream clang trunk (by me):
[X86] Add 'sahf' CPU feature to frontend
Summary:
Make clang accept `-msahf` (and `-mno-sahf`) flags to activate the
`+sahf` feature for the backend, for bug 36028 (Incorrect use of
pushf/popf enables/disables interrupts on amd64 kernels). This was
originally submitted in bug 36037 by Jonathan Looney
<jonlooney@gmail.com>.
As described there, GCC also uses `-msahf` for this feature, and the
backend already recognizes the `+sahf` feature. All that is needed is
to teach clang to pass this on to the backend.
The mapping of feature support onto CPUs may not be complete; rather,
it was chosen to match LLVM's idea of which CPUs support this feature
(see lib/Target/X86/X86.td).
I also updated the affected test case (CodeGen/attr-target-x86.c) to
match the emitted output.
Reviewers: craig.topper, coby, efriedma, rsmith
Reviewed By: craig.topper
Subscribers: emaste, cfe-commits
Differential Revision: https://reviews.llvm.org/D43394
Pull in r328944 from upstream llvm trunk (by Chandler Carruth):
[x86] Expose more of the condition conversion routines in the public
API for X86's instruction information. I've now got a second patch
under review that needs these same APIs. This bit is nicely
orthogonal and obvious, so landing it. NFC.
Pull in r329414 from upstream llvm trunk (by Craig Topper):
[X86] Merge itineraries for CLC, CMC, and STC.
These are very simple flag setting instructions that appear to only
be a single uop. They're unlikely to need this separation.
Pull in r329657 from upstream llvm trunk (by Chandler Carruth):
[x86] Introduce a pass to begin more systematically fixing PR36028
and similar issues.
The key idea is to lower COPY nodes populating EFLAGS by scanning the
uses of EFLAGS and introducing dedicated code to preserve the
necessary state in a GPR. In the vast majority of cases, these uses
are cmovCC and jCC instructions. For such cases, we can very easily
save and restore the necessary information by simply inserting a
setCC into a GPR where the original flags are live, and then testing
that GPR directly to feed the cmov or conditional branch.
However, things are a bit more tricky if arithmetic is using the
flags. This patch handles the vast majority of cases that seem to
come up in practice: adc, adcx, adox, rcl, and rcr; all without
taking advantage of partially preserved EFLAGS as LLVM doesn't
currently model that at all.
There are a large number of operations that techinaclly observe
EFLAGS currently but shouldn't in this case -- they typically are
using DF. Currently, they will not be handled by this approach.
However, I have never seen this issue come up in practice. It is
already pretty rare to have these patterns come up in practical code
with LLVM. I had to resort to writing MIR tests to cover most of the
logic in this pass already. I suspect even with its current amount
of coverage of arithmetic users of EFLAGS it will be a significant
improvement over the current use of pushf/popf. It will also produce
substantially faster code in most of the common patterns.
This patch also removes all of the old lowering for EFLAGS copies,
and the hack that forced us to use a frame pointer when EFLAGS copies
were found anywhere in a function so that the dynamic stack
adjustment wasn't a problem. None of this is needed as we now lower
all of these copies directly in MI and without require stack
adjustments.
Lots of thanks to Reid who came up with several aspects of this
approach, and Craig who helped me work out a couple of things
tripping me up while working on this.
Differential Revision: https://reviews.llvm.org/D45146
Pull in r329673 from upstream llvm trunk (by Chandler Carruth):
[x86] Model the direction flag (DF) separately from the rest of
EFLAGS.
This cleans up a number of operations that only claimed te use EFLAGS
due to using DF. But no instructions which we think of us setting
EFLAGS actually modify DF (other than things like popf) and so this
needlessly creates uses of EFLAGS that aren't really there.
In fact, DF is so restrictive it is pretty easy to model. Only STD,
CLD, and the whole-flags writes (WRFLAGS and POPF) need to model
this.
I've also somewhat cleaned up some of the flag management instruction
definitions to be in the correct .td file.
Adding this extra register also uncovered a failure to use the
correct datatype to hold X86 registers, and I've corrected that as
necessary here.
Differential Revision: https://reviews.llvm.org/D45154
Pull in r330264 from upstream llvm trunk (by Chandler Carruth):
[x86] Fix PR37100 by teaching the EFLAGS copy lowering to rewrite
uses across basic blocks in the limited cases where it is very
straight forward to do so.
This will also be useful for other places where we do some limited
EFLAGS propagation across CFG edges and need to handle copy rewrites
afterward. I think this is rapidly approaching the maximum we can and
should be doing here. Everything else begins to require either heroic
analysis to prove how to do PHI insertion manually, or somehow
managing arbitrary PHI-ing of EFLAGS with general PHI insertion.
Neither of these seem at all promising so if those cases come up,
we'll almost certainly need to rewrite the parts of LLVM that produce
those patterns.
We do now require dominator trees in order to reliably diagnose
patterns that would require PHI nodes. This is a bit unfortunate but
it seems better than the completely mysterious crash we would get
otherwise.
Differential Revision: https://reviews.llvm.org/D45673
Together, these should ensure clang does not use pushf/popf sequences to
save and restore flags, avoiding problems with unrelated flags (such as
the interrupt flag) being restored unexpectedly.
Requested by: jtl
PR: 225330
MFC after: 1 week
Split the matching and non-matching cases out into their own functions to
reduce future complexity. As the name implies, procmatches will eventually
process more than one match itself in the future.
-> PROC_PDEATHSIG_STATUS for consistency with other procctl(2)
operations names.
Requested by: emaste
Sponsored by: The FreeBSD Foundation
MFC after: 13 days
The lock is required to ensure that the switch to the new credentials
and the transfer of the process's accounting data from the old
credentials to the new ones is done atomically. Otherwise, some updates
may be applied to the new credentials and then additionally transferred
from the old credentials if the updates happen after proc_set_cred() and
before racct_proc_ucred_changed().
The problem is especially pronounced for RACCT_RSS because
- there is a strict accounting for this resource (it's reclaimable)
- it's updated asynchronously by the vm daemon
- it's updated by setting an absolute value instead of applying a delta
I had to remove a call to rctl_proc_ucred_changed() from
racct_proc_ucred_changed() and make all callers of latter call the
former as well. The reason is that rctl_proc_ucred_changed, as it is
implemented now, cannot be called while holding the proc lock, so the
lock is dropped after calling racct_proc_ucred_changed. Additionally,
I've added calls to crhold / crfree around the rctl call, because
without the proc lock there is no gurantee that the new credentials,
owned by the process, will stay stable. That does not eliminate a
possibility that the credentials passed to the rctl will get stale.
Ideally, rctl_proc_ucred_changed should be able to work under the proc
lock.
Many thanks to kib for pointing out the above problems.
PR: 222027
Discussed with: kib
No comment: trasz
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D15048
Using a pointer after setting it NULL is probably not a good plan.
Spotted by inspection during changes for Flexible File Layout Ioerr handling.
This code path obviously isn't normally executed.
MFC after: 1 week
during ifnet detach.
Since destroying interface is not atomic operation and due to the
lack of synhronization during destroy, it is possible, that in the
time between bpfdetach() and if_free() some queued on destroying
interface mbuf will be used by ether_input_internal() and
bpf_peers_present() can dereference NULL bpf_if pointer. To protect
from this, assign pointer to empty bpf_if_ext structure instead of
NULL pointer after bpfdetach().
Reviewed by: melifaro, eugen
Obtained from: Yandex LLC
MFC after: 1 week
Sponsored by: Yandex LLC
Differential Revision: https://reviews.freebsd.org/D15083
Summary:
POWER9 also contains 32 slbs entries as explained by the POWER9 User Manual:
"For HPT translation, the POWER9 core contains a unified (combined for both
instruction and data), 32-entry, fully-associative SLB per thread"
Submitted by: Breno Leitao
Differential Revision: https://reviews.freebsd.org/D15128
Summary:
Powerpc64 has support for a register called Data Stream Control Register
(DSCR), which basically controls how the hardware controls the caching and
prefetch for stream operations.
Since mfdscr and mtdscr are privileged instructions, we need to emulate them,
and
keep the custom DSCR configuration per thread.
The purpose of this feature is to change DSCR depending on the operation, set
to DSCR Default Prefetch Depth to deepest on string operations, as memcpy.
Submitted by: Breno Leitao
Differential Revision: https://reviews.freebsd.org/D15081
