Stop using smp_ipi_mtx to protect global shootdown state, and move/multiply the global state into pcpu. Now each CPU can initiate shootdown IPI independently from other CPUs. Initiator enters critical section, then fills its local PCPU shootdown info (pc_smp_tlb_XXX), then clears scoreboard generation at location (cpu, my_cpuid) for each target cpu. After that IPI is sent to all targets which scan for zeroed scoreboard generation words. Upon finding such word the shootdown data is read from corresponding cpu' pcpu, and generation is set. Meantime initiator loops waiting for all zeroed generations in scoreboard to update. Initiator does not disable interrupts, which should allow non-invalidation IPIs from deadlocking, it only needs to disable preemption to pin itself to the instance of the pcpu smp_tlb data. The generation is set before the actual invalidation is performed in handler. It is safe because target CPU cannot return to userspace before handler finishes. In principle only NMI can preempt the handler, but NMI would see the kernel handler frame and not touch not-invalidated user page table. Handlers loop until they do not see zeroed scoreboard generations. This, together with hardware keeping one pending IPI in LAPIC IRR should prevent lost shootdowns. Notes. 1. The code does protect writes to LAPIC ICR with exclusion. I believe this is fine because we in fact do not send IPIs from interrupt handlers. More for !x2APIC mode where ICR access for write requires two registers write, we disable interrupts around it. If considered incorrect, I can add per-cpu spinlock around ipi_send(). 2. Scoreboard lines owned by given target CPU can be padded to the cache line, to reduce ping-pong. Reviewed by: markj (previous version) Discussed with: alc Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 3 weeks Differential revision: https://reviews.freebsd.org/D25510 |
||
---|---|---|
bin | ||
cddl | ||
contrib | ||
crypto | ||
etc | ||
gnu | ||
include | ||
kerberos5 | ||
lib | ||
libexec | ||
release | ||
rescue | ||
sbin | ||
secure | ||
share | ||
stand | ||
sys | ||
targets | ||
tests | ||
tools | ||
usr.bin | ||
usr.sbin | ||
.arcconfig | ||
.arclint | ||
.cirrus.yml | ||
.clang-format | ||
.gitattributes | ||
.gitignore | ||
COPYRIGHT | ||
LOCKS | ||
MAINTAINERS | ||
Makefile | ||
Makefile.inc1 | ||
Makefile.libcompat | ||
Makefile.sys.inc | ||
ObsoleteFiles.inc | ||
README | ||
README.md | ||
RELNOTES | ||
UPDATING |
FreeBSD Source:
This is the top level of the FreeBSD source directory. This file
was last revised on:
FreeBSD
FreeBSD is an operating system used to power modern servers, desktops, and embedded platforms. A large community has continually developed it for more than thirty years. Its advanced networking, security, and storage features have made FreeBSD the platform of choice for many of the busiest web sites and most pervasive embedded networking and storage devices.
For copyright information, please see the file COPYRIGHT in this directory. Additional copyright information also exists for some sources in this tree - please see the specific source directories for more information.
The Makefile in this directory supports a number of targets for building components (or all) of the FreeBSD source tree. See build(7), config(8), https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/makeworld.html, and https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig.html for more information, including setting make(1) variables.
Source Roadmap:
bin System/user commands.
cddl Various commands and libraries under the Common Development
and Distribution License.
contrib Packages contributed by 3rd parties.
crypto Cryptography stuff (see crypto/README).
etc Template files for /etc.
gnu Various commands and libraries under the GNU Public License.
Please see gnu/COPYING* for more information.
include System include files.
kerberos5 Kerberos5 (Heimdal) package.
lib System libraries.
libexec System daemons.
release Release building Makefile & associated tools.
rescue Build system for statically linked /rescue utilities.
sbin System commands.
secure Cryptographic libraries and commands.
share Shared resources.
stand Boot loader sources.
sys Kernel sources.
sys/<arch>/conf Kernel configuration files. GENERIC is the configuration
used in release builds. NOTES contains documentation of
all possible entries.
tests Regression tests which can be run by Kyua. See tests/README
for additional information.
tools Utilities for regression testing and miscellaneous tasks.
usr.bin User commands.
usr.sbin System administration commands.
For information on synchronizing your source tree with one or more of the FreeBSD Project's development branches, please see:
https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/current-stable.html