- Distribute RX load across multiple cores, if present. This reverts
r217212, which is no longer relevant (I think because of the newer
SDK).
- Use newer APIs for pinning taskqueue entries to specific cores.
- Deepen RX buffers.
This more than doubles NAT forwarding throughput on my EdgeRouter Lite from,
with typical packet mixture, 90 Mbps to over 200 Mbps. The result matches
forwarding throughput in Linux without the UBNT hardware offload on the same
hardware, and thus likely reflects hardware limits.
Reviewed by: jhibbits
iflib is already a module, but it is unconditionally compiled into the
kernel. There are drivers which do not need iflib(4), and there are
situations where somebody might not want iflib in kernel because of
using the corresponding driver as module.
Reviewed by: marius
Discussed with: erj
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential revision: https://reviews.freebsd.org/D19041
Fix dublicate value in what is apparent copypaste mistake. The last value
in mask is supposed to be for counter 7, not counter 3.
PR: 229790
Submitted by: David Binderman <dcb314@hotmail.com>
MFC after: 1 week
cpuid and local cpu variable are unsigned so checking if value is less than zero
always yields false.
PR: 211088
Submitted by: David Binderman <dcb314@hotmail.com>
MFC after: 1 week
Per discussions on mips@, 32-bit mips SMP is now unsupported. The
files in the tree will compile for a while longer, but when the
atomic_swap_64 or similar atomic enters into the MI part of the tree,
as currently foreseen sometime next year, these ports will start to no
longer link. The JZ4780 is the only such system we have.
The UP version of this chip is unaffected by this, and will remain
supported.
Discussed on: mips@
Relnotes: yes
This is an older broadcom part that implements the mips32 ISA. 32-bit
FreeBSD/mips now requires mips32r2, so retire this config. Most of the
broadcom port is shared with newer ports, so what little code may be
unique to this part has not been GC'd at this time.
Discussed on: freebsd-mips@
Differential Revision: https://reviews.freebsd.org/D18543
This was useful in bring up. However, it causes more issues than the
support is worth (64-bit atomics being chief among them).
Discussed on: freebsd-mips@
Differential Revision: https://reviews.freebsd.org/D18543
gxemul was a nice stop-gap while qemu support for mips was firmed
up. Now MALTA* + qemu is the platform of choice retire gxemul support.
It's unknown when this was last confirmed working.
Discussed on: freebsd-mips@
Differential Revision: https://reviews.freebsd.org/D18543
relevant and is unused. It's also getting in the way of progress in
some admittedly minor ways. Better to retire it to reduce the burden
on the project.
Discussed on: freebsd-mips@
Differential Revision: https://reviews.freebsd.org/D18543
fcmpset returns true/false as a int, so make the return types and
variables match the int to be consistent with other arch.
Reviewed by: cognet@
Differential Revision: https://reviews.freebsd.org/D18557
MIPS64 has 64-bit longs, so use uint64_t for it, otherwise uint32_t.
sizeof(long) == sizeof(ptr) for all platforms, so define
atomic_swap_ptr in terms of atomic_swap_long.
Submitted by: hps@
There may be cases where cpu_model[] may not be 32bit aligned, so it is
better to not try to access it as such in order to avoid unaligned access.
Sponsored by: Smartcom - Bulgaria AD
These SoCs have CHIPID registers, which store the Chip model, according
to the manufacturer; make use of those in order to better identify
the chip we're actually running on.
If we're unable to read the CHIPID registers for some reason we will
use the string "unknown " as a value for hw.model.
Reported by: yamori813@yahoo.co.jp
Sponsored by: Smartcom - Bulgaria AD
Replace a call to DELAY(1) with a new cpu_lock_delay() KPI. Currently
cpu_lock_delay() is defined to DELAY(1) on all platforms. However,
platforms with a DELAY() implementation that uses spin locks should
implement a custom cpu_lock_delay() doesn't use locks.
Reviewed by: kib
MFC after: 3 days
Add a new 'debugger_on_trap' knob separate from 'debugger_on_panic'
and make the calls to kdb_trap() in MD fatal trap handlers prior to
calling panic() conditional on this new knob instead of
'debugger_on_panic'. Disable the new knob by default. Developers who
wish to recover from a fatal fault by adjusting saved register state
and retrying the faulting instruction can still do so by enabling the
new knob. However, for the more common case this makes the user
experience for panics due to a fatal fault match the user experience
for other panics, e.g. 'c' in DDB will generate a crash dump and
reboot the system rather than being stuck in an infinite loop of fatal
fault messages and DDB prompts.
Reviewed by: kib, avg
MFC after: 2 months
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D17768
All platforms except powerpc use the same values and powerpc shares a
majority of them.
Go ahead and declare AT_NOTELF, AT_UID, and AT_EUID in favor of the
unused AT_DCACHEBSIZE, AT_ICACHEBSIZE, and AT_UCACHEBSIZE for powerpc.
Reviewed by: jhb, imp
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D17397
Update the BOOTSTRAPPING check for libelf to require the fix for
mips64el object files committed in r338478 and re-enable kernel
modules in the MALTA64EL config file.
Reviewed by: emaste
Approved by: re (gjb)
Differential Revision: https://reviews.freebsd.org/D17054
Exposing max_offset and min_offset defines in public headers is
causing clashes with variable names, for example when building QEMU.
Based on the submission by: royger
Reviewed by: alc, markj (previous version)
Sponsored by: The FreeBSD Foundation (kib)
MFC after: 1 week
Approved by: re (marius)
Differential revision: https://reviews.freebsd.org/D16881
given in random(4).
This includes updating of the relevant man pages, and no-longer-used
harvesting parameters.
Ensure that the pseudo-unit-test still does something useful, now also
with the "other" algorithm instead of Yarrow.
PR: 230870
Reviewed by: cem
Approved by: so(delphij,gtetlow)
Approved by: re(marius)
Differential Revision: https://reviews.freebsd.org/D16898
error in the function hypercall_memfree(), where the wrong arena was being
passed to kmem_free().
Introduce a per-page flag, VPO_KMEM_EXEC, to mark physical pages that are
mapped in kmem with execute permissions. Use this flag to determine which
arena the kmem virtual addresses are returned to.
Eliminate UMA_SLAB_KRWX. The introduction of VPO_KMEM_EXEC makes it
redundant.
Update the nearby comment for UMA_SLAB_KERNEL.
Reviewed by: kib, markj
Discussed with: jeff
Approved by: re (marius)
Differential Revision: https://reviews.freebsd.org/D16845
The boot-time ifunc resolver assumes that it only needs to apply
IRELATIVE relocations to PLT entries. With an upcoming optimization,
this assumption no longer holds, so add the support required to handle
PC-relative relocations targeting GNU_IFUNC symbols.
- Provide a custom symbol lookup routine that can be used in early boot.
The default lookup routine uses kobj, which is not functional at that
point.
- Apply all existing relocations during boot rather than filtering
IRELATIVE relocations.
- Ensure that we continue to apply ifunc relocations in a second pass
when loading a kernel module.
Reviewed by: kib
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D16749
became unused in FreeBSD 12.x as a side-effect of the NUMA-related
changes.)
Reviewed by: kib, markj
Discussed with: jeff, re@
Differential Revision: https://reviews.freebsd.org/D16825
This is an amalgam of a patch by Doug Ambrisko to
generalize uart_acpi_find_device, imp moving the
ACPI table to uart_dev_ns8250.c and advice by jhb
to work around a bug in the EPYC 3151 BIOS
(the BIOS incorrectly marks the serial ports as
disabled)
Reviewed by: imp
MFC after: 8 weeks
Differential Revision: https://reviews.freebsd.org/D16432
Rather than hard-coding the number of CPUs to 2, look up the PVPE field
in MVPConf0, as the valid VPE numbers are from 0 to PVPE inclusive.
Submitted by: "James Clarke" <jrtc4@cam.ac.uk>
Reviewed by: br
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D16644
Move definitions from cpuregs.h into the cca.h, and include cca.h into vm.h.
This is required to make MIPS MD memattr definitions usable in userspace.
Sponsored by: The FreeBSD Foundation, Mellanox Technologies
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D15583
The timespecadd(3) family of macros were imported from NetBSD back in
r35029. However, they were initially guarded by #ifdef _KERNEL. In the
meantime, we have grown at least 28 syscalls that use timespecs in some
way, leading many programs both inside and outside of the base system to
redefine those macros. It's better just to make the definitions public.
Our kernel currently defines two-argument versions of timespecadd and
timespecsub. NetBSD, OpenBSD, and FreeDesktop.org's libbsd, however, define
three-argument versions. Solaris also defines a three-argument version, but
only in its kernel. This revision changes our definition to match the
common three-argument version.
Bump _FreeBSD_version due to the breaking KPI change.
Discussed with: cem, jilles, ian, bde
Differential Revision: https://reviews.freebsd.org/D14725
Track session objects in the framework, and pass handles between the
framework (OCF), consumers, and drivers. Avoid redundancy and complexity in
individual drivers by allocating session memory in the framework and
providing it to drivers in ::newsession().
Session handles are no longer integers with information encoded in various
high bits. Use of the CRYPTO_SESID2FOO() macros should be replaced with the
appropriate crypto_ses2foo() function on the opaque session handle.
Convert OCF drivers (in particular, cryptosoft, as well as myriad others) to
the opaque handle interface. Discard existing session tracking as much as
possible (quick pass). There may be additional code ripe for deletion.
Convert OCF consumers (ipsec, geom_eli, krb5, cryptodev) to handle-style
interface. The conversion is largely mechnical.
The change is documented in crypto.9.
Inspired by
https://lists.freebsd.org/pipermail/freebsd-arch/2018-January/018835.html .
No objection from: ae (ipsec portion)
Reported by: jhb
Doing so ensures that all threads sharing the pmap have a consistent
view of the mapping. This fixes the problem described in the commit
log message for r329254 without the overhead of an extra fault in the
common case. (Once the riscv pmap_enter() implementation is similarly
modified, the workaround added in r329254 can be removed, reducing the
overhead of CoW faults.)
See also r335784 for amd64. The mips implementation of pmap_enter()
already reused the PV entry from the old mapping.
Reviewed by: kib, markj
MFC after: 3 weeks
Differential Revision: https://reviews.freebsd.org/D16199
- Change pcpu zone consumers to use a stride size of PAGE_SIZE.
(defined as UMA_PCPU_ALLOC_SIZE to make future identification easier)
- Allocate page from the correct domain for a given cpu.
- Don't initialize pc_domain to non-zero value if NUMA is not defined
There are some misconceptions surrounding this field. It is the
_VM_ NUMA domain and should only ever correspond to valid domain
values as understood by the VM.
The former slab size of sizeof(struct pcpu) was somewhat arbitrary.
The new value is PAGE_SIZE because that's the smallest granularity
which the VM can allocate a slab for a given domain. If you have
fewer than PAGE_SIZE/8 counters on your system there will be some
memory wasted, but this is obviously something where you want the
cache line to be coming from the correct domain.
Reviewed by: jeff
Sponsored by: Limelight Networks
Differential Revision: https://reviews.freebsd.org/D15933
On arm64 (and possible other architectures) we are unable to use static
DPCPU data in kernel modules. This is because the compiler will generate
PC-relative accesses, however the runtime-linker expects to be able to
relocate these.
In preparation to fix this create two macros depending on if the data is
global or static.
Reviewed by: bz, emaste, markj
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D16140
It looks like the intent was to allow ZSTD support to be
compiled into the kernel with options ZSTDIO. But it doesn't look
like that was ever implemented or I'm missing how to do it.
I did a cursory audit of kernel config files and made a decision to
enable ZSTDIO in riscv GENERIC and mips MALTA configurations. All other
kernel configurations already had this option in their kernel configs
but they didn't do anything useful as the feature was declared as
"standard" prior to this.
Reviewed by: cem allanjude
Differential Revision: https://reviews.freebsd.org/D16007
When 2GB of memory is enabled for QEMU's Malta emulation, the physical
memory ends at an address of 2^32 - 1. This causes an integer overflow
to zero when computing the upper bound of the second phys_avail[] range.
As a result, FreeBSD/mips kernels were only using the first 256MB of
RAM and ignoring the remaining 1.75GB. To work around this, truncate
the extended memory size to 2GB minus one page for 32-bit mips kernels.
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D16027
pmc_process_interrupt takes 5 arguments when only 3 are needed.
cpu is always available in curcpu and inuserspace can always be
derived from the passed trapframe.
While facially a reasonable cleanup this change was motivated
by the need to workaround a compiler bug.
core2_intr(cpu, tf) ->
pmc_process_interrupt(cpu, ring, pmc, tf, inuserspace) ->
pmc_add_sample(cpu, ring, pm, tf, inuserspace)
In the process of optimizing the tail call the tf pointer was getting
clobbered:
(kgdb) up
at /storage/mmacy/devel/freebsd/sys/dev/hwpmc/hwpmc_mod.c:4709
4709 pmc_save_kernel_callchain(ps->ps_pc,
(kgdb) up
1205 error = pmc_process_interrupt(cpu, PMC_HR, pm, tf,
resulting in a crash in pmc_save_kernel_callchain.
memcpy was an alias for bcopy with arg swap. This code handles
overlapping copies, so making memmove an alias is safe. We can
eliminate the call from libkern's memmove to this bcopy as a result.
Differential Revision: https://reviews.freebsd.org/D15374
This change updates arm, arm64 and mips achitectures. Additionally, it
removes redundant checks for kdb_active where it already results in
kdb_reenter() and adds kdb_reenter() calls where they were missing.
Some architectures check the return value of kdb_trap(), but some don't.
I haven't changed any of that.
Some trap handling routines have a return code. I am not sure if I
provided correct ones for returns after kdb_reenter(). kdb_reenter
should never return unless kdb_jmpbufp is NULL for some reason.
Only compile tested for all affected architectures. There can be bugs
resulting from my poor understanding of architecture specific details.
Reported by: jhb
Reviewed by: jhb, eadler
MFC after: 4 weeks
Differential Revision: https://reviews.freebsd.org/D15431
Dumpers may wish to print messages from an initialization hook; this
change ensures that such messages aren't mixed with output from the
generic dump code.
MFC after: 1 week
Note that GDB at least implements single stepping for MIPS using software
breakpoints explicitly rather than using PT_STEP, so this has only been
tested via tests in ptrace_test which now pass rather than fail.
- Fix several places to use uintptr_t instead of int for virtual addresses.
- Check for errors from ptrace_read_int() when setting a breakpoint for a
step.
- Properly check for errors from ptrace_write_int() as it returns non-zero,
not negative values on failure.
- Change the error returns for ptrace_read_int() and ptrace_write_int() from
ENOMEM to EFAULT.
- Clear a single step breakpoint when it traps rather than waiting for it
to be cleared from ptrace(). This matches the behavior of the arm port
and in general seems a bit more reliable than waiting for ptrace() to
clear it via FIX_SSTEP.
- Drop the PROC_LOCK around ptrace_write_int() in ptrace_clear_single_step()
since it can sleep.
- Reorder the breakpoint handler in trap() to only read the instruction if
the address matches the current thread's breakpoint address.
- Replace various #if 0'd debugging printfs with KTR_PTRACE traces.
Tested on: mips64
The MIPS ptrace_single_step() unlocks the PROC_LOCK while reading and
writing instructions from userland. One failure case was not reacquiring
the lock before returning.
- Use TRAP_TRACE for traps after stepping via PT_STEP.
- Use TRAP_BRKPT for software breakpoint traps and watchpoint traps.
This was tested via the recently added siginfo ptrace() tests. PT_STEP on
MIPS has several bugs that prevent it from working yet, but this does fix
the ptrace__breakpoint_siginfo test on MIPS.
Half of implementations always failed (returned (-1)) and they were
previously used in only one place.
Reviewed by: kib, andrew
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15102
Also remove the commented out documentation. The documentation arrived
with the import of the copy.9 manpage. I suspect the implementations
came from NetBSD while bootstrapping the Arm and MIPS ports.
Reviewed by: andrew, jmallett
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D15108
xdma(4) interface.
This allows us to switch between Altera mSGDMA or SoftDMA engines used by
atse(4) device.
This also makes atse(4) driver become 25% smaller.
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D9618
Change OF_getencprop_alloc semantics to be combination of malloc and
OF_getencprop and return size of the property, not number of elements
allocated.
For the use cases where number of elements is preferred introduce
OF_getencprop_alloc_multi helper function that copies semantics
of OF_getencprop_alloc prior to this change.
This is to make OF_getencprop_alloc and OF_getencprop_alloc_multi
function signatures consistent with OF_getencprop_alloc and
OF_getencprop_alloc_multi.
Functionality-wise this patch is mostly rename of OF_getencprop_alloc
to OF_getencprop_alloc_multi except two calls in ofw_bus_setup_iinfo
where 1 was used as a block size.
OF_getprop_alloc takes element size argument and returns number of
elements in the property. There are valid use cases for such behavior
but mostly API consumers pass 1 as element size to get string
properties. What API users would expect from OF_getprop_alloc is to be
a combination of malloc + OF_getprop with the same semantic of return
value. This patch modifies API signature to match these expectations.
For the valid use cases with element size != 1 and to reduce
modification scope new OF_getprop_alloc_multi function has been
introduced that behaves the same way OF_getprop_alloc behaved prior to
this patch.
Reviewed by: ian, manu
Differential Revision: https://reviews.freebsd.org/D14850
opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.
Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c. A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.
Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.
Reviewed by: kib, cem, jhb, jtl
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14941
assym is only to be included by other .s files, and should never
actually be assembled by itself.
Reviewed by: imp, bdrewery (earlier)
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D14180
Remove NO_FUEWORD so the 'e' variants are wrapped by the non-'e'
variants. This is more correct and leaves sparc64 as the outlier.
Reviewed by: jmallett, kib
Obtained from: CheriBSD
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D14603
correctly for the data contained on each memory page.
There are several components to this change:
* Add a variable to indicate the start of the R/W portion of the
initial memory.
* Stop detecting NX bit support for each AP. Instead, use the value
from the BSP and, if supported, activate the feature on the other
APs just before loading the correct page table. (Functionally, we
already assume that the BSP and all APs had the same support or
lack of support for the NX bit.)
* Set the RW and NX bits correctly for the kernel text, data, and
BSS (subject to some caveats below).
* Ensure DDB can write to memory when necessary (such as to set a
breakpoint).
* Ensure GDB can write to memory when necessary (such as to set a
breakpoint). For this purpose, add new MD functions gdb_begin_write()
and gdb_end_write() which the GDB support code can call before and
after writing to memory.
This change is not comprehensive:
* It doesn't do anything to protect modules.
* It doesn't do anything for kernel memory allocated after the kernel
starts running.
* In order to avoid excessive memory inefficiency, it may let multiple
types of data share a 2M page, and assigns the most permissions
needed for data on that page.
Reviewed by: jhb, kib
Discussed with: emaste
MFC after: 2 weeks
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D14282
We don't support float in the boot loaders, so don't include
interfaces for float or double in systems headers. In addition, take
the unusual step of spiking double and float to prevent any more
accidental seepage.
Make vm_wait() take the vm_object argument which specifies the domain
set to wait for the min condition pass. If there is no object
associated with the wait, use curthread' policy domainset. The
mechanics of the wait in vm_wait() and vm_wait_domain() is supplied by
the new helper vm_wait_doms(), which directly takes the bitmask of the
domains to wait for passing min condition.
Eliminate pagedaemon_wait(). vm_domain_clear() handles the same
operations.
Eliminate VM_WAIT and VM_WAITPFAULT macros, the direct functions calls
are enough.
Eliminate several control state variables from vm_domain, unneeded
after the vm_wait() conversion.
Scetched and reviewed by: jeff
Tested by: pho
Sponsored by: The FreeBSD Foundation, Mellanox Technologies
Differential revision: https://reviews.freebsd.org/D14384
significant source of cache line contention from vm_page_alloc(). Use
accessors and vm_page_unwire_noq() so that the mechanism can be easily
changed in the future.
Reviewed by: markj
Discussed with: kib, glebius
Tested by: pho (earlier version)
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14273
We don't support older compilers. Most of the code in these files is
for pre-3.0 gcc, which is at least 15 years obsolete. Move to using
phk's sys/_stdargs.h for all these platforms.
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D14323
The _Alignas specifier must come before the declaration and not after. It
works if _Alignas() expands to __attribute__(aligned(x)) which was the only
case I tested before.
Approved By: jhb (mentor)
global to per-domain state. Protect reservations with the free lock
from the domain that they belong to. Refactor to make vm domains more
of a first class object.
Reviewed by: markj, kib, gallatin
Tested by: pho
Sponsored by: Netflix, Dell/EMC Isilon
Differential Revision: https://reviews.freebsd.org/D14000
The compiler/linker can align fake_preload anyway it would like. When
building the kernel with gcc+bfd this always happened to be a multiple of 8.
When I built the kernel with clang and linked with lld fake_preload
happened to only be aligned to 4 bytes which caused a an ADDRS trap because
the compiler will emit sd instructions to store to this buffer.
Reviewed By: jhb, imp
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D14018
