illumos/illumos-gate@770499e185770499e185https://www.illumos.org/issues/8021
The ARC buf data project (known simply as "ABD" since its genesis in the ZoL
community) changes the way the ARC allocates `b_pdata` memory from using linear
`void *` buffers to using scatter/gather lists of fixed-size 1KB chunks. This
improves ZFS's performance by helping to defragment the address space occupied
by the ARC, in particular for cases where compressed ARC is enabled. It could
also ease future work to allocate pages directly from `segkpm` for minimal-
overhead memory allocations, bypassing the `kmem` subsystem.
This is essentially the same change as the one which recently landed in ZFS on
Linux, although they made some platform-specific changes while adapting this
work to their codebase:
1. Implemented the equivalent of the `segkpm` suggestion for future work
mentioned above to bypass issues that they've had with the Linux kernel memory
allocator.
2. Changed the internal representation of the ABD's scatter/gather list so it
could be used to pass I/O directly into Linux block device drivers. (This
feature is not available in the illumos block device interface yet.)
FreeBSD notes:
- the actual (default) chunk size is 4KB (despite the text above saying 1KB)
- we can try to reimplement ABDs, so that they are not permanently
mapped into the KVA unless explicitly requested, especially on
platforms with scarce KVA
- we can try to use unmapped I/O and avoid intermediate allocation of a
linear, virtual memory mapped buffer
- we can try to avoid extra data copying by referring to chunks / pages
in the original ABD
Reviewed by: Matthew Ahrens <mahrens@delphix.com>
Reviewed by: George Wilson <george.wilson@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Reviewed by: John Kennedy <john.kennedy@delphix.com>
Reviewed by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Pavel Zakharov <pavel.zakharov@delphix.com>
Reviewed by: Chris Williamson <chris.williamson@delphix.com>
Approved by: Richard Lowe <richlowe@richlowe.net>
Author: Dan Kimmel <dan.kimmel@delphix.com>
MFC after: 3 weeks
I think that the change is still good, but reconciling it with a planned
merge of the ARC buf data scatter-ization is a bit more tedious
than I can handle.
MFC after: 17 days
The reason is that FreeBSD refcount.h shadows ZFS refcount.h and that
will lead to a build error after a planned import of the ARC buf data
scatter-ization.
It's possible that some day we will have an opposite problem where
a ZFS header would shadow an essential FreeBSD header.
So, we need to think about a better long term solution.
Discussed with: allanjude
MFC after: 17 days
The declaration was already inactive when INET6 was enabled
and it causes a build error in the other case because of
a conflict with the correct definition in stdlib.h.
Discussed with: dim, ume
MFC after: 2 weeks
From the linux tune2fs(8) manpage:
"Allow the kernel to initialize bitmaps and inode tables and keep a high
watermark for the unused inodes in a filesystem, to reduce e2fsck(8) time.
This first e2fsck run after enabling this feature will take the full time,
but subsequent e2fsck runs will take only a fraction of the original time,
depending on how full the file system is."
Submitted by: Fedor Uporov
Differential Revision: https://reviews.freebsd.org/D11211
When a PL310 cache is used on a system that provides hardware
coherency, the outer cache sync operation is useless, and can be
skipped. Moreover, on some systems, it is harmful as it causes
deadlocks between the Marvell coherency mechanism, the Marvell PCIe
or Crypto controllers and the Cortex-A9.
To avoid this, this commit introduces a new Device Tree property
'arm,io-coherent' for the L2 cache controller node, valid only for the
PL310 cache. It identifies the usage of the PL310 cache in an I/O
coherent configuration. Internally, it makes the driver disable the
outer cache sync operation.
Note, that other outer-cache operations are not removed, as they may
be needed for certain situations, such as booting secondary CPUs.
Moreover, in order to enable IO coherent operation, the decision
whether to use L2 cache maintenance callbacks is done in busdma
layer, which was enabled in one of the previous commits.
Submitted by: Michal Mazur <mkm@semihalf.com>
Marcin Wojtas <mw@semihalf.com>
Reviewed by: mmel
Obtained from: Semihalf
Differential revision: https://reviews.freebsd.org/D11245
There is a hardware problem between Cortex-A9 CPUs and on-chip devices
in Armada 38X SoCs that may cause hang on heavy load. This can be
however worked around by mapping all registers and PCI IO
as strongly ordered instead of device memory.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: mmel
Tested by: mw_semihalf.com
Obtained from: Semihalf
Differential revision: https://reviews.freebsd.org/D10218
vfs.zfs.min_auto_ashift is a sysctl only not a tunable so updated bsdinstall
to use the correct location /etc/sysctl.conf instead of /boot/loader.conf
Reported by: Aaron Caza
Reviewed by: allanjude
MFC after: 2 days
Sponsored by: Multiplay
Differential Revision: https://reviews.freebsd.org/D11278
Replace conditional branches with trampolines to unconditional branches when
jumping to labels within other compilation units. This increases the offset
range from +-1 MiB to +-128 MiB.
The BSDL dtc has grown the needed features (overlays mostly) and is able to
compile all of our base DTS.
You can use WITH_GPL_DTC is you need the GPL one or DTC= in make.conf(5)
to specify an alternate location for the compiler to use.
Discussed with: emaste, imp
r320070 removed the definition of maxbcachebuf from sys/param.h to
fix the build for arm.
This patch adds the definition of maxbcachebuf to sys/buf.h, which
should be ok, since sys/buf.h is not being included in arm/arm/elf_note.S.
Suggested by: kib
MFC after: 2 weeks
Do not queue dmar_map_entries with zeroed gseq to
dmar_qi_invalidate_locked(). Zero gseq stops the processing in the qi
task. Do not assign possibly uninitialized on-stack gseq to map
entries when requeuing them on unit tlb_flush queue. Random garbage
in gsec is interpreted as too high invalidation sequence number and
again stop the processing in the task.
Make the sequence numbers generation completely contained in
dmar_qi_invalidate_locked() and dmar_qi_emit_wait_seq(). Upper code
directly passes boolean requesting emiting wait command instead of
trying to provide hint to avoid it by passing NULL gseq pointer.
Microoptimize the requeueing to tlb_flush queue by doing it for the
whole queue.
Diagnosed and tested by: Brett Gutstein <bgutstein@rice.edu>
Discussed with: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Most of the lock slowpaths assert that the calling thread isn't an idle
thread. However, this may not be true if the system has panicked, and in
some cases the assertion appears before a SCHEDULER_STOPPED() check.
MFC after: 3 days
Sponsored by: Dell EMC Isilon
If the user issues a MTIOCEXTGET ioctl, and the tape drive in question has
a serial number that is longer than 80 characters, we malloc a buffer in
saextget() to hold the output of cam_strvis().
Since a mutex is held in that codepath, doing a M_WAITOK malloc could lead
to sleeping while holding a mutex. Change it to a M_NOWAIT malloc and bail
out if we fail to allocate the memory. Devices with serial numbers longer
than 80 bytes are very rare (I don't recall seeing one), so this
should be a very unusual case to hit. But it is a bug that should be fixed.
sys/cam/scsi/scsi_sa.c:
In saextget(), if we need to malloc a buffer to hold the output of
cam_strvis(), don't wait for the memory. Fail and return an error
if we can't allocate the memory immediately.
PR: kern/220094
Submitted by: Jia-Ju Bai <baijiaju1990@163.com>
MFC after: 3 days
Sponsored by: Spectra Logic
Since buildenv exports SYSROOT all of these uses will now look in
WORLDTMP by default.
sys/boot/efi/loader/Makefile
A LIBSTAND hack is no longer required for buildenv.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
VM_MAP_WIRE_SYSTEM mode when wiring the newly grown stack.
System maps do not create auto-grown stack. Any stack we handled,
even for P_SYSTEM, must be for user address space. P_SYSTEM processes
with mapped user space is either init(8) or an aio worker attached to
other user process with aio buffer pointing into stack area. In either
case, VM_MAP_WIRE_USER mode should be used.
Noted and reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
This is to allow downstream Makefiles to know for sure they are building
against a sysroot rather than only depending on ${DESTDIR} or other
assumptions.
This also exports it into buildenv.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
device nodes.
Otherwise, the current check of aio_offset == -1LL makes it possible
to pass negative file offsets down to the filesystems. This trips
assertions and is even unsafe for e.g. FFS which keeps metadata at
negative offsets.
Reported and tested by: pho
Reviewed by: jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
Differential revision: https://reviews.freebsd.org/D11266
Starting with DTS from Linux 4.11, the pins list, function, drive and pull
are no longer prefixed with "allwinner,".
Allow the pinctrl driver to handle both case.
The code still doesn't use d_off. That will come in a future commit.
The code also removes the checks for servers returning a fileno that
doesn't fit in 32bits, since that should work ok now.
Bump __FreeBSD_version since this patch changes the interface between
the NFS kernel modules.
Reviewed by: kib
use the armv6 busdma interface. This interface uses more memory than
the armv4 one, but bounces more data more often so may be more correct
than the armv4 one. It is intended for debugging purposes only at the
moment.
load and unload it all the time since the buffer never changes. In
addition, we were loading it with a hardware spin lock held, which
makes the sleepable lock in busdma (for the bounce pages) trigger a
witness warning, as well as ipend being called with it held by uart,
which made it impossible to unload.
These differences don't matter with the v4 busdma implementation, but
they do with the v6 implementation since the latter likes to bounce
transactions more, and will always do so for Atmel's driver.
It's more efficient as well as being more correct.
We can have support for reading ext4 "huge" files but we can't write
(anything) on ext4. and some filesystem. Formally enable the feature so
that we can mount such filesystems.
Submitted by: Fedor Uponov
Differential Revision: https://reviews.freebsd.org/D11209
that disk writes are more likely to be sequential. This change is
beneficial on both the solid state and mechanical disks that I've
tested. (A similar change in allocation policy was made by DragonFly
BSD in 2013 to speed up Poudriere with "stressful memory parameters".)
Increase the width of blst_meta_alloc()'s parameter "skip" and the local
variables whose values are derived from it to 64 bits. (This matches the
width of the field "skip" that is stored in the structure "blist" and
passed to blst_meta_alloc().)
Eliminate a pointless check for a NULL blist_t.
Simplify blst_meta_alloc()'s handling of the ALL-FREE case.
Address nearby style errors.
Reviewed by: kib, markj
MFC after: 5 weeks
Differential Revision: https://reviews.freebsd.org/D11247
timecounter instead of the GPT timer, freeing up the more flexible GPT
hardware for other uses. The EPIT driver is a standard (always in the
kernel) driver, and the existing GPT driver is now optional and included
only if you ask for device imx_gpt.
global timer was successful, since the implementation tries to read it.
Notably, if the platform has a variable-frequency global timer (because
of dynamic frequency scaling), it doesn't set up the global timer for use
as a system timecounter, and in that case it also can't use it for DELAY.
Such platforms use different timer hardware for both timecounter and DELAY.
list.h includes a number of FreeBSD headers as a workaround for the
LIST_HEAD name collision. To reduce pollution, avoid including list.h
in commonly used headers when it is not explicitly needed.
Reviewed by: hselasky
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D11249
arm build.
In the arm build, elf_note.S includes sys/param.h and then does an
elf macro called ELFNOTE(). Although the compile error doesn't make
sense to me, I believe it just means that an "extern ..." can't exist
in param.h for this inclusion case.
I suspect adding #if !defined(LOCORE) might fix the build, but this
commit just takes the definition out.
I will ask freebsd-current@ what is the best was to deal with this
and do a subsequent commit after that.
Reported by: melounmichal@gmail.com
Clang 4.0 accepts the smc instruction with or without specifying
.arch_extension sec, but Clang 5.0 produces an error without it.
MFC after: 3 weeks
Sponsored by: The FreeBSD Foundation
Add a make.conf DTC variable that control which DTC (Device Tree Compiler)
to use.
Reviewed by: bdrewery, imp
Differential Revision: https://reviews.freebsd.org/D9577
