We round up the psize to the nearest multiple of the asize or to the
lsize, whichever is smaller. Once that's done, we allocate a new
buffer of the appropriate size, zero the tail, and copy the data
into it. This adds a small performance cost to these kinds of writes,
but fixes the bookkeeping problems.
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Co-authored-by: Matthew Ahrens <matthew.ahrens@delphix.com>
Signed-off-by: Paul Dagnelie <pcd@delphix.com>
Closes#12522Closes#8462
Kernel commits
39f75da7bcc8 ("isystem: trim/fixup stdarg.h and other headers")
c0891ac15f04 ("isystem: ship and use stdarg.h")
564f963eabd1 ("isystem: delete global -isystem compile option")
(for now can be found in linux-next.git tree, will land into the
Linus' tree during the ongoing 5.15 cycle with one of akpm merges)
removed the -isystem flag and disallowed the inclusion of any
compiler header files. They also introduced a minimal
<linux/stdarg.h> as a replacement for <stdarg.h>.
include/os/linux/spl/sys/cmn_err.h in the ZFS source tree includes
<stdarg.h> unconditionally. Introduce a test for <linux/stdarg.h>
and include it instead of the compiler's one to prevent module
build breakage.
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Alexander Lobakin <alobakin@pm.me>
Closes#12531
When port reuse is enabled in a one-to-one-style socket, sctp_listen()
may call sctp_swap_inpcb_for_listen() to move the PCB out of the "TCP
pool". In so doing it will drop the PCB lock, yielding an LOR since we
now hold several socket locks. Reorder sctp_listen() so that it
performs this operation before beginning the conversion to a listening
socket. Also modify sctp_swap_inpcb_for_listen() to return with PCB
write-locked, since that's what sctp_listen() expects now.
Reviewed by: tuexen
Fixes: bd4a39cc93 ("socket: Properly interlock when transitioning to a listening socket")
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31879
Same as the v4 test, but with IPv6.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31789
Test prioritisation and dummynet queues.
We need to give the pipe sufficient bandwidth for dummynet to work.
Given that we can't rely on the TCP connection failing alltogether, but
we can measure the effect of dummynet by imposing a time limit on a
larger data transfer.
If TCP is prioritised it'll get most of the pipe bandwidth and easily
manage to transfer the data in 3 seconds or less. When not prioritised
this will not succeed.
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D31788
Introduce a link to the ipfw command, dnctl, for dummynet configuration.
dnctl only handles dummynet configuration, and is part of the effort to
support dummynet in pf.
/sbin/ipfw continues to accept pipe, queue and sched commands, but these can
now also be issued via the new dnctl command.
Reviewed by: donner
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D30465
The 5.15 kernel moved the backing_dev_info structure out of
the request queue structure which causes a build failure.
Rather than look in the new location for the BDI we instead
detect this upstream refactoring by the existance of either
the blk_queue_update_readahead() or disk_update_readahead()
functions. In either case, there's no longer any reason to
manually set the ra_pages value since it will be overridden
with a reasonable default (2x the block size) when
blk_queue_io_opt() is called.
Therefore, we update the compatibility wrapper to do nothing
for 5.9 and newer kernels. While it's tempting to do the
same for older kernels we want to keep the compatibility
code to preserve the existing behavior. Removing it would
effectively increase the default readahead to 128k.
Reviewed-by: Tony Nguyen <tony.nguyen@delphix.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes#12532
We want to keep our root file system open to preserve bcache segment
between file accesses, thus reducing physical disk IO.
Reviewed by: imp, allanjude, kevans (previous version)
Differential Revision: https://reviews.freebsd.org/D30848
MFC after: 1 month
Because lld 13 and higher default to garbage collecting start/stop
symbols when using --gc-sections, the linker sets used in the i386 boot
loaders will disappear. This leads to the loaders not recognizing any
commands, and failure to boot.
Until we have a good set of linker scripts for the loaders, work around
it by disabling the start-stop-gc feature.
MFC after: 1 week
When running in a virtualized environment, TLB invalidations can only
be performed on process scope, as only the hypervisor is allowed to
invalidate a global scope, or else a Program Interrupt is triggered.
Since we are here, also make sure that the register process table
hypercall returns success.
Reviewed by: jhibbits
MFC after: 2 weeks
Sponsored by: Instituto de Pesquisas Eldorado (eldorado.org.br)
Differential Revision: https://reviews.freebsd.org/D31775
This disables testing the ACPI timer by default, forcing the use of
ACPI-fast rather than ACPI-safe. The broken-ACPI-timers workaround
can be re-enabled by setting the hw.acpi.timer_test_enabled=1 tunable.
This speeds up the FreeBSD boot process by 140 ms on an EC2 c5.xlarge
instance.
This change will not be MFCed.
Assuming no problems are reported, acpi_timer_test, the associated
tunable, and the ACPI-safe timecounter should be removed in FreeBSD 15.
Relnotes: The ACPI-safe timer is disabled in favour of ACPI-fast;
if timekeeping issues are observed, please test with
hw.acpi.timer_test_enabled=1 in loader.conf and report
if that fixes the problem.
When hw.acpi.timer_test_enabled is set to 0, this makes acpi_timer_test
return 1 without actually testing the ACPI timer; this results in the
ACPI-fast timecounter always being used rather than potentially using
ACPI-safe.
The ACPI timer testing was introduced in 2002 as a workaround for
errata in Pentium II and Pentium III chipsets, and is unlikely to be
needed in 2021.
While I'm here, add TSENTER/TSEXIT to make it easier to see the time
spent on the test (if it is enabled).
Reviewed by: allanjude, imp
MFC After: 1 week
Some notable changes, from upstream's release notes:
- sshd(8): Remove support for obsolete "host/port" syntax.
- ssh(1): When prompting whether to record a new host key, accept the key
fingerprint as a synonym for "yes".
- ssh-keygen(1): when acting as a CA and signing certificates with an RSA
key, default to using the rsa-sha2-512 signature algorithm.
- ssh(1), sshd(8), ssh-keygen(1): this release removes the "ssh-rsa"
(RSA/SHA1) algorithm from those accepted for certificate signatures.
- ssh-sk-helper(8): this is a new binary. It is used by the FIDO/U2F
support to provide address-space isolation for token middleware
libraries (including the internal one).
- ssh(1): this release enables UpdateHostkeys by default subject to some
conservative preconditions.
- scp(1): this release changes the behaviour of remote to remote copies
(e.g. "scp host-a:/path host-b:") to transfer through the local host
by default.
- scp(1): experimental support for transfers using the SFTP protocol as
a replacement for the venerable SCP/RCP protocol that it has
traditionally used.
Additional integration work is needed to support FIDO/U2F in the base
system.
Deprecation Notice
------------------
OpenSSH will disable the ssh-rsa signature scheme by default in the
next release.
Reviewed by: imp
MFC after: 1 month
Relnotes: Yes
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D29985
Although it is not specified in the RFCs, the concept that
the NFSv4 server should reply to an RPC request within a
reasonable time is accepted practice within the NFSv4 community.
Without this patch, the NFSv4.2 server attempts to reply to
a Copy operation within 1second by limiting the copy to
vfs.nfs.maxcopyrange bytes (default 10Mbytes). This is crude at
best, given the large variation in I/O subsystem performance.
This patch adds a kernel only flag COPY_FILE_RANGE_TIMEO1SEC
that the NFSv4.2 can specify, which tells VOP_COPY_FILE_RANGE()
to return after approximately 1 second with a partial result and
implements this in vn_generic_copy_file_range(), used by
vop_stdcopyfilerange().
Modifying the NFSv4.2 server to set this flag will be done in
a separate patch. Also under consideration is exposing the
COPY_FILE_RANGE_TIMEO1SEC to userland for use on the FreeBSD
copy_file_range(2) syscall.
MFC after: 2 weeks
Reviewed by: khng
Differential Revision: https://reviews.freebsd.org/D31829
Current logic always selects an IFA of the same family from the
outgoing interfaces. In IPv4 over IPv6 setup there can be just
single non-127.0.0.1 ifa, attached to the loopback interface.
Create a separate rt_getifa_family() to handle entire ifa selection
for the IPv4 over IPv6.
Differential Revision: https://reviews.freebsd.org/D31868
MFC after: 1 week
This behaviour appears to date from the 4.4 BSD import. It has two
problems:
1. The update to so_state is not protected by the socket lock, so
concurrent updates to so_state may be lost.
2. Suppose two threads race to call connect(2) on a socket, and one
succeeds while the other fails. Then the failing thread may
incorrectly clear SS_ISCONNECTING, confusing the state machine.
Simply remove the update. It does not appear to be necessary:
pru_connect implementations which call soisconnecting() only do so after
all failure modes have been handled. For instance, tcp_connect() and
tcp6_connect() will never return an error after calling soisconnected().
However, we cannot correctly assert that SS_ISCONNECTED is not set after
an error from soconnect() since the socket lock is not held across the
pru_connect call, so a concurrent connect(2) may have set the flag.
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31699
... when applied to one-to-one-style sockets. sctp_listen() cannot be
used to toggle the listening state of such a socket. See RFC 6458's
description of expected listen(2) semantics for one-to-one- and
one-to-many-style sockets.
Reviewed by: tuexen
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31774
Now that SOCK_IO_*_LOCK() checks for listening sockets, we can eliminate
some racy SOLISTENING() checks. No functional change intended.
Reviewed by: tuexen
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31660
Currently, most protocols implement pru_listen with something like the
following:
SOCK_LOCK(so);
error = solisten_proto_check(so);
if (error) {
SOCK_UNLOCK(so);
return (error);
}
solisten_proto(so);
SOCK_UNLOCK(so);
solisten_proto_check() fails if the socket is connected or connecting.
However, the socket lock is not used during I/O, so this pattern is
racy.
The change modifies solisten_proto_check() to additionally acquire
socket buffer locks, and the calling thread holds them until
solisten_proto() or solisten_proto_abort() is called. Now that the
socket buffer locks are preserved across a listen(2), this change allows
socket I/O paths to properly interlock with listen(2).
This fixes a large number of syzbot reports, only one is listed below
and the rest will be dup'ed to it.
Reported by: syzbot+9fece8a63c0e27273821@syzkaller.appspotmail.com
Reviewed by: tuexen, gallatin
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31659
This is necessary to provide proper interlocking with listen(2), which
destroys the socket buffers. Otherwise, code must lock the socket
itself and check SOLISTENING(so), but most I/O paths do not otherwise
need to acquire the socket lock, so the extra overhead needed to check a
rare error case is undesirable.
listen(2) calls are relatively rare. Thus, the strategy is to have it
acquire all socket buffer locks when transitioning to a listening
socket. To do this safely, these locks must be stable, and not
destroyed during listen(2) as they are today. So, move them out of the
sockbuf and into the owning socket. For the sockbuf mutexes, keep a
pointer to the mutex in the sockbuf itself, for now. This can be
removed by replacing SOCKBUF_LOCK() etc. with macros which operate on
the socket itself, as was done for the sockbuf I/O locks.
Reviewed by: tuexen, gallatin
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31658
In preparation for moving sockbuf locks into the containing socket,
provide alternative macros for the sockbuf I/O locks:
SOCK_IO_SEND_(UN)LOCK() and SOCK_IO_RECV_(UN)LOCK(). These operate on a
socket rather than a socket buffer. Note that these locks are used only
to prevent concurrent readers and writters from interleaving I/O.
When locking for I/O, return an error if the socket is a listening
socket. Currently the check is racy since the sockbuf sx locks are
destroyed during the transition to a listening socket, but that will no
longer be true after some follow-up changes.
Modify a few places to check for errors from
sblock()/SOCK_IO_(SEND|RECV)_LOCK() where they were not before. In
particular, add checks to sendfile() and sorflush().
Reviewed by: tuexen, gallatin
MFC after: 1 month
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31657
This is in preparation for moving sockbuf locks into the owning socket,
in order to provide proper interlocking for listen(2). In particular,
listening sockets do not use the socket buffers and repurpose that space
in struct socket for their own purposes. Moving the locks out of the
socket buffers and into the socket proper makes it possible to safely
lock socket buffers and test for a listening socket before deciding how
to proceed.
Reordering these fields saves some space and helps ensure that UMA will
provide the same space efficiency for sockets as before. No functional
change intended.
Reviewed by: tuexen, gallatin
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31656
This interface is used solely by md(4) when the MD_RESERVE flag is
specified, as in `mdconfig -a -t swap -s 1G -o reserve`. It
pre-allocates swap blocks for the entire object.
The number of blocks to be reserved is specified as a vm_size_t, but
swp_pager_getswapspace() can allocate at most INT_MAX blocks. vm_size_t
also seems like the incorrect type to use here it refers only to the
size of the VM object, not the size of a mapping. So:
- change the type of "size" in swap_pager_reserve() to vm_pindex_t, and
- clamp the requested number of blocks for a single
swp_pager_getswapspace() call to INT_MAX.
Reported by: syzkaller
Reviewed by: dougm, alc, kib
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31875
The new wording for standard flags is losely based on the POSIX
description.
Make it clearer that PROT_MAX() is a local extension.
Reviewed by: alc, mckusick, imp, kib, markj
Sponsored by: DARPA
Differential Revision: https://reviews.freebsd.org/D31777
Linux fails to build now because the mkdtemp in the bootstrapped
environment wants 6 or more X's. Use 10 out of an abundance of caution.
Sponsored by: Netflix
Reviewed by: arichards
Differential Revision: https://reviews.freebsd.org/D31863
- Protect the `expire_upcalls` callout with the MFC6 mutex. The callout
handler needs this mutex anyway.
- Convert the MROUTER6 mutex to a sleepable sx lock. It is only used
when configuring the global v6 multicast routing socket, so is only
used in system call paths where sleeping is safe. This lets us drain
the callout without having to drop the lock.
- For all locking macros in the file, convert to using a _LOCKPTR macro.
Reported by: mav
MFC after: 2 weeks
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31836
Interface addresses with pending duplicate address detection (DAD) live
in a global queue. In this case, a callout is associated with each
entry. The callout transmits neighbour solicitations until the system
decides the address is no longer tentative, or until a duplicate address
is discovered. At this point the entry is dequeued and freed. DAD may
be manually stopped as well.
The callout currently runs (and potentially transmits packets) with
Giant held. Reorganize DAD queue locking to interlock properly with the
callout:
- Configure the callout to acquire the DAD queue lock before running.
The lock is dropped before transmitting any packets. Stop protecting
the callout with Giant.
- When looking up DAD queue entries for an incoming NS or NA, don't
bother fiddling with the DAD queue entry reference count.
- Split nd6_dad_starttimer() so that the caller is responsible to
transmitting a NS if it so desires.
- Remove the DAD entry from the queue before stopping the timer. Use a
temporary reference to make sure that the entry doesn't get freed by
the callout while we're draining.
Reported by: mav
Reviewed by: bz, hrs
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D31826
Make sure that the provider sector size is large enough to contain a
valid label before trying to read it. We performed this check already
for most label types, but not for several filesystem labels.
Reported by: syzbot+f52918174cdf193ae29c@syzkaller.appspotmail.com
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
- The SCTP_PCB_FLAGS_SND_ITERATOR_UP check was racy, since two threads
could observe that the flag is not set and then both set it. I'm not
sure if this is actually a problem in practice, i.e., maybe there's no
problem having multiple sends for a single PCB in the iterator list?
- sctp_sendall() was modifying sctp_flags without the inp lock held.
The change simply acquires the PCB write lock before toggling the flag,
fixing both problems.
Reviewed by: tuexen
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31813
This appears to be unused in usrsctp as well. No functional change
intended.
Reviewed by: tuexen
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31812
sctp_close() and sctp_abort() disassociate the PCB from its socket.
As a part of this, they attempt to free the PCB, which may end up
lingering. Fix some bugs in this area:
- For some reason, sctp_close() and sctp_abort() set
SCTP_PCB_FLAGS_SOCKET_GONE using an atomic compare-and-set without the
PCB lock held. This is racy since sctp_flags is normally updated
without atomics, using the PCB lock to synchronize. So, the update
can be lost, which can cause all sort of races with other SCTP
components which look for the _GONE flag. Fix the problem simply by
acquiring the PCB lock in order to set the flag. Note that we have to
drop and re-acquire the lock again in sctp_inpcb_free(), but I don't
see a good way around that for now. If it's a real problem, the _GONE
flag could be split out of sctp_flags and into a dedicated sctp_inpcb
field.
- In sctp_inpcb_free(), load sctp_socket after acquiring the PCB lock,
to avoid possible races with parallel sctp_inpcb_free() calls.
- Add an assertion sctp_inpcb_free() to verify that _ALLGONE is not set.
Reviewed by: tuexen
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D31811
Some installations may experience CTRL-EVENT-SCAN-FAILED when
associating to an AP. Installations that specify
ifconfig_wlan0="WPA ... up" in rc.conf do not experience
the problem whereas those which specify ifconfig_wlan0="WPA" without
the "up" will experience CTRL-EVENT-SCAN_FAILED.
However those that specify "up" in ifconfig_wlan0 will be able to
reproduce this problem by service netif stop wlan0;
service netif start wlan0. Interestingly The service netif stop/start
problem is reproducible on the older wpa 2.9 as well.
Reported by: dhw
Reported by: "Oleg V. Nauman" <oleg _ theweb_org_ua>
Reported by: Filipe da Silva Santos <contact _ shiori_com_br>
Reported by: Jakob Alvermark <jakob _ alvermark_net>
MFC after: 3 days
