Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode when a
receive key is added to the socket.
The firmware is only able to switch a "plain" TOE connection to TLS
mode if the head of the pending socket data is the start of a TLS
record, so the connection is migrated to TLS mode as a multi-step
process.
When TOE TLS RX is enabled, the associated connection's receive side
is frozen via a flag in the TCB. The state of the socket buffer is
then examined to determine if the pending data in the socket buffer
ends on a TLS record boundary. If so, the connection is migrated to
TLS mode and unfrozen. Otherwise, the connection is unfrozen
temporarily until more data arrives. Once more data arrives, the
receive queue is frozen again and rechecked. This continues until the
connection is paused at a record boundary. Any records received
before TLS mode is enabled are decrypted as software records.
Note that this removes the 'rx_tls_ports' sysctl. TOE TLS offload for
receive is now enabled automatically on existing TOE connections when
using a KTLS-aware SSL library just as it was previously enabled
automatically for TLS transmit. This also enables TLS offload for TOE
connections which enable TLS after passing initial data in the clear
(e.g. STARTTLS with SMTP).
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37351
If a TLS record is too small to contain the required explicit IV,
record_type (TLS 1.3), and MAC, reject attempts to decrypt it with
EMSGSIZE without submitting it to OCF. OCF drivers may not properly
detect that regions in the crypto request are outside the bounds of
the mbuf chain. The caller isn't supposed to submit such requests.
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37372
This is mainly intended to provide a fallback for TOE TLS which may
need to use software decryption for an initial record at the start
of a connection.
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37370
These were originally in locore.S as they are only needed so we have
a valid value to put into the vbar_el2 register. As these will soon
be used by bhyve so move them to a new file as we already have with
the EL1 exception vectors in exception.S.
Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Zero the vttbr_el2 register on each CPU so we can tell if we are
running the host or guest kernel from a hypervisor.
Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
For completeness add accessors for the MIDR field. As the field is
always 0xf on arm64 it is unneeded in the current MICR handling, but
will be used in the vmm module for bhyve.
Obtained from: https://github.com/FreeBSD-UPB/freebsd-src (earlier version)
Sponsored by: The FreeBSD Foundation
The option is not even recognized and with that patched it does not
compile. Even if it did work, it would be prohibitively expensive to
use.
Interested parties can use pmcstat or dtrace instead.
Add a memset_startat() macro which sets a pattern from a struct member
to the end of the struct. Needed by a wireless driver.
MFC after: 3 days
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D37389
These all work on stage 1 tables. Rename them so we can add similar
functions that operate on stage 2 tables.
Reviewed by: alc, markj, kib
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37302
When modifying a stage 2 mapping we may need to call into the
hypervisor to invalidate the TLB. Until it is known if the cost of
this operation is less than the performance gains superpages offers
disable their use.
Reviewed by: kib. markj
Sponsored by: Innovate UK
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D37299
The recent refactoring to prepare for pfsync over IPv6 introduced a
memory leak.
If we don't have a sync peer configured we return early (without sending
out a packet), but failed to free the newly allocated packet.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Rather than passing control packets through the ioctl interface allow
them to pass through the normal UDP socket flow.
This simplifies both kernel and userspace, and matches the approach
taken (or the one that will be taken) on the Linux side of things.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37317
Currently bhyve's NVMe controller cannot save feature values cross
reboot. It should return a FEATURE_NOT_SAVEABLE error when the command
specifies a save flag.
Quote from NVMe specification, page 205:
https://nvmexpress.org/wp-content/uploads/NVM-Express-1_4-2019.06.10-Ratified.pdf
If the Feature Identifier specified in the Set Features command is not
saveable by the controller and the controller receives a Set Features
command with the Save bit set to one, then the command shall be aborted
with a status of Feature Identifier Not Saveable.
Reviewed by: chuck (older version)
Approved by: manu (mentor)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D32767
A misaligned frame pointer is certainly not a valid frame pointer and
with strict alignment enabled (as on CHERI) can cause panics when it is
loaded from later in the code.
Reviewed By: jhb
Differential Revision: https://reviews.freebsd.org/D34646
We reference count to ensure we don't release the socket while we still
have data in flight. That means that we can end up releasing the socket
from ovpn_encrypt_tx_cb().
We must have a vnet context set when calling sorele() (which asserts
this from within sofree()), so move the CURVNET_SET()/CURVNET_RESTORE()
to ensure this is the case.
While here also add a couple of assertions to make this more obvious,
and to ease future debugging.
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D37326
All the needed infrastructure updates have been made to allow
snapshots to be taken on UFS/FFS filesystems that are using journaled
soft updates. The most immediate benefit is the ability to use a
snapshot to take a consistent filesystem dump on a live filesystem
using the -L option to dump(8).
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36491
ktls_cleanup() does not free ktls session objects, it merely
cleans (and frees) members of the object.
Change callers to use ktls_free() instead.
Reviewed by: gallatin, markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D37352
On Linux, the `kmalloc()` family of functions returns a special value if
the size of the allocation is zero. This macro verifies if the pointer
is NULL (the allocation failed) or the size is 0 (the allocation was not
performed AFAIU). This special value can be passed to `kfree()`.
On FreeBSD, our `malloc(9)` functions don't return a special value for
0-size allocations. Therefore we can simply compare the result against
NULL.
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D37367
They are not really used in this header. However they are included in
Linux and at least the DRM drivers unfortunately rely on this namespace
pollution.
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D37365
This is used by `i915_gem.c` in the i915 DRM driver to get access to
`wbinvd_on_all_cpus()`.
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D36970
The M1 has no EL3, so we're limited to a spin-table implementation if we
want to eventually use bhyve on it. Implement spin-table now, but note
that we still prefer PSCI where possible.
Reviewed by: mmel
Differential Revision: https://reviews.freebsd.org/D34661
This command already prints a tremendous amount of output, and properly
obeys the pager. It no longer makes sense to arbitrarily limit the pages
that are printed, as the reader will not be aware that this has
happened.
Reviewed by: markj
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D37361
I took the implementation from OpenBSD, commit
d55ef580b1748517027c3eabdb715316ca5b1442.
The only difference is the addition of `dma_buf_map_is_equal()`.
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D36963
It simply includes the same header in FreeBSD (which is located
elsewhere).
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D36960
In FreeBSD, this is a wrapper on top of `realloc()`.
V2: Check if `n * size` would overflow and return `NULL` if that's the
case. Suggested by hselasky@ and emaste@.
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D36959
In Linux, this affects how the value is formatted. In FreeBSD, this
remains an unsigned integer.
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D36958
The previous `llnode` field is moved inside another field `node`.
This `node` field is a `struct __call_single_node` in Linux. Here, we
simply add an anonymous struct with the `llnode` field inside. That
field's new name is `llist` now.
V2: Use an anonymous union to keep the structure backward compatible
with drivers using the previous `llnode` field. This was suggested
by wufl@ and hselasky@. Thank you!
Reviewed by: manu
Approved by: manu
Differential Revision: https://reviews.freebsd.org/D36955
arm64 is the only currently supported architecture that has
${MACHINE_CPUARCH} set to a different value (aarch64) than ${MACHINE}
(arm64), as described in arch(7). However, there is no source directory
associated with arm64 that has a name set to ${MACHINE_CPUARCH}.
Remove the dead code that adds a directory with a name set to
${MACHINE_CPUARCH} to a list of directories indexed with cscope.
This change allows to use the cscope target on arm64.
Discussed with: jhb
Reviewed by: imp jhb
Approved by: oshogbo (mentor)
Differential Revision: https://reviews.freebsd.org/D36402
