Commit Graph

272 Commits

Author SHA1 Message Date
John Baldwin
688f8b822c Remove the cloned file descriptors for /dev/crypto.
Crypto file descriptors were added in the original OCF import as a way
to provide per-open data (specifically the list of symmetric
sessions).  However, this gives a bit of a confusing API where one has
to open /dev/crypto and then invoke an ioctl to obtain a second file
descriptor.  This also does not match the API used with /dev/crypto on
other BSDs or with Linux's /dev/crypto driver.

Character devices have gained support for per-open data via cdevpriv
since OCF was imported, so use cdevpriv to simplify the userland API
by permitting ioctls directly on /dev/crypto descriptors.

To provide backwards compatibility, CRIOGET now opens another
/dev/crypto descriptor via kern_openat() rather than dup'ing the
existing file descriptor.  This preserves prior semantics in case
CRIOGET is invoked multiple times on a single file descriptor.

Reviewed by:	markj
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27302
2020-11-25 00:10:54 +00:00
John Baldwin
3acf4d2374 Use void * in place of caddr_t.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27065
2020-11-06 18:09:52 +00:00
John Baldwin
c423784dc5 Group session management routines together before first use.
- Rename cse*() to cse_*() to more closely match other local APIs in
  this file.

- Merge the old csecreate() into cryptodev_create_session() and rename
  the new function to cse_create().

Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27070
2020-11-06 18:05:29 +00:00
John Baldwin
f5074add75 Move cryptof_ioctl() below the routines it calls.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27069
2020-11-06 00:15:52 +00:00
John Baldwin
b19d4c075f Split logic to create new sessions into a separate function.
This simplifies cryptof_ioctl as it now a wrapper around functions that
contain the bulk of the per-ioctl logic.

Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27068
2020-11-06 00:10:58 +00:00
John Baldwin
c54004c6a9 Move cryptodev_cb earlier before it is used.
This is consistent with cryptodevkey_cb being defined before it is used
and removes a prototype in the middle of the file.

Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27067
2020-11-05 23:42:36 +00:00
John Baldwin
5973f4922d Style fixes for function prototypes and definitions.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27066
2020-11-05 23:28:05 +00:00
John Baldwin
84fea065db Don't modify the destination pointer in ioctl requests.
This breaks the case where the original pointer was NULL but an
in-line IV was used.

Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27064
2020-11-05 23:26:02 +00:00
John Baldwin
9038e6a1e4 Replace some K&R function definitions with ANSI C.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27062
2020-11-03 22:32:30 +00:00
John Baldwin
d3d79e968b Consistently use C99 fixed-width types in the in-kernel crypto code.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27061
2020-11-03 22:27:54 +00:00
Mark Johnston
d588dc7d84 opencrypto: Annotate hmac_init_(i|o)pad() to make auth_hash const
This makes them friendlier to drivers that try to use const pointers
whenever possible in their internal structures.

Reviewed by:	jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D26901
2020-10-30 17:05:36 +00:00
John Baldwin
e7f6b6cf69 Fix a couple of bugs for asym crypto introduced in r359374.
- Check for null pointers in the crypto_drivers[] array when checking
  for empty slots in crypto_select_kdriver().

- Handle the case where crypto_kdone() is invoked on a request where
  krq_cap is NULL due to not finding a matching driver.

Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D26811
2020-10-19 20:04:03 +00:00
John Baldwin
ecedef531b Mark asymmetric cryptography via OCF deprecated for 14.0.
Only one MIPS-specific driver implements support for one of the
asymmetric operations.  There are no in-kernel users besides
/dev/crypto.  The only known user of the /dev/crypto interface was the
engine in OpenSSL releases before 1.1.0.  1.1.0 includes a rewritten
engine that does not use the asymmetric operations due to lack of
documentation.

Reviewed by:	cem, markj
MFC after:	1 week
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D26810
2020-10-19 18:21:41 +00:00
Marcin Wojtas
6038018ab1 Add support for ESN in cryptosoft
This patch adds support for IPsec ESN (Extended Sequence Numbers) in
encrypt and authenticate mode (eg. AES-CBC and SHA256) and combined mode
(eg. AES-GCM).

For encrypt and authenticate mode the ESN is stored in separate crp_esn
buffer because the high-order 32 bits of the sequence number are
appended after the Next Header (RFC 4303).

For combined modes the high-order 32 bits of the sequence number [e.g.
RFC 4106, Chapter 5 AAD Construction] are part of crp_aad (prepared by
netipsec layer in case of ESN support enabled), therefore non visible
diff around combined modes.

Submitted by:           Grzegorz Jaszczyk <jaz@semihalf.com>
                        Patryk Duda <pdk@semihalf.com>
Reviewed by:            jhb
Differential revision:  https://reviews.freebsd.org/D22364
Obtained from:          Semihalf
Sponsored by:           Stormshield
2020-10-16 11:18:13 +00:00
Marcin Wojtas
7e89ae49db Prepare crypto framework for IPsec ESN support
This permits requests (netipsec ESP and AH protocol) to provide the
IPsec ESN (Extended Sequence Numbers) in a separate buffer.

As with separate output buffer and separate AAD buffer not all drivers
support this feature. Consumer must request use of this feature via new
session flag.

Submitted by:           Grzegorz Jaszczyk <jaz@semihalf.com>
                        Patryk Duda <pdk@semihalf.com>
Reviewed by:            jhb
Differential revision:  https://reviews.freebsd.org/D24838
Obtained from:          Semihalf
Sponsored by:           Stormshield
2020-10-16 11:06:33 +00:00
John Baldwin
47e2650ea4 Add support to the KTLS OCF module for AES-CBC MTE ciphersuites.
This is a simplistic approach which encrypts each TLS record in two
separate passes: one to generate the MAC and a second to encrypt.
This supports TLS 1.0 connections with implicit IVs as well as TLS
1.1+ with explicit IVs.

Reviewed by:	gallatin
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26730
2020-10-13 18:04:19 +00:00
John Baldwin
e0b155fe4a Simplify swcr_authcompute() after removal of deprecated algorithms.
- Just use sw->octx != NULL to handle the HMAC case when finalizing
  the MAC.

- Explicitly zero the on-stack auth context.

Reviewed by:	markj
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26688
2020-10-06 18:07:52 +00:00
Warner Losh
2215adc5ff Include sys/types.h here
It's included by header pollution in most of the compile
environments. However, in the standalone envirnment, it's not
included. Go ahead and include it always since the overhead is low and
it is simpler that way.

MFC After: 3 days
2020-09-15 15:21:29 +00:00
John Baldwin
62cddd0e03 Name the on-stack union of compat thunks.
C does not permit an anonymous union at a top-level scope.

Pointy hat to:	jhb
2020-08-26 22:36:08 +00:00
John Baldwin
113bcc82a2 Add freebsd32 compat support for CIOCCRYPTAEAD.
Reviewed by:	markj (earlier version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26179
2020-08-26 21:28:47 +00:00
John Baldwin
5612fcb17d Simplify compat shims for /dev/crypto.
- Make session handling always use the CIOGSESSION2 structure.
  CIOGSESSION requests use a thunk similar to COMPAT_FREEBSD32 session
  requests.  This permits the ioctl handler to use the 'crid' field
  unconditionally.

- Move COMPAT_FREEBSD32 handling out of the main ioctl handler body
  and instead do conversions in/out of thunk structures in dedicated
  blocks at the start and end of the ioctl function.

Reviewed by:	markj (earlier version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D26178
2020-08-26 21:17:18 +00:00
Alan Somers
e6f6d0c9bc crypto(9): add CRYPTO_BUF_VMPAGE
crypto(9) functions can now be used on buffers composed of an array of
vm_page_t structures, such as those stored in an unmapped struct bio.  It
requires the running to kernel to support the direct memory map, so not all
architectures can use it.

Reviewed by:	markj, kib, jhb, mjg, mat, bcr (manpages)
MFC after:	1 week
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D25671
2020-08-26 02:37:42 +00:00
John Baldwin
3c0e568505 Add support for KTLS RX via software decryption.
Allow TLS records to be decrypted in the kernel after being received
by a NIC.  At a high level this is somewhat similar to software KTLS
for the transmit path except in reverse.  Protocols enqueue mbufs
containing encrypted TLS records (or portions of records) into the
tail of a socket buffer and the KTLS layer decrypts those records
before returning them to userland applications.  However, there is an
important difference:

- In the transmit case, the socket buffer is always a single "record"
  holding a chain of mbufs.  Not-yet-encrypted mbufs are marked not
  ready (M_NOTREADY) and released to protocols for transmit by marking
  mbufs ready once their data is encrypted.

- In the receive case, incoming (encrypted) data appended to the
  socket buffer is still a single stream of data from the protocol,
  but decrypted TLS records are stored as separate records in the
  socket buffer and read individually via recvmsg().

Initially I tried to make this work by marking incoming mbufs as
M_NOTREADY, but there didn't seemed to be a non-gross way to deal with
picking a portion of the mbuf chain and turning it into a new record
in the socket buffer after decrypting the TLS record it contained
(along with prepending a control message).  Also, such mbufs would
also need to be "pinned" in some way while they are being decrypted
such that a concurrent sbcut() wouldn't free them out from under the
thread performing decryption.

As such, I settled on the following solution:

- Socket buffers now contain an additional chain of mbufs (sb_mtls,
  sb_mtlstail, and sb_tlscc) containing encrypted mbufs appended by
  the protocol layer.  These mbufs are still marked M_NOTREADY, but
  soreceive*() generally don't know about them (except that they will
  block waiting for data to be decrypted for a blocking read).

- Each time a new mbuf is appended to this TLS mbuf chain, the socket
  buffer peeks at the TLS record header at the head of the chain to
  determine the encrypted record's length.  If enough data is queued
  for the TLS record, the socket is placed on a per-CPU TLS workqueue
  (reusing the existing KTLS workqueues and worker threads).

- The worker thread loops over the TLS mbuf chain decrypting records
  until it runs out of data.  Each record is detached from the TLS
  mbuf chain while it is being decrypted to keep the mbufs "pinned".
  However, a new sb_dtlscc field tracks the character count of the
  detached record and sbcut()/sbdrop() is updated to account for the
  detached record.  After the record is decrypted, the worker thread
  first checks to see if sbcut() dropped the record.  If so, it is
  freed (can happen when a socket is closed with pending data).
  Otherwise, the header and trailer are stripped from the original
  mbufs, a control message is created holding the decrypted TLS
  header, and the decrypted TLS record is appended to the "normal"
  socket buffer chain.

(Side note: the SBCHECK() infrastucture was very useful as I was
 able to add assertions there about the TLS chain that caught several
 bugs during development.)

Tested by:	rmacklem (various versions)
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24628
2020-07-23 23:48:18 +00:00
John Baldwin
70d1a4351a Consolidate duplicated code into a ktls_ocf_dispatch function.
This function manages the loop around crypto_dispatch and coordination
with ktls_ocf_callback.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D25757
2020-07-23 21:43:06 +00:00
John Baldwin
33a1a488d5 Don't dynamically allocate data structures for KTLS crypto requests.
Allocate iovec arrays and struct cryptop and struct ocf_operation
objects on the stack to reduce avoid the overhead of malloc().

These structures are all small enough to fit on the stack of the KTLS
worker threads.

Reviewed by:	gallatin
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D25692
2020-07-20 22:32:39 +00:00
Mark Johnston
e1a82b35bf crypto(9): Stop checking for failures from malloc(M_WAITOK).
PR:		240545
Submitted by:	Andrew Reiter <arr@watson.org>
Reviewed by:	cem, delphij, jhb
MFC after:	1 week
Event:		July 2020 Bugathon
2020-07-20 17:44:13 +00:00
Mark Johnston
e5587cbbc2 Clean up crypto_init().
The function is called from a KLD load handler, so it may sleep.

- Stop checking for errors from uma_zcreate(), they don't happen.
- Convert M_NOWAIT allocations to M_WAITOK.
- Remove error handling for existing M_WAITOK allocations.
- Fix style.

Reviewed by:	cem, delphij, jhb
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25696
2020-07-17 14:45:16 +00:00
John Baldwin
946b8f6fb0 Add crypto_initreq() and crypto_destroyreq().
These routines are similar to crypto_getreq() and crypto_freereq() but
operate on caller-supplied storage instead of allocating crypto
requests from a UMA zone.

Reviewed by:	markj
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D25691
2020-07-16 21:30:46 +00:00
Mark Johnston
7290cb47fc Convert cryptostats to a counter_u64 array.
The global counters were not SMP-friendly.  Use per-CPU counters
instead.

Reviewed by:	jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D25466
2020-06-30 22:01:21 +00:00
Mark Johnston
a5ae70f5a0 Remove unused 32-bit compatibility structures from cryptodev.
The counters are exported by a sysctl and have the same width on all
platforms anyway.

Reviewed by:	cem, delphij, jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D25465
2020-06-30 15:57:11 +00:00
Mark Johnston
a5c053f5a7 Remove CRYPTO_TIMING.
It was added a very long time ago.  It is single-threaded, so only
really useful for basic measurements, and in the meantime we've gotten
some more sophisticated profiling tools.

Reviewed by:	cem, delphij, jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D25464
2020-06-30 15:56:54 +00:00
John Baldwin
17a831ea25 Zero the temporary HMAC key in hmac_init_pad().
Reviewed by:	delphij
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25436
2020-06-25 20:18:55 +00:00
John Baldwin
4a711b8d04 Use zfree() instead of explicit_bzero() and free().
In addition to reducing lines of code, this also ensures that the full
allocation is always zeroed avoiding possible bugs with incorrect
lengths passed to explicit_bzero().

Suggested by:	cem
Reviewed by:	cem, delphij
Approved by:	csprng (cem)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25435
2020-06-25 20:17:34 +00:00
John Baldwin
5b750b9a68 Store the AAD in a separate buffer for KTLS.
For TLS 1.2 this permits reusing one of the existing iovecs without
always having to duplicate both.

While here, only duplicate the output iovec for TLS 1.3 if it will be
used.

Reviewed by:	gallatin
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25291
2020-06-23 00:02:28 +00:00
John Baldwin
9b774dc0c5 Add support to the crypto framework for separate AAD buffers.
This permits requests to provide the AAD in a separate side buffer
instead of as a region in the crypto request input buffer.  This is
useful when the main data buffer might not contain the full AAD
(e.g. for TLS or IPsec with ESN).

Unlike separate IVs which are constrained in size and stored in an
array in struct cryptop, separate AAD is provided by the caller
setting a new crp_aad pointer to the buffer.  The caller must ensure
the pointer remains valid and the buffer contents static until the
request is completed (e.g. when the callback routine is invoked).

As with separate output buffers, not all drivers support this feature.
Consumers must request use of this feature via a new session flag.

To aid in driver testing, kern.crypto.cryptodev_separate_aad can be
set to force /dev/crypto requests to use a separate AAD buffer.

Discussed with:	cem
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25288
2020-06-22 23:20:43 +00:00
John Baldwin
26d292d3e2 Various optimizations to software AES-CCM and AES-GCM.
- Make use of cursors to avoid data copies for AES-CCM and AES-GCM.

  Pass pointers into the request's input and/or output buffers
  directly to the Update, encrypt, and decrypt hooks rather than
  always copying all data into a temporary block buffer on the stack.

- Move handling for partial final blocks out of the main loop.

  This removes branches from the main loop and permits using
  encrypt/decrypt_last which avoids a memset to clear the rest of the
  block on the stack.

- Shrink the on-stack buffers to assume AES block sizes and CCM/GCM
  tag lengths.

- For AAD data, pass larger chunks to axf->Update.  CCM can take each
  AAD segment in a single call.  GMAC can take multiple blocks at a
  time.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D25058
2020-06-12 23:10:30 +00:00
John Baldwin
4e6a381306 Fix a regression in r361804 for TLS 1.3.
I was not including the record type stored in the first byte of the
trailer as part of the payload to be encrypted and hashed.

Sponsored by:	Netflix
2020-06-12 22:27:26 +00:00
John Baldwin
9b6b2f8608 Adjust crypto_apply function callbacks for OCF.
- crypto_apply() is only used for reading a buffer to compute a
  digest, so change the data pointer to a const pointer.

- To better match m_apply(), change the data pointer type to void *
  and the length from uint16_t to u_int.  The length field in
  particular matters as none of the apply logic was splitting requests
  larger than UINT16_MAX.

- Adjust the auth_xform Update callback to match the function
  prototype passed to crypto_apply() and crypto_apply_buf().  This
  removes the needs for casts when using the Update callback.

- Change the Reinit and Setkey callbacks to also use a u_int length
  instead of uint16_t.

- Update auth transforms for the changes.  While here, use C99
  initializers for auth_hash structures and avoid casts on callbacks.

Reviewed by:	cem
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25171
2020-06-10 21:18:19 +00:00
John Baldwin
a3d565a118 Add a crypto capability flag for accelerated software drivers.
Use this in GELI to print out a different message when accelerated
software such as AESNI is used vs plain software crypto.

While here, simplify the logic in GELI a bit for determing which type
of crypto driver was chosen the first time by examining the
capabilities of the matched driver after a single call to
crypto_newsession rather than making separate calls with different
flags.

Reviewed by:	delphij
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25126
2020-06-09 22:26:07 +00:00
John Baldwin
080933c050 Use separate output buffers for OCF requests in KTLS.
KTLS encryption requests for file-backed data such as from sendfile(2)
require the encrypted data to be stored in a separate buffer from the
unencrypted file input data.  Previously the OCF backend for KTLS
manually copied the data from the input buffer to the output buffer
before queueing the crypto request.  Now the OCF backend will use a
separate output buffer for such requests and avoid the copy.  This
mostly helps when an async co-processor is used by saving CPU cycles
used on the copy.

Reviewed by:	gallatin (earlier version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24545
2020-06-04 20:12:34 +00:00
John Baldwin
20c128da91 Add explicit bzero's of sensitive data in software crypto consumers.
Explicitly zero IVs, block buffers, and hashes/digests.

Reviewed by:	delphij
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D25057
2020-06-03 22:11:05 +00:00
John Baldwin
4542cd9379 Increment the correct pointer when a crypto buffer spans an mbuf or iovec.
When a crypto_cursor_copyback() request spanned multiple mbufs or
iovecs, the pointer into the mbuf/iovec was incremented instead of the
pointer into the source buffer being copied from.

PR:		246737
Reported by:	Jenkins, ZFS test suite
Sponsored by:	Netflix
2020-05-29 05:41:21 +00:00
John Baldwin
ba63e5e701 Add a sysctl knob to use separate output buffers for /dev/crypto.
This is a testing aid to permit using testing a driver's support of
separate output buffers via cryptocheck.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24545
2020-05-25 22:21:09 +00:00
John Baldwin
33f3bad35e Export the _kern_crypto sysctl node from crypto.c.
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24545
2020-05-25 22:18:33 +00:00
John Baldwin
9c0e3d3a53 Add support for optional separate output buffers to in-kernel crypto.
Some crypto consumers such as GELI and KTLS for file-backed sendfile
need to store their output in a separate buffer from the input.
Currently these consumers copy the contents of the input buffer into
the output buffer and queue an in-place crypto operation on the output
buffer.  Using a separate output buffer avoids this copy.

- Create a new 'struct crypto_buffer' describing a crypto buffer
  containing a type and type-specific fields.  crp_ilen is gone,
  instead buffers that use a flat kernel buffer have a cb_buf_len
  field for their length.  The length of other buffer types is
  inferred from the backing store (e.g. uio_resid for a uio).
  Requests now have two such structures: crp_buf for the input buffer,
  and crp_obuf for the output buffer.

- Consumers now use helper functions (crypto_use_*,
  e.g. crypto_use_mbuf()) to configure the input buffer.  If an output
  buffer is not configured, the request still modifies the input
  buffer in-place.  A consumer uses a second set of helper functions
  (crypto_use_output_*) to configure an output buffer.

- Consumers must request support for separate output buffers when
  creating a crypto session via the CSP_F_SEPARATE_OUTPUT flag and are
  only permitted to queue a request with a separate output buffer on
  sessions with this flag set.  Existing drivers already reject
  sessions with unknown flags, so this permits drivers to be modified
  to support this extension without requiring all drivers to change.

- Several data-related functions now have matching versions that
  operate on an explicit buffer (e.g. crypto_apply_buf,
  crypto_contiguous_subsegment_buf, bus_dma_load_crp_buf).

- Most of the existing data-related functions operate on the input
  buffer.  However crypto_copyback always writes to the output buffer
  if a request uses a separate output buffer.

- For the regions in input/output buffers, the following conventions
  are followed:
  - AAD and IV are always present in input only and their
    fields are offsets into the input buffer.
  - payload is always present in both buffers.  If a request uses a
    separate output buffer, it must set a new crp_payload_start_output
    field to the offset of the payload in the output buffer.
  - digest is in the input buffer for verify operations, and in the
    output buffer for compute operations.  crp_digest_start is relative
    to the appropriate buffer.

- Add a crypto buffer cursor abstraction.  This is a more general form
  of some bits in the cryptosoft driver that tried to always use uio's.
  However, compared to the original code, this avoids rewalking the uio
  iovec array for requests with multiple vectors.  It also avoids
  allocate an iovec array for mbufs and populating it by instead walking
  the mbuf chain directly.

- Update the cryptosoft(4) driver to support separate output buffers
  making use of the cursor abstraction.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24545
2020-05-25 22:12:04 +00:00
John Baldwin
d65771e472 Correct the minimum key length for Camellia to 16 bytes (128 bits).
MFC after:	1 week
2020-05-22 17:21:22 +00:00
John Baldwin
723d87648e Improve support for stream ciphers in the software encryption interface.
Add a 'native_blocksize' member to 'struct enc_xform' that ciphers can
use if they support a partial final block.  This is particular useful
for stream ciphers, but can also apply to other ciphers.  cryptosoft
will only pass in native blocks to the encrypt and decrypt hooks.  For
the final partial block, 'struct enc_xform' now has new
encrypt_last/decrypt_last hooks which accept the length of the final
block.  The multi_block methods are also retired.

Mark AES-ICM (AES-CTR) as a stream cipher.  This has some interesting
effects on IPsec in that FreeBSD can now properly receive all packets
sent by Linux when using AES-CTR, but FreeBSD can no longer
interoperate with OpenBSD and older verisons of FreeBSD which assume
AES-CTR packets have a payload padded to a 16-byte boundary.  Kornel
has offered to work on a patch to add a compatiblity sysctl to enforce
additional padding for AES-CTR in esp_output to permit compatibility
with OpenBSD and older versions of FreeBSD.

AES-XTS continues to use a block size of a single AES block length.
It is possible to adjust it to support partial final blocks by
implementing cipher text stealing via encrypt_last/decrypt_last hooks,
but I have not done so.

Reviewed by:	cem (earlier version)
Tested by:	Kornel Dulęba <mindal@semihalf.com> (AES-CTR with IPsec)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24906
2020-05-22 16:29:09 +00:00
John Baldwin
4f98ffdd1d Fix libstand build breakage after r361298.
- Use enc_xform_aes_xts.setkey() directly instead of duplicating the code
  now that it no longer calls malloc().
- Rather than bringing back all of xform_userland.h, add a conditional
  #include of <stand.h> to xform_enc.h.
- Update calls to encrypt/decrypt callbacks in enc_xform_aes_xts for
  separate input/output pointers.

Pointy hat to:	jhb
2020-05-20 22:25:41 +00:00
John Baldwin
3e9470482a Various cleanups to the software encryption transform interface.
- Consistently use 'void *' for key schedules / key contexts instead
  of a mix of 'caddr_t', 'uint8_t *', and 'void *'.

- Add a ctxsize member to enc_xform similar to what auth transforms use
  and require callers to malloc/zfree the context.  The setkey callback
  now supplies the caller-allocated context pointer and the zerokey
  callback is removed.  Callers now always use zfree() to ensure
  key contexts are zeroed.

- Consistently use C99 initializers for all statically-initialized
  instances of 'struct enc_xform'.

- Change the encrypt and decrypt functions to accept separate in and
  out buffer pointers.  Almost all of the backend crypto functions
  already supported separate input and output buffers and this makes
  it simpler to support separate buffers in OCF.

- Remove xform_userland.h shim to permit transforms to be compiled in
  userland.  Transforms no longer call malloc/free directly.

Reviewed by:	cem (earlier version)
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24855
2020-05-20 21:21:01 +00:00
John Baldwin
f272bc03cc Trim a few more things I missed from xform_enc.h.
An extern declaration for the now-removed Blowfish encryption
transform, and an include of the DES header.
2020-05-13 18:36:02 +00:00