Commit Graph

87 Commits

Author SHA1 Message Date
John Baldwin
8f35841f1f crypto: Add support for the XChaCha20-Poly1305 AEAD cipher.
This cipher is a wrapper around the ChaCha20-Poly1305 AEAD cipher
which accepts a larger nonce.  Part of the nonce is used along with
the key as an input to HChaCha20 to generate a derived key used for
ChaCha20-Poly1305.

This cipher is used by WireGuard.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33523
2022-01-11 14:16:41 -08:00
John Baldwin
74d3f1b63d OCF: Add crypto_clonereq().
This function clones an existing crypto request, but associates the
new request with a specified session.  The intended use case is for
drivers to be able to fall back to software by cloning a request and
dispatch it to an internally allocated software session.

Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D33607
2022-01-04 14:22:12 -08:00
John Baldwin
d074adf18b cryptodev.h: Drop 'extern' from function prototypes.
Sponsored by:	Chelsio Communications
2021-12-21 12:33:49 -08:00
John Baldwin
47fc049585 crypto: Define POLY1305_BLOCK_LEN constant.
Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D33485
2021-12-16 13:47:16 -08:00
John Baldwin
ae18720d27 crypto: Support multiple nonce lengths for AES-CCM.
Permit nonces of lengths 7 through 13 in the OCF framework and the
cryptosoft driver.  A helper function (ccm_max_payload_length) can be
used in OCF drivers to reject CCM requests which are too large for the
specified nonce length.

Reviewed by:	sef
Sponsored by:	Chelsio Communications, The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D32111
2021-10-06 14:08:47 -07:00
John Baldwin
16676123fc cryptodev: Permit explicit IV/nonce and MAC/tag lengths.
Add 'ivlen' and 'maclen' fields to the structure used for CIOGSESSION2
to specify the explicit IV/nonce and MAC/tag lengths for crypto
sessions.  If these fields are zero, the default lengths are used.

This permits selecting an alternate nonce length for AEAD ciphers such
as AES-CCM which support multiple nonce leengths.  It also supports
truncated MACs as input to AEAD or ETA requests.

Reviewed by:	markj
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D32107
2021-10-06 14:08:46 -07:00
Mark Johnston
d8787d4f78 crypto: Constify all transform descriptors
No functional change intended.

Reviewed by:	ae, jhb
MFC after:	1 week
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D31196
2021-07-26 16:41:05 -04:00
John Baldwin
8fa5c577de crypto: Remove now-unused crypto_cursor_seg{base,len}.
Callers should use crypto_cursor_segment() instead.

Reviewed by:	markj
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D30448
2021-06-16 15:23:16 -07:00
John Baldwin
beb817edfe crypto: Add crypto_cursor_segment() to fetch both base and length.
This function combines crypto_cursor_segbase() and
crypto_cursor_seglen() into a single function.  This is mostly
beneficial in the unmapped mbuf case where back to back calls of these
two functions have to iterate over the sub-components of unmapped
mbufs twice.

Bump __FreeBSD_version for crypto drivers in ports.

Suggested by:	markj
Reviewed by:	markj
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D30445
2021-05-25 16:59:19 -07:00
John Baldwin
883a0196b6 crypto: Add a new type of crypto buffer for a single mbuf.
This is intended for use in KTLS transmit where each TLS record is
described by a single mbuf that is itself queued in the socket buffer.
Using the existing CRYPTO_BUF_MBUF would result in
bus_dmamap_load_crp() walking additional mbufs in the socket buffer
that are not relevant, but generating a S/G list that potentially
exceeds the limit of the tag (while also wasting CPU cycles).

Reviewed by:	markj
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D30136
2021-05-25 16:59:18 -07:00
John Baldwin
76681661be OCF: Remove support for asymmetric cryptographic operations.
There haven't been any non-obscure drivers that supported this
functionality and it has been impossible to test to ensure that it
still works.  The only known consumer of this interface was the engine
in OpenSSL < 1.1.  Modern OpenSSL versions do not include support for
this interface as it was not well-documented.

Reviewed by:	cem
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D29736
2021-04-12 14:28:43 -07:00
John Baldwin
fc8fc743d8 Add an OCF algorithm for ChaCha20-Poly1305 AEAD.
Note that this algorithm implements the mode defined in RFC 8439.

Reviewed by:	cem
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D27836
2021-02-18 09:26:00 -08:00
Mark Johnston
68f6800ce0 opencrypto: Introduce crypto_dispatch_async()
Currently, OpenCrypto consumers can request asynchronous dispatch by
setting a flag in the cryptop.  (Currently only IPSec may do this.)   I
think this is a bit confusing: we (conditionally) set cryptop flags to
request async dispatch, and then crypto_dispatch() immediately examines
those flags to see if the consumer wants async dispatch. The flag names
are also confusing since they don't specify what "async" applies to:
dispatch or completion.

Add a new KPI, crypto_dispatch_async(), rather than encoding the
requested dispatch type in each cryptop. crypto_dispatch_async() falls
back to crypto_dispatch() if the session's driver provides asynchronous
dispatch. Get rid of CRYPTOP_ASYNC() and CRYPTOP_ASYNC_KEEPORDER().

Similarly, add crypto_dispatch_batch() to request processing of a tailq
of cryptops, rather than encoding the scheduling policy using cryptop
flags.  Convert GELI, the only user of this interface (disabled by
default) to use the new interface.

Add CRYPTO_SESS_SYNC(), which can be used by consumers to determine
whether crypto requests will be dispatched synchronously. This is just
a helper macro. Use it instead of looking at cap flags directly.

Fix style in crypto_done(). Also get rid of CRYPTO_RETW_EMPTY() and
just check the relevant queues directly. This could result in some
unnecessary wakeups but I think it's very uncommon to be using more than
one queue per worker in a given workload, so checking all three queues
is a waste of cycles.

Reviewed by:	jhb
Sponsored by:	Ampere Computing
Submitted by:	Klara, Inc.
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D28194
2021-02-08 09:19:19 -05:00
John Baldwin
688f8b822c Remove the cloned file descriptors for /dev/crypto.
Crypto file descriptors were added in the original OCF import as a way
to provide per-open data (specifically the list of symmetric
sessions).  However, this gives a bit of a confusing API where one has
to open /dev/crypto and then invoke an ioctl to obtain a second file
descriptor.  This also does not match the API used with /dev/crypto on
other BSDs or with Linux's /dev/crypto driver.

Character devices have gained support for per-open data via cdevpriv
since OCF was imported, so use cdevpriv to simplify the userland API
by permitting ioctls directly on /dev/crypto descriptors.

To provide backwards compatibility, CRIOGET now opens another
/dev/crypto descriptor via kern_openat() rather than dup'ing the
existing file descriptor.  This preserves prior semantics in case
CRIOGET is invoked multiple times on a single file descriptor.

Reviewed by:	markj
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27302
2020-11-25 00:10:54 +00:00
John Baldwin
3acf4d2374 Use void * in place of caddr_t.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27065
2020-11-06 18:09:52 +00:00
John Baldwin
d3d79e968b Consistently use C99 fixed-width types in the in-kernel crypto code.
Reviewed by:	markj
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D27061
2020-11-03 22:27:54 +00:00
Mark Johnston
d588dc7d84 opencrypto: Annotate hmac_init_(i|o)pad() to make auth_hash const
This makes them friendlier to drivers that try to use const pointers
whenever possible in their internal structures.

Reviewed by:	jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D26901
2020-10-30 17:05:36 +00:00
Marcin Wojtas
7e89ae49db Prepare crypto framework for IPsec ESN support
This permits requests (netipsec ESP and AH protocol) to provide the
IPsec ESN (Extended Sequence Numbers) in a separate buffer.

As with separate output buffer and separate AAD buffer not all drivers
support this feature. Consumer must request use of this feature via new
session flag.

Submitted by:           Grzegorz Jaszczyk <jaz@semihalf.com>
                        Patryk Duda <pdk@semihalf.com>
Reviewed by:            jhb
Differential revision:  https://reviews.freebsd.org/D24838
Obtained from:          Semihalf
Sponsored by:           Stormshield
2020-10-16 11:06:33 +00:00
Alan Somers
e6f6d0c9bc crypto(9): add CRYPTO_BUF_VMPAGE
crypto(9) functions can now be used on buffers composed of an array of
vm_page_t structures, such as those stored in an unmapped struct bio.  It
requires the running to kernel to support the direct memory map, so not all
architectures can use it.

Reviewed by:	markj, kib, jhb, mjg, mat, bcr (manpages)
MFC after:	1 week
Sponsored by:	Axcient
Differential Revision:	https://reviews.freebsd.org/D25671
2020-08-26 02:37:42 +00:00
John Baldwin
946b8f6fb0 Add crypto_initreq() and crypto_destroyreq().
These routines are similar to crypto_getreq() and crypto_freereq() but
operate on caller-supplied storage instead of allocating crypto
requests from a UMA zone.

Reviewed by:	markj
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D25691
2020-07-16 21:30:46 +00:00
Mark Johnston
7290cb47fc Convert cryptostats to a counter_u64 array.
The global counters were not SMP-friendly.  Use per-CPU counters
instead.

Reviewed by:	jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D25466
2020-06-30 22:01:21 +00:00
Mark Johnston
a5c053f5a7 Remove CRYPTO_TIMING.
It was added a very long time ago.  It is single-threaded, so only
really useful for basic measurements, and in the meantime we've gotten
some more sophisticated profiling tools.

Reviewed by:	cem, delphij, jhb
Sponsored by:	Rubicon Communications, LLC (Netgate)
Differential Revision:	https://reviews.freebsd.org/D25464
2020-06-30 15:56:54 +00:00
John Baldwin
9b774dc0c5 Add support to the crypto framework for separate AAD buffers.
This permits requests to provide the AAD in a separate side buffer
instead of as a region in the crypto request input buffer.  This is
useful when the main data buffer might not contain the full AAD
(e.g. for TLS or IPsec with ESN).

Unlike separate IVs which are constrained in size and stored in an
array in struct cryptop, separate AAD is provided by the caller
setting a new crp_aad pointer to the buffer.  The caller must ensure
the pointer remains valid and the buffer contents static until the
request is completed (e.g. when the callback routine is invoked).

As with separate output buffers, not all drivers support this feature.
Consumers must request use of this feature via a new session flag.

To aid in driver testing, kern.crypto.cryptodev_separate_aad can be
set to force /dev/crypto requests to use a separate AAD buffer.

Discussed with:	cem
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25288
2020-06-22 23:20:43 +00:00
John Baldwin
9b6b2f8608 Adjust crypto_apply function callbacks for OCF.
- crypto_apply() is only used for reading a buffer to compute a
  digest, so change the data pointer to a const pointer.

- To better match m_apply(), change the data pointer type to void *
  and the length from uint16_t to u_int.  The length field in
  particular matters as none of the apply logic was splitting requests
  larger than UINT16_MAX.

- Adjust the auth_xform Update callback to match the function
  prototype passed to crypto_apply() and crypto_apply_buf().  This
  removes the needs for casts when using the Update callback.

- Change the Reinit and Setkey callbacks to also use a u_int length
  instead of uint16_t.

- Update auth transforms for the changes.  While here, use C99
  initializers for auth_hash structures and avoid casts on callbacks.

Reviewed by:	cem
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25171
2020-06-10 21:18:19 +00:00
John Baldwin
a3d565a118 Add a crypto capability flag for accelerated software drivers.
Use this in GELI to print out a different message when accelerated
software such as AESNI is used vs plain software crypto.

While here, simplify the logic in GELI a bit for determing which type
of crypto driver was chosen the first time by examining the
capabilities of the matched driver after a single call to
crypto_newsession rather than making separate calls with different
flags.

Reviewed by:	delphij
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D25126
2020-06-09 22:26:07 +00:00
John Baldwin
33f3bad35e Export the _kern_crypto sysctl node from crypto.c.
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24545
2020-05-25 22:18:33 +00:00
John Baldwin
9c0e3d3a53 Add support for optional separate output buffers to in-kernel crypto.
Some crypto consumers such as GELI and KTLS for file-backed sendfile
need to store their output in a separate buffer from the input.
Currently these consumers copy the contents of the input buffer into
the output buffer and queue an in-place crypto operation on the output
buffer.  Using a separate output buffer avoids this copy.

- Create a new 'struct crypto_buffer' describing a crypto buffer
  containing a type and type-specific fields.  crp_ilen is gone,
  instead buffers that use a flat kernel buffer have a cb_buf_len
  field for their length.  The length of other buffer types is
  inferred from the backing store (e.g. uio_resid for a uio).
  Requests now have two such structures: crp_buf for the input buffer,
  and crp_obuf for the output buffer.

- Consumers now use helper functions (crypto_use_*,
  e.g. crypto_use_mbuf()) to configure the input buffer.  If an output
  buffer is not configured, the request still modifies the input
  buffer in-place.  A consumer uses a second set of helper functions
  (crypto_use_output_*) to configure an output buffer.

- Consumers must request support for separate output buffers when
  creating a crypto session via the CSP_F_SEPARATE_OUTPUT flag and are
  only permitted to queue a request with a separate output buffer on
  sessions with this flag set.  Existing drivers already reject
  sessions with unknown flags, so this permits drivers to be modified
  to support this extension without requiring all drivers to change.

- Several data-related functions now have matching versions that
  operate on an explicit buffer (e.g. crypto_apply_buf,
  crypto_contiguous_subsegment_buf, bus_dma_load_crp_buf).

- Most of the existing data-related functions operate on the input
  buffer.  However crypto_copyback always writes to the output buffer
  if a request uses a separate output buffer.

- For the regions in input/output buffers, the following conventions
  are followed:
  - AAD and IV are always present in input only and their
    fields are offsets into the input buffer.
  - payload is always present in both buffers.  If a request uses a
    separate output buffer, it must set a new crp_payload_start_output
    field to the offset of the payload in the output buffer.
  - digest is in the input buffer for verify operations, and in the
    output buffer for compute operations.  crp_digest_start is relative
    to the appropriate buffer.

- Add a crypto buffer cursor abstraction.  This is a more general form
  of some bits in the cryptosoft driver that tried to always use uio's.
  However, compared to the original code, this avoids rewalking the uio
  iovec array for requests with multiple vectors.  It also avoids
  allocate an iovec array for mbufs and populating it by instead walking
  the mbuf chain directly.

- Update the cryptosoft(4) driver to support separate output buffers
  making use of the cursor abstraction.

Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24545
2020-05-25 22:12:04 +00:00
John Baldwin
d65771e472 Correct the minimum key length for Camellia to 16 bytes (128 bits).
MFC after:	1 week
2020-05-22 17:21:22 +00:00
John Baldwin
63823cac92 Remove MD5 HMAC from OCF.
There are no in-kernel consumers.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24775
2020-05-11 22:08:08 +00:00
John Baldwin
0e00c709d7 Remove support for DES and Triple DES from OCF.
It no longer has any in-kernel consumers via OCF.  smbfs still uses
single DES directly, so sys/crypto/des remains for that use case.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24773
2020-05-11 21:34:29 +00:00
John Baldwin
32075647ef Remove support for the Blowfish algorithm from OCF.
It no longer has any in-kernel consumers.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24772
2020-05-11 21:24:05 +00:00
John Baldwin
33fb013e16 Remove support for the ARC4 algorithm from OCF.
There are no longer any in-kernel consumers.  The software
implementation was also a non-functional stub.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24771
2020-05-11 21:17:08 +00:00
John Baldwin
3a0b6a93a7 Remove support for keyed MD5 and SHA1 authentication hashes.
They no longer have any in-tree consumers.  Note that these are a
different from MD5-HMAC and SHA1-HMAC and were only used with IPsec.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24770
2020-05-11 21:04:59 +00:00
John Baldwin
5e46d47f93 Remove support for the skipjack encryption algorithm.
This was removed from IPsec in r286100 and no longer has any in-tree
consumers.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24769
2020-05-11 20:54:59 +00:00
John Baldwin
7971a6f911 Remove support for the cast128 encryption algorithm.
It no longer has any in-tree consumers.

Reviewed by:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D24768
2020-05-11 20:52:18 +00:00
John Baldwin
29fe41ddd7 Retire the CRYPTO_F_IV_GENERATE flag.
The sole in-tree user of this flag has been retired, so remove this
complexity from all drivers.  While here, add a helper routine drivers
can use to read the current request's IV into a local buffer.  Use
this routine to replace duplicated code in nearly all drivers.

Reviewed by:	cem
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D24450
2020-04-20 22:24:49 +00:00
John Baldwin
c034143269 Refactor driver and consumer interfaces for OCF (in-kernel crypto).
- The linked list of cryptoini structures used in session
  initialization is replaced with a new flat structure: struct
  crypto_session_params.  This session includes a new mode to define
  how the other fields should be interpreted.  Available modes
  include:

  - COMPRESS (for compression/decompression)
  - CIPHER (for simply encryption/decryption)
  - DIGEST (computing and verifying digests)
  - AEAD (combined auth and encryption such as AES-GCM and AES-CCM)
  - ETA (combined auth and encryption using encrypt-then-authenticate)

  Additional modes could be added in the future (e.g. if we wanted to
  support TLS MtE for AES-CBC in the kernel we could add a new mode
  for that.  TLS modes might also affect how AAD is interpreted, etc.)

  The flat structure also includes the key lengths and algorithms as
  before.  However, code doesn't have to walk the linked list and
  switch on the algorithm to determine which key is the auth key vs
  encryption key.  The 'csp_auth_*' fields are always used for auth
  keys and settings and 'csp_cipher_*' for cipher.  (Compression
  algorithms are stored in csp_cipher_alg.)

- Drivers no longer register a list of supported algorithms.  This
  doesn't quite work when you factor in modes (e.g. a driver might
  support both AES-CBC and SHA2-256-HMAC separately but not combined
  for ETA).  Instead, a new 'crypto_probesession' method has been
  added to the kobj interface for symmteric crypto drivers.  This
  method returns a negative value on success (similar to how
  device_probe works) and the crypto framework uses this value to pick
  the "best" driver.  There are three constants for hardware
  (e.g. ccr), accelerated software (e.g. aesni), and plain software
  (cryptosoft) that give preference in that order.  One effect of this
  is that if you request only hardware when creating a new session,
  you will no longer get a session using accelerated software.
  Another effect is that the default setting to disallow software
  crypto via /dev/crypto now disables accelerated software.

  Once a driver is chosen, 'crypto_newsession' is invoked as before.

- Crypto operations are now solely described by the flat 'cryptop'
  structure.  The linked list of descriptors has been removed.

  A separate enum has been added to describe the type of data buffer
  in use instead of using CRYPTO_F_* flags to make it easier to add
  more types in the future if needed (e.g. wired userspace buffers for
  zero-copy).  It will also make it easier to re-introduce separate
  input and output buffers (in-kernel TLS would benefit from this).

  Try to make the flags related to IV handling less insane:

  - CRYPTO_F_IV_SEPARATE means that the IV is stored in the 'crp_iv'
    member of the operation structure.  If this flag is not set, the
    IV is stored in the data buffer at the 'crp_iv_start' offset.

  - CRYPTO_F_IV_GENERATE means that a random IV should be generated
    and stored into the data buffer.  This cannot be used with
    CRYPTO_F_IV_SEPARATE.

  If a consumer wants to deal with explicit vs implicit IVs, etc. it
  can always generate the IV however it needs and store partial IVs in
  the buffer and the full IV/nonce in crp_iv and set
  CRYPTO_F_IV_SEPARATE.

  The layout of the buffer is now described via fields in cryptop.
  crp_aad_start and crp_aad_length define the boundaries of any AAD.
  Previously with GCM and CCM you defined an auth crd with this range,
  but for ETA your auth crd had to span both the AAD and plaintext
  (and they had to be adjacent).

  crp_payload_start and crp_payload_length define the boundaries of
  the plaintext/ciphertext.  Modes that only do a single operation
  (COMPRESS, CIPHER, DIGEST) should only use this region and leave the
  AAD region empty.

  If a digest is present (or should be generated), it's starting
  location is marked by crp_digest_start.

  Instead of using the CRD_F_ENCRYPT flag to determine the direction
  of the operation, cryptop now includes an 'op' field defining the
  operation to perform.  For digests I've added a new VERIFY digest
  mode which assumes a digest is present in the input and fails the
  request with EBADMSG if it doesn't match the internally-computed
  digest.  GCM and CCM already assumed this, and the new AEAD mode
  requires this for decryption.  The new ETA mode now also requires
  this for decryption, so IPsec and GELI no longer do their own
  authentication verification.  Simple DIGEST operations can also do
  this, though there are no in-tree consumers.

  To eventually support some refcounting to close races, the session
  cookie is now passed to crypto_getop() and clients should no longer
  set crp_sesssion directly.

- Assymteric crypto operation structures should be allocated via
  crypto_getkreq() and freed via crypto_freekreq().  This permits the
  crypto layer to track open asym requests and close races with a
  driver trying to unregister while asym requests are in flight.

- crypto_copyback, crypto_copydata, crypto_apply, and
  crypto_contiguous_subsegment now accept the 'crp' object as the
  first parameter instead of individual members.  This makes it easier
  to deal with different buffer types in the future as well as
  separate input and output buffers.  It's also simpler for driver
  writers to use.

- bus_dmamap_load_crp() loads a DMA mapping for a crypto buffer.
  This understands the various types of buffers so that drivers that
  use DMA do not have to be aware of different buffer types.

- Helper routines now exist to build an auth context for HMAC IPAD
  and OPAD.  This reduces some duplicated work among drivers.

- Key buffers are now treated as const throughout the framework and in
  device drivers.  However, session key buffers provided when a session
  is created are expected to remain alive for the duration of the
  session.

- GCM and CCM sessions now only specify a cipher algorithm and a cipher
  key.  The redundant auth information is not needed or used.

- For cryptosoft, split up the code a bit such that the 'process'
  callback now invokes a function pointer in the session.  This
  function pointer is set based on the mode (in effect) though it
  simplifies a few edge cases that would otherwise be in the switch in
  'process'.

  It does split up GCM vs CCM which I think is more readable even if there
  is some duplication.

- I changed /dev/crypto to support GMAC requests using CRYPTO_AES_NIST_GMAC
  as an auth algorithm and updated cryptocheck to work with it.

- Combined cipher and auth sessions via /dev/crypto now always use ETA
  mode.  The COP_F_CIPHER_FIRST flag is now a no-op that is ignored.
  This was actually documented as being true in crypto(4) before, but
  the code had not implemented this before I added the CIPHER_FIRST
  flag.

- I have not yet updated /dev/crypto to be aware of explicit modes for
  sessions.  I will probably do that at some point in the future as well
  as teach it about IV/nonce and tag lengths for AEAD so we can support
  all of the NIST KAT tests for GCM and CCM.

- I've split up the exising crypto.9 manpage into several pages
  of which many are written from scratch.

- I have converted all drivers and consumers in the tree and verified
  that they compile, but I have not tested all of them.  I have tested
  the following drivers:

  - cryptosoft
  - aesni (AES only)
  - blake2
  - ccr

  and the following consumers:

  - cryptodev
  - IPsec
  - ktls_ocf
  - GELI (lightly)

  I have not tested the following:

  - ccp
  - aesni with sha
  - hifn
  - kgssapi_krb5
  - ubsec
  - padlock
  - safe
  - armv8_crypto (aarch64)
  - glxsb (i386)
  - sec (ppc)
  - cesa (armv7)
  - cryptocteon (mips64)
  - nlmsec (mips64)

Discussed with:	cem
Relnotes:	yes
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D23677
2020-03-27 18:25:23 +00:00
Sean Eric Fagan
507281e55e Add AES-CCM encryption, and plumb into OCF.
This commit essentially has three parts:

* Add the AES-CCM encryption hooks.  This is in and of itself fairly small,
as there is only a small difference between CCM and the other ICM-based
algorithms.
* Hook the code into the OpenCrypto framework.  This is the bulk of the
changes, as the algorithm type has to be checked for, and the differences
between it and GCM dealt with.
* Update the cryptocheck tool to be aware of it.  This is invaluable for
confirming that the code works.

This is a software-only implementation, meaning that the performance is very
low.

Sponsored by:	iXsystems Inc.
Differential Revision:	https://reviews.freebsd.org/D19090
2019-02-15 03:53:03 +00:00
Sean Eric Fagan
a99bc4c3eb Add CBC-MAC authentication.
This adds the CBC-MAC code to the kernel, but does not hook it up to
anything (that comes in the next commit).

https://tools.ietf.org/html/rfc3610 describes the algorithm.

Note that this is a software-only implementation, which means it is
fairly slow.

Sponsored by:   iXsystems Inc
Differential Revision:  https://reviews.freebsd.org/D18592
2019-02-15 03:46:39 +00:00
Marius Strobl
345c692d18 As struct cryptop is wrapped in #ifdef _KERNEL, userland doesn't
need to drag in <sys/_task.h> either.
2019-02-10 21:27:03 +00:00
Matt Macy
ff2038a9bf Generalize AES iov optimization
Right now, aesni_cipher_alloc does a bit of special-casing
for CRYPTO_F_IOV, to not do any allocation if the first uio
is large enough for the requested size. While working on ZFS
crypto port, I ran into horrible performance because the code
uses scatter-gather, and many of the times the data to encrypt
was in the second entry. This code looks through the list, and
tries to see if there is a single uio that can contain the
requested data, and, if so, uses that.

This has a slight impact on the current consumers, in that the
check is a little more complicated for the ones that use
CRYPTO_F_IOV -- but none of them meet the criteria for testing
more than one.

Submitted by:	sef at ixsystems.com
Reviewed by:	cem@
MFC after:	3 days
Sponsored by:	iX Systems
Differential Revision:	https://reviews.freebsd.org/D18522
2018-12-13 04:40:53 +00:00
Conrad Meyer
01d5de8fca Add xform-conforming auth_hash wrapper for Poly-1305
The wrapper is a thin shim around libsodium's Poly-1305 implementation.  For
now, we just use the C algorithm and do not attempt to build the
SSE-optimized variant for x86 processors.

The algorithm support has not yet been plumbed through cryptodev, or added
to cryptosoft.
2018-08-17 00:30:04 +00:00
Conrad Meyer
1b0909d51a OpenCrypto: Convert sessions to opaque handles instead of integers
Track session objects in the framework, and pass handles between the
framework (OCF), consumers, and drivers.  Avoid redundancy and complexity in
individual drivers by allocating session memory in the framework and
providing it to drivers in ::newsession().

Session handles are no longer integers with information encoded in various
high bits.  Use of the CRYPTO_SESID2FOO() macros should be replaced with the
appropriate crypto_ses2foo() function on the opaque session handle.

Convert OCF drivers (in particular, cryptosoft, as well as myriad others) to
the opaque handle interface.  Discard existing session tracking as much as
possible (quick pass).  There may be additional code ripe for deletion.

Convert OCF consumers (ipsec, geom_eli, krb5, cryptodev) to handle-style
interface.  The conversion is largely mechnical.

The change is documented in crypto.9.

Inspired by
https://lists.freebsd.org/pipermail/freebsd-arch/2018-January/018835.html .

No objection from:	ae (ipsec portion)
Reported by:	jhb
2018-07-18 00:56:25 +00:00
Conrad Meyer
2e08e39ff5 OCF: Add a typedef for session identifiers
No functional change.

This should ease the transition from an integer session identifier model to
an opaque pointer model.
2018-07-13 23:46:07 +00:00
Conrad Meyer
c4729f6e89 OCF: Add plain hash modes
In part, to support OpenSSL's use of cryptodev, which puts the HMAC pieces
in software and only offloads the raw hash primitive.

The following cryptodev identifiers are added:

 * CRYPTO_RIPEMD160 (not hooked up)
 * CRYPTO_SHA2_224
 * CRYPTO_SHA2_256
 * CRYPTO_SHA2_384
 * CRYPTO_SHA2_512

The plain SHA1 and 2 hashes are plumbed through cryptodev (feels like there
is a lot of redundancy here...) and cryptosoft.

This adds new auth_hash implementations for the plain hashes, as well as
SHA1 (which had a cryptodev.h identifier, but no implementation).

Add plain SHA 1 and 2 hash tests to the cryptocheck tool.

Motivation stems from John Baldwin's earlier OCF email,
https://lists.freebsd.org/pipermail/freebsd-arch/2018-January/018835.html .
2018-07-09 07:28:13 +00:00
Conrad Meyer
c97f39ce17 OCF: Add CRYPTO_SHA2_224_HMAC mode
Round out the complete set of basic SHA2 HMAC modes with SHA2-224.

Support is added to the cryptocheck test tool.
2018-07-09 07:26:12 +00:00
Conrad Meyer
590adc1bc2 Remove "HMAC" from <HASH>_HMAC_BLOCK_LEN macro names
The block size is a property of the underlying hash algorithm, and has
nothing to do with the HMAC construction.

No functional change.
2018-07-09 07:21:37 +00:00
Conrad Meyer
61590291a8 opencrypto: Integrate Chacha20 algorithm into OCF
Mostly this is a thin shim around existing code to integrate with enc_xform
and cryptosoft (+ cryptodev).

Expand the cryptodev buffer used to match that of Chacha20's native block
size as a performance enhancement for chacha20_xform_crypt_multi.
2018-03-29 04:02:50 +00:00
Conrad Meyer
289b9798be OCF: CRYPTDEB(): Enhance to allow formatted logging
Sponsored by:	Dell EMC Isilon
2018-03-26 22:31:29 +00:00
Conrad Meyer
0e33efe4e4 Import Blake2 algorithms (blake2b, blake2s) from libb2
The upstream repository is on github BLAKE2/libb2.  Files landed in
sys/contrib/libb2 are the unmodified upstream files, except for one
difference:  secure_zero_memory's contents have been replaced with
explicit_bzero() only because the previous implementation broke powerpc
link.  Preferential use of explicit_bzero() is in progress upstream, so
it is anticipated we will be able to drop this diff in the future.

sys/crypto/blake2 contains the source files needed to port libb2 to our
build system, a wrapped (limited) variant of the algorithm to match the API
of our auth_transform softcrypto abstraction, incorporation into the Open
Crypto Framework (OCF) cryptosoft(4) driver, as well as an x86 SSE/AVX
accelerated OCF driver, blake2(4).

Optimized variants of blake2 are compiled for a number of x86 machines
(anything from SSE2 to AVX + XOP).  On those machines, FPU context will need
to be explicitly saved before using blake2(4)-provided algorithms directly.
Use via cryptodev / OCF saves FPU state automatically, and use via the
auth_transform softcrypto abstraction does not use FPU.

The intent of the OCF driver is mostly to enable testing in userspace via
/dev/crypto.  ATF tests are added with published KAT test vectors to
validate correctness.

Reviewed by:	jhb, markj
Obtained from:	github BLAKE2/libb2
Differential Revision:	https://reviews.freebsd.org/D14662
2018-03-21 16:18:14 +00:00