doc: fix grammar and formatting in compressdev guide
Small improvements made to the compressdev programmer's guide. This includes rephrasing some sentences, fixing grammar, and aligning formatting. Fixes:a584d3bea9
("doc: add compressdev library guide") Fixes:f7095d41bb
("doc: clarify data plane error handling in compressdev") Cc: stable@dpdk.org Signed-off-by: Ciara Power <ciara.power@intel.com> Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
This commit is contained in:
parent
03f0e3608d
commit
4c9484373f
@ -2,7 +2,7 @@
|
||||
Copyright(c) 2017-2018 Cavium Networks.
|
||||
|
||||
Compression Device Library
|
||||
===========================
|
||||
==========================
|
||||
|
||||
The compression framework provides a generic set of APIs to perform compression services
|
||||
as well as to query and configure compression devices both physical(hardware) and virtual(software)
|
||||
@ -32,10 +32,10 @@ From the command line using the --vdev EAL option
|
||||
|
||||
.. Note::
|
||||
|
||||
* If DPDK application requires multiple software compression PMD devices then required
|
||||
number of ``--vdev`` with appropriate libraries are to be added.
|
||||
* If a DPDK application requires multiple software compression PMD devices then the
|
||||
required number of ``--vdev`` args with appropriate libraries are to be added.
|
||||
|
||||
* An Application with multiple compression device instances exposed by the same PMD must
|
||||
* An application with multiple compression device instances exposed by the same PMD must
|
||||
specify a unique name for each device.
|
||||
|
||||
Example: ``--vdev 'pmd0' --vdev 'pmd1'``
|
||||
@ -53,7 +53,7 @@ All virtual compression devices support the following initialization parameters:
|
||||
Device Identification
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Each device, whether virtual or physical is uniquely designated by two
|
||||
Each device, whether virtual or physical, is uniquely designated by two
|
||||
identifiers:
|
||||
|
||||
- A unique device index used to designate the compression device in all functions
|
||||
@ -76,7 +76,7 @@ The ``rte_compressdev_configure`` API is used to configure a compression device.
|
||||
The ``rte_compressdev_config`` structure is used to pass the configuration
|
||||
parameters.
|
||||
|
||||
See *DPDK API Reference* for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
Configuration of Queue Pairs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@ -85,87 +85,88 @@ Each compression device queue pair is individually configured through the
|
||||
``rte_compressdev_queue_pair_setup`` API.
|
||||
|
||||
The ``max_inflight_ops`` is used to pass maximum number of
|
||||
rte_comp_op that could be present in a queue at-a-time.
|
||||
PMD then can allocate resources accordingly on a specified socket.
|
||||
``rte_comp_op`` that could be present in a queue at a time.
|
||||
The PMD can then allocate resources accordingly on a specified socket.
|
||||
|
||||
See *DPDK API Reference* for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
Logical Cores, Memory and Queues Pair Relationships
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Logical Cores, Memory and Queue Pair Relationships
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Library supports NUMA similarly as described in Cryptodev library section.
|
||||
The Compressdev library supports NUMA similarly as described in Cryptodev library section.
|
||||
|
||||
A queue pair cannot be shared and should be exclusively used by a single processing
|
||||
context for enqueuing operations or dequeuing operations on the same compression device
|
||||
A queue pair cannot be shared, and should be exclusively used by a single processing
|
||||
context for enqueuing operations or dequeuing operations on the same compression device,
|
||||
since sharing would require global locks and hinder performance. It is however possible
|
||||
to use a different logical core to dequeue an operation on a queue pair from the logical
|
||||
core on which it was enqueued. This means that a compression burst enqueue/dequeue
|
||||
core on which it was enqueued. This means that for a compression burst, enqueue/dequeue
|
||||
APIs are a logical place to transition from one logical core to another in a
|
||||
data processing pipeline.
|
||||
|
||||
Device Features and Capabilities
|
||||
---------------------------------
|
||||
--------------------------------
|
||||
|
||||
Compression devices define their functionality through two mechanisms, global device
|
||||
features and algorithm features. Global devices features identify device
|
||||
wide level features which are applicable to the whole device such as supported hardware
|
||||
features and algorithm features. Global device features identify device
|
||||
wide level features which are applicable to the whole device, such as supported hardware
|
||||
acceleration and CPU features. List of compression device features can be seen in the
|
||||
RTE_COMPDEV_FF_XXX macros.
|
||||
|
||||
The algorithm features lists individual algo feature which device supports per-algorithm,
|
||||
such as a stateful compression/decompression, checksums operation etc. List of algorithm
|
||||
features can be seen in the RTE_COMP_FF_XXX macros.
|
||||
The algorithm features are features which the device supports per-algorithm,
|
||||
such as a stateful compression/decompression, checksums operation etc.
|
||||
The list of algorithm features can be seen in the RTE_COMP_FF_XXX macros.
|
||||
|
||||
Capabilities
|
||||
~~~~~~~~~~~~
|
||||
Each PMD has a list of capabilities, including algorithms listed in
|
||||
enum ``rte_comp_algorithm`` and its associated feature flag and
|
||||
sliding window range in log base 2 value. Sliding window tells
|
||||
the minimum and maximum size of lookup window that algorithm uses
|
||||
the enum ``rte_comp_algorithm``, its associated feature flag, and
|
||||
sliding window range in log base 2 value. The sliding window range
|
||||
defines the minimum and maximum size of a lookup window that an algorithm uses
|
||||
to find duplicates.
|
||||
|
||||
See *DPDK API Reference* for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
Each Compression poll mode driver defines its array of capabilities
|
||||
for each algorithm it supports. See PMD implementation for capability
|
||||
for each algorithm it supports. See the PMD implementation for capability
|
||||
initialization.
|
||||
|
||||
Capabilities Discovery
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
PMD capability and features are discovered via ``rte_compressdev_info_get`` function.
|
||||
PMD capability and features are discovered via the ``rte_compressdev_info_get`` function.
|
||||
|
||||
The ``rte_compressdev_info`` structure contains all the relevant information for the device.
|
||||
|
||||
See *DPDK API Reference* for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
Compression Operation
|
||||
----------------------
|
||||
---------------------
|
||||
|
||||
DPDK compression supports two types of compression methodologies:
|
||||
|
||||
- Stateless, data associated to a compression operation is compressed without any reference
|
||||
- Stateless - data associated with a compression operation is compressed without any reference
|
||||
to another compression operation.
|
||||
|
||||
- Stateful, data in each compression operation is compressed with reference to previous compression
|
||||
- Stateful - data in each compression operation is compressed with reference to previous compression
|
||||
operations in the same data stream i.e. history of data is maintained between the operations.
|
||||
|
||||
For more explanation, please refer RFC https://www.ietf.org/rfc/rfc1951.txt
|
||||
For more explanation, please refer to the RFC https://www.ietf.org/rfc/rfc1951.txt
|
||||
|
||||
Operation Representation
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Compression operation is described via ``struct rte_comp_op``, which contains both input and
|
||||
A compression operation is described via ``struct rte_comp_op``, which contains both input and
|
||||
output data. The operation structure includes the operation type (stateless or stateful),
|
||||
the operation status and the priv_xform/stream handle, source, destination and checksum buffer
|
||||
the operation status, the priv_xform/stream handle, source, destination and checksum buffer
|
||||
pointers. It also contains the source mempool from which the operation is allocated.
|
||||
PMD updates consumed field with amount of data read from source buffer and produced
|
||||
field with amount of data of written into destination buffer along with status of
|
||||
operation. See section *Produced, Consumed And Operation Status* for more details.
|
||||
The PMD updates the consumed field with the amount of data read from the source buffer,
|
||||
and the produced field with the amount of data written into the destination buffer,
|
||||
along with status of operation.
|
||||
See the section :ref:`compressdev_prod_cons_op_status`: for more details.
|
||||
|
||||
Compression operations mempool also has an ability to allocate private memory with the
|
||||
operation for application's purposes. Application software is responsible for specifying
|
||||
all the operation specific fields in the ``rte_comp_op`` structure which are then used
|
||||
The compression operations mempool also has the ability to allocate private memory with the
|
||||
operation for the application's use. The application software is responsible for specifying
|
||||
all the operation specific fields in the ``rte_comp_op`` structure, which are then used
|
||||
by the compression PMD to process the requested operation.
|
||||
|
||||
|
||||
@ -181,27 +182,27 @@ A ``rte_comp_op`` contains a field indicating the pool it originated from.
|
||||
|
||||
``rte_comp_op_alloc()`` and ``rte_comp_op_bulk_alloc()`` are used to allocate
|
||||
compression operations from a given compression operation mempool.
|
||||
The operation gets reset before being returned to a user so that operation
|
||||
The operation gets reset before being returned to a user so that the operation
|
||||
is always in a good known state before use by the application.
|
||||
|
||||
``rte_comp_op_free()`` is called by the application to return an operation to
|
||||
its allocating pool.
|
||||
|
||||
See *DPDK API Reference* for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
Passing source data as mbuf-chain
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
If input data is scattered across several different buffers, then
|
||||
Application can either parse through all such buffers and make one
|
||||
the application can either parse through all such buffers and make one
|
||||
mbuf-chain and enqueue it for processing or, alternatively, it can
|
||||
make multiple sequential enqueue_burst() calls for each of them
|
||||
processing them statefully. See *Compression API Stateful Operation*
|
||||
make multiple sequential enqueue_burst() calls for each of them,
|
||||
processing them statefully. See :ref:`compressdev_stateful_op`:
|
||||
for stateful processing of ops.
|
||||
|
||||
Operation Status
|
||||
~~~~~~~~~~~~~~~~
|
||||
Each operation carries a status information updated by PMD after it is processed.
|
||||
Following are currently supported:
|
||||
Each operation carries status information updated by the PMD after it is processed.
|
||||
The following are currently supported:
|
||||
|
||||
- RTE_COMP_OP_STATUS_SUCCESS,
|
||||
Operation is successfully completed
|
||||
@ -225,22 +226,25 @@ Following are currently supported:
|
||||
- RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE,
|
||||
Output buffer ran out of space before operation completed, but this
|
||||
is not an error case. Output data up to op.produced can be used and
|
||||
next op in the stream should continue on from op.consumed+1.
|
||||
the next op in the stream should continue on from op.consumed+1.
|
||||
|
||||
Operation status after enqueue / dequeue
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Some of the above values may arise in the op after an
|
||||
``rte_compressdev_enqueue_burst()``. If number ops enqueued < number ops requested then
|
||||
the app should check the op.status of nb_enqd+1. If status is RTE_COMP_OP_STATUS_NOT_PROCESSED,
|
||||
it likely indicates a full-queue case for a hardware device and a retry after dequeuing some ops is likely
|
||||
to be successful. If the op holds any other status, e.g. RTE_COMP_OP_STATUS_INVALID_ARGS, a retry with
|
||||
``rte_compressdev_enqueue_burst()``. If the number of ops enqueued < the number of ops requested
|
||||
then the app should check the op.status of nb_enqd+1.
|
||||
If the status is RTE_COMP_OP_STATUS_NOT_PROCESSED, it likely indicates a full-queue case for a
|
||||
hardware device, and a retry after dequeuing some ops is likely to be successful.
|
||||
If the op holds any other status, e.g. RTE_COMP_OP_STATUS_INVALID_ARGS, a retry with
|
||||
the same op is unlikely to be successful.
|
||||
|
||||
|
||||
.. _compressdev_prod_cons_op_status:
|
||||
|
||||
Produced, Consumed And Operation Status
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
- If status is RTE_COMP_OP_STATUS_SUCCESS,
|
||||
- If the status is RTE_COMP_OP_STATUS_SUCCESS,
|
||||
consumed = amount of data read from input buffer, and
|
||||
produced = amount of data written in destination buffer
|
||||
- If status is RTE_COMP_OP_STATUS_ERROR,
|
||||
@ -253,37 +257,37 @@ Produced, Consumed And Operation Status
|
||||
- If status is RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE,
|
||||
consumed = amount of data read, and
|
||||
produced = amount of data successfully produced until
|
||||
out of space condition hit. PMD has ability to recover
|
||||
from here, so application can submit next op from
|
||||
consumed+1 and a destination buffer with available space.
|
||||
out of space condition hit. The PMD has ability to recover
|
||||
from here, so an application can submit the next op from
|
||||
consumed+1, and a destination buffer with available space.
|
||||
|
||||
Transforms
|
||||
----------
|
||||
|
||||
Compression transforms (``rte_comp_xform``) are the mechanism
|
||||
to specify the details of the compression operation such as algorithm,
|
||||
window size and checksum.
|
||||
window size, and checksum.
|
||||
|
||||
Compression API Hash support
|
||||
----------------------------
|
||||
|
||||
Compression API allows application to enable digest calculation
|
||||
The compression API allows an application to enable digest calculation
|
||||
alongside compression and decompression of data. A PMD reflects its
|
||||
support for hash algorithms via capability algo feature flags.
|
||||
If supported, PMD calculates digest always on plaintext i.e.
|
||||
If supported, the PMD always calculates the digest on plaintext i.e.
|
||||
before compression and after decompression.
|
||||
|
||||
Currently supported list of hash algos are SHA-1 and SHA2 family
|
||||
SHA256.
|
||||
|
||||
See *DPDK API Reference* for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
If required, application should set valid hash algo in compress
|
||||
If required, the application should set the valid hash algo in compress
|
||||
or decompress xforms during ``rte_compressdev_stream_create()``
|
||||
or ``rte_compressdev_private_xform_create()`` and pass a valid
|
||||
or ``rte_compressdev_private_xform_create()``, and pass a valid
|
||||
output buffer in ``rte_comp_op`` hash field struct to store the
|
||||
resulting digest. Buffer passed should be contiguous and large
|
||||
enough to store digest which is 20 bytes for SHA-1 and
|
||||
resulting digest. The buffer passed should be contiguous and large
|
||||
enough to store digest, which is 20 bytes for SHA-1 and
|
||||
32 bytes for SHA2-256.
|
||||
|
||||
Compression API Stateless operation
|
||||
@ -295,20 +299,21 @@ An op is processed stateless if it has
|
||||
(required only on compression side),
|
||||
- All required input in source buffer
|
||||
|
||||
When all of the above conditions are met, PMD initiates stateless processing
|
||||
When all of the above conditions are met, the PMD initiates stateless processing
|
||||
and releases acquired resources after processing of current operation is
|
||||
complete. Application can enqueue multiple stateless ops in a single burst
|
||||
complete. The application can enqueue multiple stateless ops in a single burst
|
||||
and must attach priv_xform handle to such ops.
|
||||
|
||||
priv_xform in Stateless operation
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
priv_xform is PMD internally managed private data that it maintains to do stateless processing.
|
||||
priv_xforms are initialized provided a generic xform structure by an application via making call
|
||||
to ``rte_compressdev_private_xform_create``, at an output PMD returns an opaque priv_xform reference.
|
||||
If PMD support SHAREABLE priv_xform indicated via algorithm feature flag, then application can
|
||||
attach same priv_xform with many stateless ops at-a-time. If not, then application needs to
|
||||
create as many priv_xforms as it expects to have stateless operations in-flight.
|
||||
A priv_xform is private data managed internally by the PMD to do stateless processing.
|
||||
A priv_xform is initialized by an application providing a generic xform structure
|
||||
to ``rte_compressdev_private_xform_create``, which returns an opaque priv_xform reference.
|
||||
If the PMD supports SHAREABLE priv_xform, indicated via algorithm feature flag,
|
||||
then the application can attach the same priv_xform with many stateless ops at a time.
|
||||
If not, then the application needs to create as many priv_xforms as it expects to have
|
||||
stateless operations in-flight.
|
||||
|
||||
.. figure:: img/stateless-op.*
|
||||
|
||||
@ -320,8 +325,9 @@ create as many priv_xforms as it expects to have stateless operations in-flight.
|
||||
Stateless Ops using Shareable priv_xform
|
||||
|
||||
|
||||
Application should call ``rte_compressdev_private_xform_create()`` and attach to stateless op before
|
||||
enqueuing them for processing and free via ``rte_compressdev_private_xform_free()`` during termination.
|
||||
The application should call ``rte_compressdev_private_xform_create()`` and attach it to a stateless
|
||||
op before enqueuing them for processing and free via ``rte_compressdev_private_xform_free()``
|
||||
during termination.
|
||||
|
||||
An example pseudocode to setup and process NUM_OPS stateless ops with each of length OP_LEN
|
||||
using priv_xform would look like:
|
||||
@ -399,75 +405,80 @@ using priv_xform would look like:
|
||||
|
||||
|
||||
Stateless and OUT_OF_SPACE
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
OUT_OF_SPACE is a condition when output buffer runs out of space and where PMD
|
||||
still has more data to produce. If PMD runs into such condition, then PMD returns
|
||||
RTE_COMP_OP_OUT_OF_SPACE_TERMINATED error. In such case, PMD resets itself and can set
|
||||
OUT_OF_SPACE is a condition when the output buffer runs out of space and where the PMD
|
||||
still has more data to produce. If the PMD runs into such condition, then the PMD returns
|
||||
RTE_COMP_OP_OUT_OF_SPACE_TERMINATED error. In such case, the PMD resets itself and can set
|
||||
consumed=0 and produced=amount of output it could produce before hitting out_of_space.
|
||||
Application would need to resubmit the whole input with a larger output buffer, if it
|
||||
The application would need to resubmit the whole input with a larger output buffer, if it
|
||||
wants the operation to be completed.
|
||||
|
||||
Hash in Stateless
|
||||
~~~~~~~~~~~~~~~~~
|
||||
If hash is enabled, digest buffer will contain valid data after op is successfully
|
||||
If hash is enabled, the digest buffer will contain valid data after an op is successfully
|
||||
processed i.e. dequeued with status = RTE_COMP_OP_STATUS_SUCCESS.
|
||||
|
||||
Checksum in Stateless
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
If checksum is enabled, checksum will only be available after op is successfully
|
||||
If checksum is enabled, checksum will only be available after an op is successfully
|
||||
processed i.e. dequeued with status = RTE_COMP_OP_STATUS_SUCCESS.
|
||||
|
||||
.. _compressdev_stateful_op:
|
||||
|
||||
Compression API Stateful operation
|
||||
-----------------------------------
|
||||
|
||||
Compression API provide RTE_COMP_FF_STATEFUL_COMPRESSION and
|
||||
RTE_COMP_FF_STATEFUL_DECOMPRESSION feature flag for PMD to reflect
|
||||
The compression API provides RTE_COMP_FF_STATEFUL_COMPRESSION and
|
||||
RTE_COMP_FF_STATEFUL_DECOMPRESSION feature flag for the PMD to reflect
|
||||
its support for Stateful operations.
|
||||
|
||||
A Stateful operation in DPDK compression means application invokes enqueue
|
||||
burst() multiple times to process related chunk of data because
|
||||
application broke data into several ops.
|
||||
A Stateful operation in DPDK compression means the application invokes enqueue
|
||||
burst() multiple times to process a related chunk of data because the
|
||||
application broke the data into several ops.
|
||||
|
||||
In such case
|
||||
In such cases
|
||||
- ops are setup with op_type RTE_COMP_OP_STATEFUL,
|
||||
- all ops except last set to flush value = RTE_COMP_FLUSH_NONE/SYNC
|
||||
and last set to flush value RTE_COMP_FLUSH_FULL/FINAL.
|
||||
- all ops except the last are set with flush value = RTE_COMP_FLUSH_NONE/SYNC
|
||||
and the last is set with flush value RTE_COMP_FLUSH_FULL/FINAL.
|
||||
|
||||
In case of either one or all of the above conditions, PMD initiates
|
||||
stateful processing and releases acquired resources after processing
|
||||
In case of either one or all of the above conditions, the PMD initiates
|
||||
stateful processing and releases acquired resources after processing the
|
||||
operation with flush value = RTE_COMP_FLUSH_FULL/FINAL is complete.
|
||||
Unlike stateless, application can enqueue only one stateful op from
|
||||
a particular stream at a time and must attach stream handle
|
||||
Unlike stateless, the application can enqueue only one stateful op from
|
||||
a particular stream at a time and must attach a stream handle
|
||||
to each op.
|
||||
|
||||
Stream in Stateful operation
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
`stream` in DPDK compression is a logical entity which identifies related set of ops, say, a one large
|
||||
file broken into multiple chunks then file is represented by a stream and each chunk of that file is
|
||||
represented by compression op `rte_comp_op`. Whenever application wants a stateful processing of such
|
||||
data, then it must get a stream handle via making call to ``rte_compressdev_stream_create()``
|
||||
with xform, at an output the target PMD will return an opaque stream handle to application which
|
||||
it must attach to all of the ops carrying data of that stream. In stateful processing, every op
|
||||
requires previous op data for compression/decompression. A PMD allocates and set up resources such
|
||||
as history, states, etc. within a stream, which are maintained during the processing of the related ops.
|
||||
A stream in DPDK compression is a logical entity which identifies a related set of ops.
|
||||
For example, one large file broken into multiple chunks, then the file is represented by a stream,
|
||||
and each chunk of that file is represented by a compression op ``rte_comp_op``.
|
||||
Whenever an application wants stateful processing of such data, then it must get a stream handle
|
||||
via making call to ``rte_compressdev_stream_create()`` with an xform, which will return an opaque
|
||||
stream handle to attach to all of the ops carrying data of that stream.
|
||||
In stateful processing, every op requires previous op data for compression/decompression.
|
||||
A PMD allocates and sets up resources such as history, states, etc. within a stream,
|
||||
which are maintained during the processing of related ops.
|
||||
|
||||
Unlike priv_xforms, stream is always a NON_SHAREABLE entity. One stream handle must be attached to only
|
||||
one set of related ops and cannot be reused until all of them are processed with status Success or failure.
|
||||
Unlike priv_xforms, a stream is always a NON_SHAREABLE entity. One stream handle must be attached
|
||||
to only one set of related ops and cannot be reused until all of them are processed with a
|
||||
success/failure status.
|
||||
|
||||
.. figure:: img/stateful-op.*
|
||||
|
||||
Stateful Ops
|
||||
|
||||
|
||||
Application should call ``rte_compressdev_stream_create()`` and attach to op before
|
||||
An application should call ``rte_compressdev_stream_create()`` and attach it to the op before
|
||||
enqueuing them for processing and free via ``rte_compressdev_stream_free()`` during
|
||||
termination. All ops that are to be processed statefully should carry *same* stream.
|
||||
termination. All ops that are to be processed statefully should carry the *same* stream.
|
||||
|
||||
See *DPDK API Reference* document for details.
|
||||
See the `DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_ for details.
|
||||
|
||||
An example pseudocode to set up and process a stream having NUM_CHUNKS with each chunk size of CHUNK_LEN would look like:
|
||||
An example pseudocode to set up and process a stream having NUM_CHUNKS,
|
||||
with each chunk size of CHUNK_LEN, would look like:
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
@ -549,64 +560,65 @@ An example pseudocode to set up and process a stream having NUM_CHUNKS with each
|
||||
|
||||
|
||||
Stateful and OUT_OF_SPACE
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
If PMD supports stateful operation, then OUT_OF_SPACE status is not an actual
|
||||
error for the PMD. In such case, PMD returns with status
|
||||
If a PMD supports stateful operation, then an OUT_OF_SPACE status is not an actual
|
||||
error for the PMD. In such a case, the PMD returns with status
|
||||
RTE_COMP_OP_STATUS_OUT_OF_SPACE_RECOVERABLE with consumed = number of input bytes
|
||||
read and produced = length of complete output buffer.
|
||||
Application should enqueue next op with source starting at consumed+1 and an
|
||||
read, and produced = length of complete output buffer.
|
||||
The application should enqueue the next op with source starting at consumed+1, and an
|
||||
output buffer with available space.
|
||||
|
||||
Hash in Stateful
|
||||
~~~~~~~~~~~~~~~~
|
||||
If enabled, digest buffer will contain valid digest after last op in stream
|
||||
If enabled, the digest buffer will contain valid digest after the last op in a stream
|
||||
(having flush = RTE_COMP_FLUSH_FINAL) is successfully processed i.e. dequeued
|
||||
with status = RTE_COMP_OP_STATUS_SUCCESS.
|
||||
|
||||
Checksum in Stateful
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
If enabled, checksum will only be available after last op in stream
|
||||
If enabled, the checksum will only be available after the last op in a stream
|
||||
(having flush = RTE_COMP_FLUSH_FINAL) is successfully processed i.e. dequeued
|
||||
with status = RTE_COMP_OP_STATUS_SUCCESS.
|
||||
|
||||
Burst in compression API
|
||||
-------------------------
|
||||
------------------------
|
||||
|
||||
Scheduling of compression operations on DPDK's application data path is
|
||||
performed using a burst oriented asynchronous API set. A queue pair on a compression
|
||||
device accepts a burst of compression operations using enqueue burst API. On physical
|
||||
devices the enqueue burst API will place the operations to be processed
|
||||
device accepts a burst of compression operations using the enqueue burst API.
|
||||
On physical devices the enqueue burst API will place the operations to be processed
|
||||
on the device's hardware input queue, for virtual devices the processing of the
|
||||
operations is usually completed during the enqueue call to the compression
|
||||
device. The dequeue burst API will retrieve any processed operations available
|
||||
from the queue pair on the compression device, from physical devices this is usually
|
||||
directly from the devices processed queue, and for virtual device's from a
|
||||
directly from the devices processed queue, and for virtual device's from an
|
||||
``rte_ring`` where processed operations are placed after being processed on the
|
||||
enqueue call.
|
||||
|
||||
A burst in DPDK compression can be a combination of stateless and stateful operations with a condition
|
||||
that for stateful ops only one op at-a-time should be enqueued from a particular stream i.e. no-two ops
|
||||
should belong to same stream in a single burst. However a burst may contain multiple stateful ops as long
|
||||
as each op is attached to a different stream i.e. a burst can look like:
|
||||
A burst in DPDK compression can be a combination of stateless and stateful operations with a
|
||||
condition that for stateful ops only one op at a time should be enqueued from a particular stream
|
||||
i.e. two ops should never belong to the same stream in a single burst.
|
||||
However, a burst may contain multiple stateful ops, as long as each op is attached to a different
|
||||
stream, i.e. a burst can look like:
|
||||
|
||||
+---------------+--------------+--------------+-----------------+--------------+--------------+
|
||||
| enqueue_burst | op1.no_flush | op2.no_flush | op3.flush_final | op4.no_flush | op5.no_flush |
|
||||
+---------------+--------------+--------------+-----------------+--------------+--------------+
|
||||
|
||||
Where, op1 .. op5 all belong to different independent data units. op1, op2, op4, op5 must be stateful
|
||||
as stateless ops can only use flush full or final and op3 can be of type stateless or stateful.
|
||||
Every op with type set to RTE_COMP_OP_STATELESS must be attached to priv_xform and
|
||||
Every op with type set to RTE_COMP_OP_STATEFUL *must* be attached to stream.
|
||||
Where, op1 .. op5 all belong to different independent data units. op1, op2, op4, op5 must be
|
||||
stateful as stateless ops can only use flush full or final and op3 can be of type stateless or
|
||||
stateful. Every op with type set to RTE_COMP_OP_STATELESS must be attached to priv_xform and
|
||||
every op with type set to RTE_COMP_OP_STATEFUL *must* be attached to stream.
|
||||
|
||||
Since each operation in a burst is independent and thus can be completed
|
||||
out-of-order, applications which need ordering, should setup per-op user data
|
||||
area with reordering information so that it can determine enqueue order at
|
||||
out of order, applications which need ordering should setup a per-op user data
|
||||
area, with reordering information so that it can determine enqueue order at
|
||||
dequeue.
|
||||
|
||||
Also if multiple threads calls enqueue_burst() on same queue pair then it’s
|
||||
application onus to use proper locking mechanism to ensure exclusive enqueuing
|
||||
of operations.
|
||||
Also, if multiple threads calls enqueue_burst() on the same queue pair then it's
|
||||
the application's responsibility to use a proper locking mechanism to ensure
|
||||
exclusive enqueuing of operations.
|
||||
|
||||
Enqueue / Dequeue Burst APIs
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
@ -629,9 +641,10 @@ Sample code
|
||||
-----------
|
||||
|
||||
There are unit test applications that show how to use the compressdev library inside
|
||||
app/test/test_compressdev.c
|
||||
``app/test/test_compressdev.c``
|
||||
|
||||
Compression Device API
|
||||
~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
The compressdev Library API is described in the *DPDK API Reference* document.
|
||||
The compressdev Library API is described in the
|
||||
`DPDK API Reference <https://doc.dpdk.org/api/rte__compressdev_8h.html>`_.
|
||||
|
Loading…
Reference in New Issue
Block a user