We allocate from the head, so it's better to free to
the head too for better cache utilization.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c32244f446bd7a1df12eefc81245b3ef7e24070
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10193
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The NOTICELOGs really clutter the output during
application start - it's better to make these DEBUGLOGs
instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3ae37d5d057d7b972017befbc0834de414b9710b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This limitation doesn't really take effect currently,
since the typical number of slots per channel isn't
bigger than MAX_COMPLETIONS_PER_POLL. But there's
no reason for this limit anymore - we should always
poll as many completions as we find.
It's better to remove this now, in case we have
configs in the future with higher number of slots
per channel.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5d90d41e5142622b79d9765fbc62da1516e2b8be
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10189
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Prior a regular round robin could result in strange performance
if an idxd device from another socket was used.
Signed-off-by: peluse <peluse@localhost.localdomain>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Id863c79067beabe73ef89d92b3fb3c436821b97a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9367
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Used to be in the lib directory but an upcoming patch needs
access to it so move it to a more appropriate location.
The changes in the .h file were needed to address compile
error when in the /include dir (didn't get errors elsewhere)
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I44d11fc620f213b13683d62dca899e2792ca6ed5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9450
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The issue is that:
We call cb_fn firstly, then add the related idxd_ops into the
chan->ops_list. Then if in user's callback, we call spdk_idxd_submit_*
function gain, then we will allocate new idxd_opts first. So if there is
recursive spdk_idxd_submit_* in the call back function, then
we will exhaust all the ops_pool resources. And the function will
report -EBUSY issue in _idxd_prep_command function.
And this patch fixes this issue by adding back idxd_ops before calling
user's call_cb function.
Change-Id: I32239fb29be875bc452803a48dc24ff5fa533016
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9401
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add more information to users which implementation cannot
be set for idxd.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ib26f117dc19fa1f28cc1d548549caba0b696a915
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9399
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Small but noticable on perf top. We were using a list to track
valid batches and checking it on every submission. Instead just
put a valid channel ptr in the batch struct and clear it when the
batch is freed. There was no other reason for the list.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia26ceee578be4a82f4fd8abb2358ff18c56271a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9149
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Was set at a pretty high number during early development.
Instead of a #define, lets use the same math we use to
determine the size of the operation and descriptor pools as
that's the max number of batches that will be needed.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ide5311b4b6c931010413d0f82baed0e05d5fcde7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9099
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When address translation failed we were not putting the op back
into the pool. In one place also changed the return error from
translation from hardcoded value to the rc returned by the call
to be consistent with the rest of the code.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6d0c9c0f469a213721c6a5ce37c4e9dbdcef8183
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9097
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
These are fixed, no reason to tranlate them with every IO. This
patch is just for non-batched IO. Batched IO will come later.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ifb7bdf147658a23714d94bb7f88d7fedd1749153
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9073
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In prep for upcoming patch
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iebc5bf5f6715c85cc09b404128338cd6f08c421e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We can instead use a combination of the op code and the batch
element in the op structure to determine if the op that is
completing is part of a batch or not so we know whether to
return it to the free list or not.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I8e27c7b991f5a20e82394a25a52ece560cab5543
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9071
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
No real material impact but easier to read and cleans up error
handling when submitting a batch. Before we put the operation on
the list of operations to poll when it was prepared, a few LOC
before it was submitted to HW. Now we do it in the hw submission
function. For batch elements, they are added just before the
batch operation itself (as was before).
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icfeaeeb650d5b63f81e13a8ffd27e349c7118089
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9069
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The bit arrays were used for dynamic flow control in a previous
implementation. They are no longer needed as flow control is
now static and managed solely in the idxd plug in module. Use
simple lists of descriptors and completion records instead.
This is a simpler implementation and will allow for some future
clean up of structures as well.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I8c4cce12e88ac5416e3fe29a416487759214ec9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8922
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
If there is hardware issues, we do not need to assign
the result. Because we will report the error status to the uplayer.
Change-Id: I647ddd609a1d5d0d52cc4fee59699b9992da4fa4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8864
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Those functions are exported publicly, so better to
add some assert functions to detect some null pointer
errors.
We do not use if/else check, because it is too heavy.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I16814efb84a5a41876657f0caf5f0a6d0c2db8f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8863
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This assert is used to make sure that there is no
active batch (spdk_accel_batch) task is used.
If there are active batches found, it means that
we did not handle this case in a good manner.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7ea6247d2d5a40cf4f3f31cd8b1240fed4648d62
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8857
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This batch_op field is not necessary because we can
use the comp_ctx->desc->opcode to judge whether it is related
a batched task or not.
Change-Id: Id329221ccf272c4c3bb8c1b5ec08433029a9a1f8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8865
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Previously we used a counter of our own to make sure all batch
elements plus the batch itself were done before we freed the batch.
This was due to some observations early on that the batch desc
could complete before the individual elements and a lack of clarity
as to whether this was due to the simulator or the fact that
we poll on completions and could therefore "see" completions in
a different order at that time (we were using bit arrays to poll).
Now we use an ordered (in time) list to poll locations so if we
instead put the elements on the list first and then the batch desc
itself we are assured to always "see" them in order provided the
underlying device meets spec which there's no reason to assume it
does not.
This simplifies things a bit at the same time and still assures
that we call list calbacks in order and then the batch callback
without "special" handling.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4d9e3997786f2116ce6515682b8117799c645f51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Left over from when the field was a void *
Signed-off-by: paul Luse <paul.e.luse@intel.com>
Change-Id: If9dfe2878f6afd6137d6d8efec90e310baf417f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8280
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Allows for better performance by not hitting the same portal
address with every submission.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I1ec8eae6f3acec9e98161029cd5406ec08603aa6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
This patch is used to add the support for users to configure
use kernel or userspace idxd library.
Change-Id: Ie159b897bc9595894ad8f333168efaea6c2a3d78
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7332
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch is used to add the kernel idxd support.
Without this patch, we can use userspace idxd driver
under accel_engine library (module/accel/idxd/accel_engine).
With this patch, we can also kernel idxd driver under the
accel_engine library.
Our approach is implementing a wrapper library to use IDXD
device by leveraging the kernel DSA driver in SPDK idxd library
(lib/idxd).
Then users can leverage the RPC later to configure how to
use the DSA device by user space driver or kernel driver.
In this patch, our approach is to use the idxd-config library
to export the WQs (Working Queues) exported by the kernel.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I3a25a4fe0327bd626bf6883dfbe54437d3209e51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7331
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Just remove this function pointer and add a new one,i.e.,
dump_sw_error.
Because this function pointer is only used to
read a sw err info. We can hide it in the detailed
idxd implemenation.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I42fe2220dae85df307b5af64e37acfd7f748915b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8707
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Was using "dst" in some cases and "crc_dst" in others for crc32c
related calls. Update them to always use crc_dst
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icf200f1734c64c29881f23b02b8d12bad81b3ca0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8186
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Recent work identified race conditions having to do with the
dynamic flow control mechanism for the idxd engine. In order
to both address the issue and simplify the code a new scheme
is now in place. Essentially every DSA device will be allowed
to accomodate 8 channels and each channel will get a fixed 1/8
the number of work queue entries regardless of how many
channels there are. Assignment of channels to devices is round
robin and if/when no more channels can be accommodated the get
channel request will fail.
The performance tests also revealed another issue that was
masked before, it's a one-line so is in this patch for convenience.
In the idxd poller we limit the number of completions allowed
during one run to avoid the poller thread from starving other
threads since as operations complete on this thread they are
immediately replaced up to the limit for the channel.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I913e809a934b562feb495815a9b9c605d622285c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8171
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In spdk_idxd_configure_chan(), if memory allocation fails in
TAILQ_FOREACH() {} code range, we will goto err_user_comp and
err_user_desc tag, in which we donot free chan->completions
and confused batch->user_completions with chan->completions.
Memleak problem and double free problem may occurs.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: I0e588a35184d97cab0ea6b6c013ca8b3342f940a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8432
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
In spdk_idxd_get_channel(), if chan->batch_base is allocated
faild, we should free chan before returning NULL.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: Ia652c334aead592429c1171da73d67160879686d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8301
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Upcoming patches will add accel_fw support for batching this cmd
and then vectored versions later along with accel_perf to exercise
them.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I59f577f4365fbf063d7419cc6052e10c998b58bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8143
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Was using reserved field. Similar fix to what was done earlier
for direct submission of crc32c operation.
fixes#1972
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie9867e72f60c7f38aa1af0273a036f34580ed4c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
To match regular sumission prep function and allow caller to
modify both descriptor and completion structures. Also allows
for more accurate error reporting. Needed for upcoming patch to
fix batch CRC submissions.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4b7b5b24e2f54149b513d4b23ba32f3802aff3e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8144
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Upcoming patches will add support to the accel fw, the idxd engine and
the accel_perf tool. Also following will come vectored support and
batch versions.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie0fbd4b8da9f727426000898c0b511587adac65b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8139
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Perf improvement, directs DSA to write to cache as opposed to
mem.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I0d6ba157af8f1b54f8aae3b8e54a6f7754e4a9de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8169
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We already keep a list of outstanding completion locations to
poll and were previously polling all of them. New ones are
added at the tail and we poll the oldest first from the head
so if we break when we find a slot that hasn't completed we
can get more work done while the HW finishes. This is a proven
performance improvement in limited testing.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icc15041605586f9a31435d447d253c381c00b1f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8161
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Was using reserved field in CRC to store the final address of where
to put the result, this not legal. Move to the completion record
and slightly re-arrange the struct to keep it at 96 bytes.
Refacorted the IO prep function so the caller can udpate both the
descriptor and completion records instead of continuing to add
parameters to the prep function for opcodes that need something
unique in the completion record. This also allowed for a minor
fix where the prep function was returning NULL when vtophys failed
which would have indicated busy as opposed to failire. Now we can
proprely fail that path.
fixes#1929
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic23bc7b68bdd5757c30b7963880677f423368e20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Not just one with extra available info. Also remove the extra
read of the error register, not required.
fixes: #1927
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I28badb45d8cc8d16b72f7019bd2a2044998fc402
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7729
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Purpose: We will also support the kernel idxd driver, so we do not
need export this feature in the module file.
Change-Id: I965e031497920f527962ba187bccd81de6977b8f
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7336
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6ad71a65244526e99a36920c630096cc9739d94d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7809
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Purpose: This patch is used to prepare to add the kernel
idxd support later.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: If89665f95d622c7342ab75050664158ec6fc615a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7330
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This patch is used to update the field definition related with
work queue in the header file.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I68b81d9dfc2497db89e96f0730785be03dcb8add
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Add print to confirm how groups/queues/engines are being
programmed based on the init RPC used.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic9462c19c6899478a803433f90d9db9249dd5ca1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Developer convenience - make this based on a specific version.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I228b8aff6e8957cad5e8c1fae5615b113e16cfb5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5950
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
In the completion handler there's no need to do an MMIO read
unless the completion record indicates there's an error.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic4850d803a23413f9813da30ac6f1b611804f1b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5847
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For flow control reasons we have to resize the bit arrays we
use to manage flow as channels come and go. However since
channels are assigned to devices, until the channel count
reaches the device count there's no sharing so no resize of
the array is needed. So, when we use a device for the first
time there's no need to run through the rest of the channels
and re-balance.
Same thing is done on destruction. The code to free idxd
specific resources was moved from the rebalance function to
the idxd put channel function which is a much more logical
place for it as well.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib4df163286906f413dd6429dc6833af7b68e208c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5846
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
