Returning an error from this function is not useful - there
is nothing the caller can do with that information. So
change the return value to void. Also add ERRLOG and assert
if a transport actually returns a non-zero status, to
force the transport implementer (which must be an out-of-tree
transport) to make changes as necessary.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I402afec045265db178af821d25b99a6dbe066eab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8659
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
It is not uncommon for delete_io_qpair to fail, for
example when a controller is hot removed. So even
if SQ or CQ deletion fails, continue with freeing
resources and report success back up the stack.
There is really nothing the application can do to
account for this failing anyways.
Upcoming patches will add additional checks to
ensure failing delete_io_qpair status never gets
propagated to the caller.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iac007c1eba30f7a8c4936b3ffb6c837f28ee12ae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8658
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Due to the recent changes for non block size multiples write I/O,
the data digest feature was degraded. If Linux iSCSI host enables
data digest and tries to detect LU from SPDK iSCSI target, data
mismatch error is detected and the connection is disconnected
unexpectedly.
The cause was that pdu->data_valid_bytes was not set for non-write
response PDUs which have a data segment.
iscsi_pdu_calc_data_digest() has been used only for non-write response
PDUs. Hence we did not need to change iscsi_pdu_calc_data_digest().
Restore the original implementation of iscsi_pdu_calc_data_digest().
Additionally, to avoid future degradation, rename the related
functions to iscsi_pdu_calc_partial_data_digest() and
iscsi_pdu_calc_partial_data_digest_done(), and add comments for
clarification.
This fix was verified by the reporter.
Fixes#2029.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6babcd1b56e79d3fa3cd26b2dfaad87a52788e63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8635
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Fixes#2022
If queued aborts are present when trying to fail a ctrlr
using spdk_nvme_ctrlr_fail(), then the abort command completion
will attempt to retry one of the queued aborts.
This eventually leads to a segfault that can be avoided by not
retrying any queued aborts.
Change-Id: I897dcb8809e16af8bdd39d4381ab531e1cc29822
Signed-off-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8585
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The vfio-user target emulated NVMe device is treated as
PCIe NVMe SSD in the Guest VM, so when doing controller
reset or shutdown, we should abort the AERs which in the
NVMf library.
Users may switch kernel NVMe driver to SPDK NVMe driver
in the VM, without this fix, we will got "AERL exceeded"
response very frequently, because the AERs submitted by
previous driver will never be aborted in runtime.
Change-Id: I0222ed509629ccb0e98217414dd9043857105686
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8558
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When users remove kernel NVMe driver in the VM, after 120 seconds,
SPDK NVMf target will disconnect ADMIN queue pair due to association
timer timeout, and for vfio-user transport, the ADMIN queue pair
connection is associated with the socket connection, so when probing
the NVMe controller again, because there is no active ADMIN connection
for fabric register R/W commands, it will cause segment fault.
Here we set the association timeout value to 0 for vfio-user transport,
so that the ADMIN connection will not be disconnected when shutdown the
controller, the ADMIN queue pair will be disconnected when the socket
connection breaks.
Change-Id: I3613169229bae384405889653e50f581d30d7c07
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8557
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The NVMf library will set cdw0 based on specific command,
so we use it directly in vfio-user, otherwise, some NVMe
commands such as AER can't work.
Fix issue #2016.
Change-Id: Ie1a80a92c0856b61822ee51ce5d8faaaf1d463de
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8556
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Also fix one incorrect print log.
Change-Id: I3254baf4bbff4acfc0ef43f628d025931e8589ea
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
These macros are only valid for Fabric transports.
Change-Id: Ia456eebdcdab28e81226c1b3a7211fcb41b5e481
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8554
Community-CI: Mellanox Build Bot
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_bdev_register() and spdk_bdev_add_alias() had not held mutex when
adding bdev name or alias to global bdev name tree. This bug caused unexpected
error when traversing global bdev name tree.
The next patch will fix the bug. This patch is a preparation for the fix.
spdk_bdev_get_by_name() had not held mutex while traversing bdev
name tree. The major callers to spdk_bdev_get_by_name() had held mutex
when calling it. However, this was not clear.
Factor out the internal of spdk_bdev_get_by_name() into a helper
function bdev_get_by_name() and then change spdk_bdev_get_by_name()
to lock and unlock when calling bdev_get_by_name().
Then replace spdk_bdev_get_by_name() call in spdk_bdev_alias_add() and
bdev_register() by bdev_get_by_name() call.
spdk_bdev_get_by_name() call in spdk_bdev_examine() is not changed.
This is called only from JSON RPC and not related with the bug. So
we want to fix only unlocked access to global bdev name tree.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I25f07694e569eec10dba6c3c8543f6ce77412fe8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8523
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
It is better to not fail connect commands when a subsystem
is not ready. The host will not be expecting that and will
typically treat it as a catastrophic failure (i.e. it won't
retry the connect).
So instead when this situation occurs, start a poller for
the connect request. We will continue to retry processing
it until the subsystem is ready to handle it.
Fixes issue #1985.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8835df8f0edf1e889fdd7e754e261c2a880cbb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8571
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It is possible for a controller to get added to the
subsystem before its admin_qpair has been assigned.
We need to account for that when traversing the subsystem's
ctrlr list when determining ns and ana_changes that need
to be reported for the ctrlr.
Found while doing stress testing with connects and
subsystem ns add/remove.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie54dc6ac202faeaeace054e6599f2dea2f30211e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8570
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
After correct trstring initialization, it is
overwritten with trstring value of the current
probe ctx. That leads to a problem when initiator
connects to a sbusystem with listeners of different
transport types (e.g. TCP and RDMA). If probe_ctx has
TCP type, than discovery probe initialized probe trid
with trtype=RDMA and trstring=TCP. As results, SPDK
creates TCP controller with trtype=RDMA and we hit
assert in nvme_tcp_qpair function.
Change-Id: I9355450c40c58fa55b016220703f6f7ae36b2571
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Pollers are supposed to return SPDK_POLLER_{BUSY,IDLE}.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I92bd184aaba9e3efb730b68a6024ebc9757ffd8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8559
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
If we continuous setup and teardown cuse session, It will teardown
uninitialized cuse session and cause segment fault, New function
cuse_session_create will do the session create operation and under
g_cuse_mtx to avoid this issue.
Signed-off-by: Weifeng Su <suweifeng1@huawei.com>
Change-Id: I2b32e81c0990ede00eea6d4ed3a7e44d534d4df3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8231
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This update will allow us to use spdk_nvme_detach_async() and
spdk_nvme_detach_poll_async() easier to aggregate multiple detachments.
Previously, we could do:
spdk_nvme_detach_async()
spdk_nvme_detach_async()
spdk_nvme_detach_async()
and then started doing spdk_nvme_detach_poll_async().
Hence aggregating multiple detachments is already supported.
After this patch, the following sequence is possible:
spdk_nvme_detach_async() = 0
spdk_nvme_detach_async() = 0
spdk_nvme_detach_async() = 0
spdk_nvme_detach_poll_async() = -EAGAIN
spdk_nvme_detach_async() = 0
spdk_nvme_detach_async() = 0
spdk_nvme_detach_poll_async() = -EAGAIN
spdk_nvme_detach_poll_async() = -EAGAIN
spdk_nvme_detach_poll_async() = -EAGAIN
spdk_nvme_detach_poll_async() = 0
The actual changes is to remove the variable polling_started from
struct spdk_nvme_detach_ctx because it is not necessary anymore.
Clarify this change via updating the header file and CHANGELOG.
Verify this change by unit test.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iebdf6c27c5304a2097b7084c315ccc99634ffa1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Add a new function spdk_nvme_detach_poll() to simplify a common
use case to continue polling until all detachments complete.
Then use the function for the common use case throughout.
Besides, usage by simple_copy application was not correct, and
fix it in this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic14711cd8478bf221c0fe375301e77b395b37f26
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The function comment was referring to a non-existent caller; instead, expand
with a little more detail on the path taken for new QPs.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I42478194f3cfc18a6ff6c434964630ac42866f1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
When the property is 8 bytes but the host only requested
4, we need to mask and only return the bytes requested
by the host. Wait to do the DEBUGLOG until after
that has happened.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8f476a47e9fd07bf652fd64f3b1c17d650374167
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8506
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Interpret bare --with-dpdk opt as user's request to find installed
(provided by the distro) DPDK's libs|include files and use them during
the build.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I9da99671b95af0121194b3a6d53636b0ded71f1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tom Nabarro <tom.nabarro@intel.com>
Reviewed-by: <tomasz.rochumski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Was using "dst" in some cases and "crc_dst" in others for crc32c
related calls. Update them to always use crc_dst
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icf200f1734c64c29881f23b02b8d12bad81b3ca0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8186
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Recent work identified race conditions having to do with the
dynamic flow control mechanism for the idxd engine. In order
to both address the issue and simplify the code a new scheme
is now in place. Essentially every DSA device will be allowed
to accomodate 8 channels and each channel will get a fixed 1/8
the number of work queue entries regardless of how many
channels there are. Assignment of channels to devices is round
robin and if/when no more channels can be accommodated the get
channel request will fail.
The performance tests also revealed another issue that was
masked before, it's a one-line so is in this patch for convenience.
In the idxd poller we limit the number of completions allowed
during one run to avoid the poller thread from starving other
threads since as operations complete on this thread they are
immediately replaced up to the limit for the channel.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I913e809a934b562feb495815a9b9c605d622285c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8171
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In spdk_idxd_configure_chan(), if memory allocation fails in
TAILQ_FOREACH() {} code range, we will goto err_user_comp and
err_user_desc tag, in which we donot free chan->completions
and confused batch->user_completions with chan->completions.
Memleak problem and double free problem may occurs.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: I0e588a35184d97cab0ea6b6c013ca8b3342f940a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8432
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
When a qpair is destroyed and the qpair is the last,
_nvmf_ctrlr_free_from_qpair() (in lib/nvmf/nvmf.c) sends two messages,
one is for _nvmf_ctrlr_destruct() and another is for
_nvmf_transport_qpair_fini().
We do not know which of two completes earlier.
_nvmf_ctrlr_destruct() frees the qpair->ctrlr in the end.
On the other hand, _nvmf_ctrlr_free_from_qpair() calls
spdk_nvmf_poll_group_remove() in the end, and spdk_nvmf_poll_group_remove()
accesses the qpair->ctrlr to free queued requests to the qpair.
Before one recent change, spdk_nvmf_poll_group_remove() had been called
before _nvmf_ctrlr_free_from_qpair() was called.
Hence extrace the operation to free queued requests from
spdk_nvmf_poll_group_remove() and inline it into _nvmf_qpair_destroy().
Fixes one showstopper error to investigate the issue reported in #1819.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I29c43ff7b289fc77a5de9c33e0266301c412e208
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Previously core load was only considered for main lcore.
Other cores were used based on cpumask only.
Once an active thread was placed on core it remained there
until idle. If _get_next_target_core() looped around,
the core might receive another active thread.
This patch makes the core load matter for placement of any thread.
As of this patch if no core can fit a thread it will remain there.
Later in the series least busy core will be used to balance
threads when every core is already busy.
Modified the functional test that depended on always selecting
consecutive core, even if 'current' one fit the bill.
Later in the series the round robin logic for core selection
is removed all together.
Fixed typo in test while here.
Note: _can_core_fit_thread() intentionally does not check
core->interrupt_mode and uses tsc. That flag is only updated
at the end of balancing right now. Meanwhile tsc is updated
one first thread moved to the core, so it is no longer
considered in interrupt mode.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I95f58c94e3f5ae8a468723d1dd6e53b0e417dcc3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8069
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Idle threads are always moved to main core, there are no
other considations. Doing it as separate first pass,
allows to have the core stats be up to date for second
pass for active threads.
Core load stats will be used later in the series to determine
optimal target core for an active thread.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6a9bc11b86e954e461f7badebf3a6e4d1718f63c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This will be needed when doing multiple passes over
all threads. See next patch.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4e9c749d69314fc268cbcb9334862392100b651e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
When picking a path to go down with a thread,
conditions unnecessarily piled up.
Instead do it either of two ways:
- move idle threads to main core
- find best core for active threads and move them there
There is no need to worry about cpumask of the thread,
since _find_optimal_core() will always return a core
within the cpumask.
If the found core is the same one as the current,
_move_thread() won't perform any action.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0f4782766c15c86b5db0c970cfc9547058845b2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Refactors logic for finding the optimal core for a thread
to single function.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc2b09acb6f698640ce9602fec4f567eb32b79fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6732
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Refactor all thread moves and core stats updates to single function.
At this time in series only tsc of main core was modified and
only idle tsc of main core was used. Main core would be either
the destination core or the source core. In both cases, the idle
time for main core had to be updated.
This patch generalizes this logic to always move the execution
time from source core to destination core.
As a byproduct cores besides main core have the stats updated,
which will be useful later in the series. Once core load will
be the deciding factor for choosing a core.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I57564e8b2632f919869d74e8f10b01fb3dda3be9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6658
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Now that the trace library can handle multiple arguments, there's no
point in passing 0 for tracepoints that don't have any arguments. This
patch removes all such instances. It allows us to to verify that
`spdk_trace_record()` was issued with the exact number of arguments as
specified in the definition of the tracepoint.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idbdb6f5111bd6175e145a12c1f0c095b62d744a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Replaced calls to `spdk_trace_record_tsc(spdk_get_ticks(), ...)` with
`spdk_trace_record(...)`, which does the same thing but is more consise.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib96e0bc0225490dadf857e1ddd2a3ecbf71e98c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Now that each tracepoint can have more than one argument, we cannot pad
the missing ones, as it would take too much space. Therefore, we put
them at the end of a line and simply skip the missing ones.
Additionally, since empty arguments are no longer padded, this patch
stops recording arguments with names consisting of an empty string
(containing just '\0').
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5199a3219a31d6afd3178324a4f48563b84e6149
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7958
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Now that `spdk_trace_record` receives variadic arguments, we no longer
have to pass strings as uint64_t, but can pass them directly as
pointers. That also means that the recorded strings can be longer than
8B (up to 40B).
This patch changes the blobfs code to pass the filenames as strings and
gets rid of the code that converted them to uint64_t.
Additionally, the maximum length of string arguments printed by
`app/trace/trace` has been extended to 16 and they're also padded to 16
characters, to better align with other argument types.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibe94452bf1b27eba2b15ca8608d0c3b55c2db360
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch allows tracepoint to record a variable number of arugments.
An additional function has been added,
`spdk_trace_register_description_ext()`, which allows the user to
register definitions for tracepoints specifying all the arugments that
they accept. Users can also call `spdk_trace_register_description()` to
register tpoints with a single argument (or none).
Currently, all of the tracepoint arguments need to be passed as
uint64_t.
The trace record functions use variable arguments and rely on tracepoint
description to know the order and the format of the arguments passed.
That means that the user needs to take care that they're always in sync.
Moreover, this patch extends the tracepoint entry size from 32B to 64B,
meaning that there are 40B that can be utilized for passing arguments,
which in turn means that there can be up to 5 arguments per tracepoint.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I9993eabb2663078052439320e6d2f6ae607a47ff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Move the definition of structure spdk_io_channel into
lib/thread/thread_internal.h, so we don't have to update SO_VER for
other libraries in future when we need to change the internal details on
the structure.
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Change-Id: I3d2ca7a8737972e0b33ce92e464da42c48f89dec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8189
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If we're not a DEBUG build, vfu_setup_log() was effectively forcing a
libvfio-user logging level of LOG_ERR. Instead, let the log handler decide what
to report, so we can respect the SPDK levels.
Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ib3ad62589f495a377885f7deabaf02b428e83d30
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Commit b7cc4dd added support multiple AERs, but didn't
remove the code that hardcodes aerl=0 for non-discovery
controllers. So even though the target now supports
multiple AERs, we never indicate that for non-discovery
controllers.
The spec also recommends that implementations support
a minimum of 4 AERs - so the current behavior is not
recommended.
It seems that at least on Windows (when testing with
vfio-user transport) we see the limit get exceeded
which results in ERRLOGs. Let's keep the ERRLOG there
for now, assuming that once we report we support 4
AERs that Windows won't try to send more than that.
Fixes issue #2000.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6a07a6f37aaa6e531ae2cf1e1c46da036b00785b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8488
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
We cannot control what the host may send to the target.
For example, we have empirical evidence that Windows
will send vendor-specific IDs for features and log pages
(when testing with the vfio-user target transport).
So let's change the ERRLOGs in these cases to DEBUGLOGs.
Fixes issues #2004, #2007, #2008.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8d5b92fc5e33d698af246f2f1c34f7cf51e6488a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
1. Update with latest vfio-user specification changes.
2. The new libvfio-user will not expose dma_sg_t data structure
any more, SPDK should use pointer and allocate memory for it.
Change-Id: I619b0c0828cbe3b050c628bff4c4ce7ee840510f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8377
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
When creating queue pairs, the original code uses a stack
queue variable and copy it to queue pair in insert_queue
function, the coming changes in libvfio-user doesn't expose
dma_sg_t data structure any more, we need to change it to
a pointer and allocate memory for it, so here we eliminate
insert_queue function as a preparation.
Change-Id: Iee94029d24bc8882ec169665e229e6cbc11564c0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8376
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Thread is private data of spdk_io_channel, bdev should use
spdk_io_channel_get_thread() to access it. This prepares for the upcoming
change to make the definition of struct spdk_io_channel private.
Change-Id: I643c8d677e22f6d8dde2faf91bb2711d3f5d81b8
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
in-capsule data over 4KiB when using the Linux initiator.
This is fixed in the latest kernel. See
https://lists.infradead.org/pipermail/linux-nvme/2021-May/025641.htmlFixes#1823
Change-Id: Ie383ea774ee31ef8fe255119095b21603483c33f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
In blob_load_cpl(), spdk_realloc() is called to realloc
memory of ctx->pages. If spdk_realloc() return NULL,
the ctx->pages is set to NULL without being freed,
and then a memleak problem occurs.
Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
Change-Id: Idf21b690e89beab0245ba57a5de66a4f506d54fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8308
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
If the transport returns error when polling for
completions, it gets to a uint32_t and we end up
trying to resubmit all of the requests that are
currently queued. But that's not correct - if
the transport returns an error we shouldn't be
trying to resubmit requests at all.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9198e3e2d71875cc1e46e0ac928338bb983487f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8395
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
nvmf_ctrlr_get_log_page used req->data to store the log page result.
While the req->data only contains the first iov, if req->iovcnt is
larger than 1, the req->data may not hold the complete log page; and
even worse, the log page result may be written to invalid address and
cause memory corruption.
Change-Id: Ie6415a6bd2327419fe4b32f21ac814fd827c9e95
Signed-off-by: Jiewei Ke <jiewei@smartx.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7970
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
