Commit Graph

12773 Commits

Author SHA1 Message Date
Shuhei Matsumoto
040ee27c16 nvmf/tcp: Wait until request is abortable if it is transferring
If the state of the request is TRANSFERRING_HOST_TO_CONTROLLER,
we cannot abort it now but may be able to abort it when its state
is EXECUTING. Hence wait until its state is EXECUTING, and then
retry aborting.

The following patch will make the timeout value configurable as
an new transport option.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I98347b68e8b6b4a804c47894964cb81eae215aaa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3010
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-16 07:29:46 +00:00
Sven Breuner
0190e71eb6 module/bdev_compress: Fix use logical block size in vbdev_compress_claim
Use logical block size instead of base bdev block size in
vbdev_compress_claim. This is relevant to enable the compress bdev to
present a logical block size that is different from the block size of
the base bdev (e.g. 512B logical on 4KB base bdev).
However, this is just an internal fix/preparation and there is
currently not yet a way for users to specify a different
logical block size, e.g. through an rpc.py parameter.

Signed-off-by: Sven Breuner <sven@excelero.com>
Change-Id: Id7562f0da234ce11242b7ee78837a317ee51c211
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-15 07:09:40 +00:00
Ziye Yang
9a6a561149 lib/iscsi: Change the order of executing iscsi_clear_all_transfer_task
In my mind, we should firstly remove the sock from the group, then
all the call_cb of pdus will be handled, then we clear the task,
we can avoid the duplicated resource recycling.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ic5fad130e58c9dcb42a237973979b042eabdf43a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3309
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-15 07:09:24 +00:00
Seth Howell
8e65bfc7e4 lib/log: add a check for localtime() failure
localtime() can technically fail and return a NULL
pointer. We should handle this case.

Change-Id: I5f9d11c5f992453ec3e3804bd17f5d05a863d526
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3245
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-15 07:09:19 +00:00
Maciej Wawryk
c7d1abba18 test/vm_setup: Add clearlinux package manager
Also remove unnecessary jq from pkgdep

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I259e8f00a4a6e9c5463fa9e8bdefb19f9bdaa93d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-14 07:55:19 +00:00
Maciej Wawryk
3c8451e69a scripts/perf: Add bandwidth measurement to nvmf test
Bandwidth measurement is realized by bwm-ng tool.
Using this measure tool will help us to check network saturation.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7b95b9b4ac3ca3b1847ec72ab7ea333807befe89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3190
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-14 07:55:08 +00:00
Shuhei Matsumoto
72eedc578d nvmf/tcp: Abort request whose CID matches if it is pending
If the request is queued and is not in completing, we can abort
it safely.

If the state of the request is NEED_BUFFERING, the request is
queued to both tqpair->group->group.pending_buf_queue and
the queue per state.

If the state is AWAITING_R2T_ACK, the request is queued to the
queue per state.

Dequeueing from the queue per state is done in
nvmf_tcp_req_set_state(). Hence explicit dequeuing only when the
state of the request is NEED_BUFFERING.

Most abort operation is common between two cases. We can use fallthrough
in switch-case but factor out the common operation into a helper
function nvmf_tcp_req_set_abort_status() instead because we may use
the helper function in future and using helper function is easier to
read than fallthrough.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1695b084d5d1f2537fbdd512bc3cd136e0f6a65b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3009
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
b8a87e6af5 nvmf/tcp: Abort request whose CID matches if it is outstanding
Call nvmf_ctrlr_abort_request() if the request whose CID matches
is found and its state is executing.

nvmf_tcp_qpair_abort_request() returns immediately if rc is
SPDK_NVMF_REQUEST_EXEC_STATUS_ASYNCHRONOUS or calls
spdk_nvmf_request_complete() otherwise.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1abceecc211ee79d8ac18a82dc63b13d313a6f27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3008
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
a7d59c9d75 lib/nvmf: Add three variables to spdk_nvmf_request to avoid temp context for abort operation
Add three variables, poller, timeout_tsc, and req_to_abort to
struct spdk_nvmf_request to wait until the request to abort is
abortable without using any dynamically allocated context.

struct spdk_nvmf_request is already very large (968 bytes), and
the pahole tool checked these variables did not create any extra hole.

Poller pointer and timeout value are usable for other potential
cases, and so use generic names.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7202ade181365ab586d8c30383b3ea0fef82dcff
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3260
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
604b4503c4 lib/nvmf: Add nvmf_transport_qpair_abort_request()
State machine is different among NVMe-oF transports and is
encapsulated to the transport neutral NVMe-oF controller and
NVMe-oF qpair.

To implement abort operation for each NVMe-oF transport,
add a function pointer qpair_abort_request to struct spdk_nvmf_transport_ops
and a stub nvmf_transport_qpair_abort_request() to encapsulate
which transport is used.

The following patches will implement qpair_abort_request for each
transport. Each qpair_abort_request() is responsible to call
spdk_nvmf_request_complete() for the abort request.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2beac959ed428c5108cf33691226b7fae5cd24d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3007
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
a8beb79ea3 lib/nvmf: Add nvmf_ctrlr_abort_request() to call nvmf_bdev_ctrlr_abort_cmd() from transport
If nvmf_ctrlr_abort_request() returns SPDK_NVMF_REQUEST_EXEC_STATUS_COMPLETE,
the caller is responsible to call spdk_nvmf_request_complete().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6ed5f8611bcacd20385dd925b9b2e29ed2200269
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3006
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
91b80fe287 lib/nvmf: Add nvmf_bdev_ctrlr_abort_cmd() to call spdk_bdev_abort()
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idc9957aabe0c3c24d2d36f1ac84b847b7455e7d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2959
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
f045d924fc lib/nvmf: Factor out abort operation on the specific qpair into a function
Factor out abort operation on the specific qpair into a helper
function nvmf_qpair_abort_request().

After this refactoring, nvmf_ctrlr_abort_done() calls
_nvmf_request_complete() only if the passed status is zero.
If the passed status is not zero, nvmf_qpair_abort() is responsible
for calling _nvmf_request_complete() instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4828c0e21cc7650210675661d6e1c0fd54c7a2cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2991
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
9975d4a1d2 lib/nvmf: Always return success and only toggle bit 0 of CDW0 for abort command
Description is not clear but according to the NVMe specification,
always set the completion status to success and differentiate only
the bit 0 of CDW0 between success and failure for abort command.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0195e72fe1d7fcc2592f47e9dcf92ac56912282c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1965
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
47b0d4275c bdev/nvme: Support abort IO by using spdk_nvme_ctrlr_cmd_abort_ext()
Change NVMe bdev module to enable abort as IO type.

Change _bdev_nvme_submit_request() to process abort request when the
IO type is abort.

The current thread tries aborting I/O command in the I/O qpair first.
If no I/O command to abort was found, send message to the thread which
is registered when creating controller. The controller thread tries
aborting admin command in the admin qpair next. If no admin command
to abort was found, complete the abort request with failure.

spdk_nvme_ctrlr_cmd_abort_ext() is used to try aborting command whose
cb_arg matches. qpair is set to NULL when trying to abort admin command.

Before calling spdk_nvme_ctrlr_cmd_abort_ext(), save the current
thread to process admin command completion correctly.

spdk_bdev_abort() supports any bdev module other than NVMe bdev
module and does not check CDW0 but checks only if the completion
status is success or failure. So add bdev_nvme_abort_done() and
converts the NVMe completion status to the bdev completion status.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If6aebae0ba2f6c5834ee926e161af9c4d825f341
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
613c9386ff bdev/nvme: Add thread pointer to struct nvme_bdev_ctrlr
Add thread pointer to struct nvme_bdev_ctrlr. The thread which
created nvme_bdev_ctrlr is set to the pointer.

The thread pointer will be used to limit only one thread to submit
admin abort.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia39d5cbc7a13b0e0022c0d5591069ea8776ef774
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3244
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
9d6bb6decc bdev/nvme: Use SPDK_POLLER_REGISTER to use function name as poller name
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I30da42413501d03433ef4550327675471185df3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3243
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
71da42e953 lib/nvme: Add spdk_nvme_ctrlr_cmd_abort_ext() to abort commands whose cb_arg matches
A new API spdk_nvme_ctrlr_cmd_abort_ext() gets cmd_cb_arg as a
parameter, and use it to abort requests whose cb_arg matches cmd_cb_arg.

The caller can set the parameter qpair to NULL if it wants to abort
requests on admin qpair.

Hold ctrlr->ctrlr_lock throughout because abort request is submitted
to admin qpair.

The API is not critical for performance, and so initialize parent
data first.

The API is for a specific qpair, and so hold SQID in the parent,
and it is copied to the children. On the other hand, CID is set
to child when request to abort is found.

Use an new helper function nvme_transport_qpair_iterate_requests()
to add abort request for each outstanding request which has
cmd_cb_arg as its callback context.

The case is possible such that the request to abort is not outstanding
yet but queued. Hence abort queued requests which has cmd_cb_arg
as its callback context too, but it is done only if there is no error
so far.

If only queued requests are aborted and there is no outstanding
request to abort, complete with success synchronously.

If there is no outstanding or queued request to abort, return -ENOENT.

When any abort request is submitted, the difference between success
and failure is only bit 0 of CDW0 according to the NVMe specification.

We cannot the existing helper functions nvme_request_add_child() and
nvme_cb_complete_child() but can use nvme_request_remove_child().
nvme_qpair_submit_request() may use only nvme_request_remove_child()
from these three helper functions. Hence we use req->parent as other
types of request do.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3a271c6542f8e2e6b425b3bf6151f41e924bc200
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2039
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
beff519ea9 lib/nvme: Add a macro spdk_nvme_cpl_is_abort_success to check if abort succeeded
Add an new macro spdk_nvme_cpl_is_abort_success(). This will be used
in NVMe driver and NVMe bdev module later.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8348f54ed8c9fd0238661f7d7e61e9e6f36f33c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2800
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:40:42 +00:00
Shuhei Matsumoto
576a373d58 lib/nvme: Abort queued requests whose cb_arg matches
Use another list dedicated to hold queued requests being aborted
to avoid potential infinite recursive calls.

Add a helper function nvme_qpair_abort_queued_req() to move requests
whose cb_arg matches from qpair->queued_req to qpair->aborted_queued_req.
Then nvme_qpair_resubmit_requests() aborts all requests in
qpair->aborted_queued_req.

The first idea was that nvme_qpair_abort_queued_req() aborts queued
requests directly. However, this caused infinite recursive calls.
Hence separate requesting abort to queued requests and actually
aborting queued requests.

The detail of the infinite recursive calls is as follows:

Some SPDK tool submits the next request from the callback to the completion
of a request in the completion polling loop. For such tool, if the callback
submits a request and then aborts the request immediately, and the request
could not be submitted but queued, it will create infinite recursive calls
by request submit and abort, and it will not be able to get out of
completion polling loop.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8196182b981bc52dee2074d7642498a5d6ef97d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2891
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:40:42 +00:00
Alexey Marchuk
e137881e4e nvme/tcp: Insert free req at the head of the list
lifo model is more cache friendly

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Id937ab0c1b8b4ce121136144c7d6013bbe5eb963
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3282
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-13 08:40:31 +00:00
Evgeniy Kochetov
35429c9b5d sock: Save socket subsystem configuration in JSON format
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I32c25e6410c418ffa00a76559aa7b6999e2269ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-13 08:40:15 +00:00
Karol Latecki
140a772746 scripts/vagrant: use IDE bus for Arch Linux VMs
The original image was prepared using SCSI bus.
Using IDE explicitly speeds up boot process as the
system does not have to look where to boot from.

Change-Id: Ia9f649d1c46c591135833c02ed60b3c960ad8b98
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2840
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-13 08:40:05 +00:00
Karol Latecki
778312c8d2 scripts/pkgdep.sh: install libffi for Arch Linux
Arch Linux instance is failing on that CI. Seems that
libffi is a dependency for installing configshell_fb
and it's missing.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I988f2fb5e879b9e8882409c9c2772d5bd2c5e045
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2020-07-13 08:40:05 +00:00
Vitaliy Mysak
367c980b45 lib/conf: accept entries of form key=value
In INI config, split key based on whitespace AND symbol '='
This allows to have same format as FIO configs

FIO style config will be used for bdevperf config file

Change-Id: I92c501c839842919afb5c23ed935be010ae2b168
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3151
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-13 08:39:57 +00:00
Shuhei Matsumoto
ee7f961655 lib/iscsi: Unify outstanding_r2t_tasks array and active_r2t_tasks list into the latter
We have no particular requirement to keep both conn->outstanding_r2t_tasks
array and conn->active_r2t_tasks list now.

To improve readability and maintaineability, unify two into the latter,
conn->outstanding_r2t_tasks list. Update unit test accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I25cf7cffbe39ac66e102eb3052340de6ef65c8f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3115
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:39:50 +00:00
Shuhei Matsumoto
d5f3f48b60 lib/iscsi: Fix iscsi_del_transfer_task deletes from both array and tailq
Previously iscsi_del_transfer_task() dequeued the task only from
the array conn->outstanding_r2t_tasks[].

process_non_read_task_completion() had dequeued the task from
the tailq conn->active_r2t_tasks then.

However abort_transfer_task_in_task_mgmt_resp had not dequeued the
task from the tailq conn->active_r2t_tasks then.

This was an apparent bug, and is fixed here. Update unit tests
accordingly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I93f02b2fb670dcee4c32d61c264e3ad5b4f9f43e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3108
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:39:50 +00:00
Shuhei Matsumoto
41f59559e1 lib/iscsi: Return if conn->state is already exited when adding conn to poll group
Previously, we fixed a similar bug that iSCSI target got seg. fault
if connection is being exited between spdk_iscsi_conn_write_pdu()
and its callback iscsi_conn_login_pdu_success_complete() are executed.

This fix was not sufficient and we still saw similar error.

Not only socket write but also SPDK message is asynchronous and
connection may start exiting between iscsi_conn_schedule() and
iscsi_conn_full_feature_migrate().

This patch fix this new issue.

The previous fix was
https://review.spdk.io/gerrit/c/spdk/spdk/+/1275

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5cc422529a335e5f1982693bdf910ac7debe6f17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3074
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-13 08:39:50 +00:00
Shuhei Matsumoto
66aec44a4b lib/iscsi: Manage active connections by TAILQ
Following the last patch, add g_active_conns to manage active connections
by TAILQ. We cannot remove the is_valid bit and g_conns_array
to support iSCSI top application, but this change simplifies the
code a little.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8b449f3056ffaed19f23f42d83c912dfba9a7b75
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3090
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-13 08:39:50 +00:00
Shuhei Matsumoto
89ae0e3c62 lib/iscsi: Manage free connections not by array but by TAILQ
Previously free connections had been managed by g_conns_array,
and allocate_conn() gets the lowest free connection. This had worked
almost as LIFO, and the just freed connection had been reused
immediately to the new connection.

Using TAILQ makes management of free connections FIFO, and this will
be more intuitive and simpler, and avoid potential issues due to the
fact that we do not know the state INVALID is the current connection
or the current connection is exited and the new connection is allocated.

This patch includes following updates.

Remove the test condition that the connection ID should be zero.
Connection ID is used as Target Transfer Tag (TTT) and TTT is opaque
number. Hence requiring connection ID to be zero is not meaningful.

iscsi_conn_free() calls free_conn() while holding g_conns_mutex, but
iscsi_conn_construct() does not call free_conn() without holding
g_conns_mutex. Hence add g_conns_mutex to the latter.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I204f66469f0bf54845c773da5b4ac86f3c8dca60
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-13 08:39:50 +00:00
Shuhei Matsumoto
e685db2caf lib/iscsi: Rename link pointer to pg->connections from link to pg_link
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I50902ebf0ac1ee103d2bf999578e68637efcc123
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3088
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-13 08:39:50 +00:00
paul luse
fc250841ca idxd: add batch capability to accel framework and IDXD back-end
This patch only includes the basic framework for batching and the
ability to batch one type of command, copy. Follow-on patches will
add the ability to batch other commands and include an example of
how to do so via the accel perf tool.  SW engine support for batching
will also come in a future patch. Documentation will also be coming.

Batching allows the application to submit a list of independent
descriptors to DSA with one single "batch" descriptor. This is beneficial
when the application is in a position to have several operations ready
at once; batching saves the overhead of submitting each one separately.

The way batching works in SPDK is as follows:

1) The app gets a handle to a new batch with spdk_accel_batch_create()
2) The app uses that handle to prepare a command to be included in the
batch. For copy the command is spdk_accel_batch_prep_copy(). The
app many continue to prep commands for the batch up to the max via
calling spdk_accel_batch_get_max()
3) The app then submits the batch with spdk_accel_batch_submit()
4) The callback provided for each command in the batch will be called as
they complete, the callback provided to the batch submit itself will be
called then the entire batch is done.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4102e9291fe59a245cedde6888f42a923b6dbafd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2248
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:31:09 +00:00
paul luse
cf83cac493 module/accel/idxd: change name of global lock and expand scope
Previous lock protected against a channel counter, renamed as it
now protects multiple values associated with configuration and
reconfiguration.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icfe48aa7df192a7bdf568de418d49c7984d20acf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3175
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-10 07:31:09 +00:00
Jacek Kalwas
ed4b7d12ec nvmf: suppress scan-build warning
Logic error     Dereference of null pointer     ctrlr.c
nvmf_ctrlr_async_event_request  1522

Dereference of null pointer is not possible if sgroup obtained using
ctrlr obj. Adding corresponding asserts suppresses the warning.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I78b32fadd5449ee9b533f65193c70e55cf9a8f1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3251
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
4d9ab1e9a1 nvme: pretty print dptr
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I576878fbbafc3d17617ceeec99e40565be7d5d3d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3213
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
64f05eb5c5 nvme: pretty print fabric cmd
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ib4bc28026cab208d45c8b876714fa525e5bb38f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3200
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
9cd4723913 nvme: pretty print set/get features
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ib6f1811da9a6294983bce04cff01ba1fb5e45607
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3179
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
41b360d54e nvme: add missing cmds to admin opc string
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I6cdcf675ebc8ad31d88b5469f87e1eae066b2e3c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3178
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:30:59 +00:00
Jacek Kalwas
61668cc43e nvme: introduce new set of cmd/cpl printers
Having functions without qpair on the interface allows for wider usage
e.g. by nvmf layer.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I3a51ad53f00eb29e2ba2681ef4ff0cc2a197b65d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
2020-07-10 07:30:59 +00:00
Ziye Yang
f096f252d0 uring/sock: remove the SPDK_UNREACHABLE
For some exceptional cases (e.g.,
https://github.com/spdk/spdk/issues/1486),
we may detect POLLERR or other events. So for those events,
we can just ingore it, but not use SPDK_UNREACHABLE.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I073575408783ff75e50b40d45ddf09388a2cab96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3262
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-10 07:30:45 +00:00
Ziye Yang
08b9603531 scripts/pkgdep: Install liburing lib in /usr/lib64.
In some OSes (e.g., Fedora30), /usr/lib is not one of the default
library loading paths. So Let's change it into /usr/lib64.

To address: #1471

Change-Id: I89a66b6096fc1096be7f7e7bb6414f0998b3d974
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3261
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-10 07:30:39 +00:00
Jacek Kalwas
0555074965 event: expose base virtaddr as an option
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I062ac2b1de5a89d521bde926433139457f9168f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-09 07:23:40 +00:00
GangCao
249f95daeb BlobFS: fix the case where configuration field is not set
In the case that the configuration field is not set, it will
return "-1" and then implicitly convert to the unsigned
case where the variable is wrongly set.

Change-Id: I7b2c64d653badd731d8e6df46629231343a0ae6d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3236
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-09 07:23:34 +00:00
Ziye Yang
4c9aad0299 nvme/tcp: Report the free entries if sending_ack is set
Previous we fix the same issue in this commit:
cb98b2ab3e

But we forget to fix it here. And we also need to update here,
otherwise we will still face the same issue described in
commit: cb98b2ab3e

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I3660dbb6e97c92ea4cb347cfce4bf23c6dfe97ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3242
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-09 07:23:19 +00:00
Michal Berger
0bec3c3c64 pkgdep/git: Don't fail install_qat() if qat_c62x is not loaded
Change-Id: I3784e3f7983a7f66ea68b2bd8b97f98b282e3405
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3201
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-08 08:11:20 +00:00
Michal Berger
f0ba1e9e44 pkgdep/git: Remove vpp
VPP is going to be deprecated in 20.07, thus remove support for it
from vm_setup.sh.

Change-Id: Ic5891178bc439ff08475adbb2925b81c4a33525f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3199
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-08 08:11:00 +00:00
Seth Howell
f4d0ab7111 Revert "bdev/nvme: add a function for specifying a multipath trid."
This reverts commit 5f13e0f67f.

Reverting this change to allow for a secondary review.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iad208603034bd892f3f248836459005d3c2b5297
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3041
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:11 +00:00
Seth Howell
0344ba4064 Revert "bdev/nvme: add failover option to bdev_nvme_reset"
This reverts commit 62ded32fd7.

Reverting this change to allow for a secondary review.

Change-Id: I85a059c434f6e634e014e680ae228e95fffdee92
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3040
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:11 +00:00
Seth Howell
3b830202ca lib/nvmf: add an acceptor_backlog transport opt.
This allows users to configure the number of
connection requests outstanding to an rdma port
at once.

RPC included.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8a2bb86b2fb7565cb10288088d39af763b778703
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3097
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:04 +00:00
Evgeniy Kochetov
37bf49b1d5 sock: Add dependency on socket subsystem to other subsystems
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I665760166d54f6156b54735015bd2b586e91c345
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/616
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-08 07:56:46 +00:00