Commit Graph

10494 Commits

Author SHA1 Message Date
GangCao
a31ae23f38 test/bdevperf: add the QoS testing
Add a required check that the reasonable IOPS limit is quarter
of the measured actual IOPS without QoS. As I/O size is set
to 4K, so that bandwidth limit is also able to meet once the
IOPS limit is achieved.

Change-Id: Id9a401ce59f97245c505ebe0d1f4c18049177132
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-18 13:15:45 +00:00
Maciej Wawryk
809a6f82de scripts/vagrant: Allow running multiple VMs in parallel
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7047c3152558911a782be86e5ccf8214f2418c02
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:15:17 +00:00
Maciej Wawryk
ff231506d3 scripts/vagrant: Add more NVMe emulation options
Current scripts/vagrant allow us to create only a single file for
emulating NVMe with single namespace.
This patch allow to create multiple emulated drives with multiple namespaces.
This patch also allows to emulate OCSSD disks which are used in OCF tests.

Change-Id: Ia7a4667bd405ba18235e72bcdf608270002b5242
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:15:17 +00:00
Richael Zhuang
8cad9cc10a spdk: enable isa-l by default aarch64
In addition to x86, enable isa-l by default on aarch64.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I2db3f248bdc857326ec9b6f4b040a95d2a2ca1ca
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:14:41 +00:00
Seth Howell
c4d4800a46 test/nvmf: add a test for bad discovery ctrlr.
Make sure we don't crash or do any other bad memory accesses when we
fail to connect to a discovery controller.

Change-Id: I7c3e25899bc78f4beff9ad017612bacf286d52c7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474153
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:06:49 +00:00
Seth Howell
08d5d8474b test: fix SC2103 errors on older shellcheck.
Shellcheck version 0.6.0 reports all instances where we cd .. or cd - as
instances of SC2103. The version we are using on the test pool must be
newer, because we don't get the same errors, but for people running
check_format locally, it can make it difficult to parse the output.

Change-Id: I09f81a83c6f37480f13c36eb622e500364a1c437
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2019-11-18 13:06:49 +00:00
Ziye Yang
2df770535a test/ceph: fix the dev name in start and stop.sh
I am confused with the dev_backend variable, I think that
we may not need this variable.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I26fa5e8a829e574bb1b97add753b21ad1b0fc23e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:01:58 +00:00
Jin Yu
301c5aeec9 pci: fix the hotplug issue
DPDK intr thread is designed that it can't unregister the src
callback in this callback handler. So I think we can't detach
the PCI device in the hotremove callback as it needs to unregister
the VFIO notification callback which will be not successful
but it still can free the device. So at the next req notification
in the handler function, we meet the freed device.

Fix #994

Change-Id: Id4b45a2d0fe6b45b132355d59471bc80240fad70
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-18 12:54:01 +00:00
Maciej Szwed
c330fbe618 bdev/nvme: Correct variable name
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: If7d55636eed5e03d002ca6ea6a24d96b127ae705

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471621
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-18 12:53:17 +00:00
Maciej Szwed
c30609c9ea bdev/nvme: Allow multiple bdevs per namespace
Some future NVMe namespace types will be able to
create multiple bdevs per one namespace. This
patch makes it possible.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I47b4c1fc545c59dcc3171ab0960f1835b6aa6d24
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471620
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 12:53:17 +00:00
Maciej Szwed
77dcf4534b bdev/nvme: Use Namespace object for tracking bdevs
In future implementations it will be possible to create
different types of namespaces (standard, OCSSD, ...).
This patch introduces new nvme_namespace type which
will make possible to implement such cases.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I27747d3985915f45c0e0a28dd5f391cca06b13cc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471273
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 12:53:17 +00:00
Shuhei Matsumoto
c9275afb62 ut/iscsi: Add out or order completion of read split
Out-of-order case didn't have UT code. Besides, remove unnecessary
if check in the in-order case together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic3c09de73f57033a1db9522519347d7d201e38be
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-11-18 12:51:38 +00:00
Shuhei Matsumoto
83271409c0 lib/iscsi: Simplify DIF insert/strip by doing at each PDU type handler
We have separated PDU header handler and payload handler, and have
PDU header handlers for each PDU type now.

By using this refinement, we can remove an aggregated helper function
spdk_iscsi_get_dif_ctx() and embed spdk_scsi_lun_get_dif_ctx() into
each PDU header handler.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib4d9939b625858466224647c545cb67a04babf86
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471699
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-11-18 12:51:38 +00:00
Shuhei Matsumoto
84482f4420 ut/iscsi: Unit test for PDU header handling
Add unit tests for PDU header handling except for SNACK Request PDU
because the caes that Error Recovery Level is not matured yet.

The purpose is mainly for increasing coverage, and most were reviewed
separately, and so merge multiple PDU types into a single patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ice8c522bee0fc8c0dadb0c6326be9f65d323d52e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471955
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-18 12:51:38 +00:00
Shuhei Matsumoto
3c4dfbd339 lib/iscsi: Assert if conn->data_in_cnt goes negative or is left positive
During testing, we observed both conn->data_in_cnt went negative or was
left positive unexpectedly.  Hence add assert to detect both cases.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I102d4eb7c8beb0e56b6a46fd0f85b3eb1c447da5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-18 12:51:38 +00:00
Shuhei Matsumoto
39bdb45ed8 lib/iscsi: Decrement conn->data_in_cnt of primary correctly for LUN hotplug
We had not decremented conn->data_in_cnt when the primary is removed
from conn->queued_datain_tasks after submitting it.

If we simply add decrement into iscsi_conn_free_tasks() and
_iscsi_conn_remove_lun(), it conflicts with iscsi_transfer_in().

By recent refinements, primary is freed in either spdk_iscsi_conn_free_pdu()
or iscsi_conn_free_tasks()/_iscsi_conn_remove_lun().

Hence let's make decrement of conn->data_in_cnt for primary follow
the management of primary.

In iscsi_conn_free_tasks()/_iscsi_conn_remove_lun(), if
primary->current_datain_offset, conn->data_in_cnt is incremented, and
hence decrement it.

In spdk_iscsi_conn_free_pdu(), if primary and all subtasks are
completed, decrement conn->data_in_cnt.

This patch will fix the issue that conn->data_in_cnt ls still
positive even after all tasks are freed when removing LUN dynamically.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I70cb431ab968387749ff7a5c77cd109904687797
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474436
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-11-18 12:51:38 +00:00
Shuhei Matsumoto
0110aad718 lib/iscsi: Re-check primary task in SCSI Data-Out PDU payload handler
We had checked LUN again but we had not checked primary task
in iscsi_pdu_payload_op_data().  This had caused unexpected behavior
during LUN hotplug.  Hence we check if primary task exists again
in iscsi_pdu_payload_op_data(), and abort the subtask immediately
if not.

This change fixes one of the failures we observed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5315badf0b90902e77dd5270dd0eda1437a771da
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-18 12:51:38 +00:00
Shuhei Matsumoto
6e4e85dcef lib/iscsi: Replace conn_free_pdu() by task_put() and put_pdu() in remove_acked_pdu()
Due to recent refinement of primary/secondary task management,
remove_acked_pdu() cannot use spdk_iscsi_conn_free_pdu(). As done
in iscsi_conn_free_tasks(), we can replace spdk_iscsi_conn_free_pdu()
by spdk_iscsi_task_put() and spdk_put_pdu() and we do that in this
patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9f83569becfc6e9440fb859709f04b6123674f25
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-18 12:51:38 +00:00
Mateusz Kozlowski
cabbe1b179 bdev/zone: Register/unregister zoned bdev
Add registration and unregistration of block zoned bdev. Attach it to the
underlying bdev during creation and unattach at deletion.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I773aff6c7609952f28c02dd1794f0529a781b2e1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468033
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 20:27:14 +00:00
Mateusz Kozlowski
76be5ff6f0 util: Promote shifted value in spdk_align64pow2 to uint64
Fix asan/valgrind error when using spdk_align64pow2 with >=32b values.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I326a32a01a5b8b5ee9b10693900a60c50ee8f900
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471058
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-11-15 20:27:14 +00:00
Mateusz Kozlowski
195be29e9b bdev/zone: Rpc functions for zoned block bdev
Added rpc handling for creation and deletion of zoned block bdev

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ib7b6f3928511aa840cbe098472682c951e92e5ed
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468031
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 20:27:14 +00:00
Mateusz Kozlowski
76fb903a74 bdev/zone: Create and delete zoned block device vdev
Adding new bdev module - zoned block device virtual bdev. It should be
possible to build on top of a regular (ie. non-OCSSD) bdev and surface
a zoned device API instead.

Added create and deletion functions for this bdev.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ia383483007117d1c826298fd391467a51fa2fe4e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468030
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-15 20:27:14 +00:00
Seth Howell
6b87dd8023 nvme_rdma: Detect stale connection failures.
This is the first step in properly reconnecting after a hard power off
event.

Change-Id: I9739bffacd66ec6d9f8f1d376bf42291c84f90f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473061
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 20:26:46 +00:00
Seth Howell
208fbb675c nvme_rdma: more cm_event validation to a helper function.
This step is going to become more involved, so it's best to keep it in a
separate function entirely.

Change-Id: Iefa9860420edf28e858c4ed8aa932985c686cfd9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473060
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 20:26:46 +00:00
Seth Howell
b35dfa2bd2 nvme/perf: Make it possible to disable keep alive.
There was an assignment in perf that prevented the user from actually
disabling keep_alive. If the user specified 0, it would get overridden
by the default opts value. So set the global in perf to a reasonable
value, and always use that value when setting the keep alive timeout.

Change-Id: I2eba14525613d679d7def29cc9ed312851cae156
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 20:26:46 +00:00
Ben Walker
af7be0acec iscsi: No longer set sndbuf size
Use whatever size the socket layer thinks is best. Before,
we limited the total amount of memory to just 32MB total. Now,
let the socket layer decide. It will likely use up to 2MB per
socket, which results in much better performance.

Change-Id: I9ef7680773b8c78a743fe74d8abb518258e19a0d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-15 20:25:15 +00:00
Ziye Yang
4579a16f30 lib/nvmf: Add a new state to wait for the req slot
Also need to update the spdk_nvmf_tcp_poll_group_poll.
Since if the tqpair recv state in wait_for_req,
we may already received the data, and there could be
not epoll event.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I9c5a202e47e57aaba63da143f954a20c135a98ae
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473626
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-15 20:25:15 +00:00
Seth Howell
bdeb41a3cd bdevperf: add force completions mode.
When we are using bdevperf for some sort of reset test, we need to make
sure that it doesn't cause failed requests to stop the test. We already
have the g_reset flag, but we can't use that for tests where we only
want to simulate a target disconnect event.

Change-Id: I0cbf495708a1e6680682b099e8c17862ec6ff3e2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-15 18:40:37 +00:00
Seth Howell
6d62ef1e81 test/nvmf: skip target disconnect on RXE.
There is an intermittent failure on the CI involving RXE and the target
disconnect test. We believe it is an internal issue in RXE. So for the
time being, disable that test when using RXE.

Fixes issue #1043

Change-Id: I5cc11e51c398e132a09baf3a26e48aa4ae7370a3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474279
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-15 18:39:29 +00:00
Seth Howell
f6646fd9fa nvme/tcp: detect cq errors.
We should alert the upper layer when the qpair becomes unusable due to
qpair errors.

Change-Id: Icdee3b55a14441a60111f3bd7a44dceef93bbb09
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474095
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-15 18:27:27 +00:00
Ben Walker
ac03d9206f nvmf: Use correct error code on bad reservation report command
The NVMe-oF target requires 128-bit identifiers for reservations,
so the extended report format must be used. If the user issues
a reservation report command without the extended format bit set,
the specification says to fail it with the HOSTID_INCONSISTENT_FORMAT
error code.

Change-Id: I2382af4f69167322d8e2c3f06cf8d9042830a70c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-11-15 18:26:54 +00:00
Alexey Marchuk
4c42b933c9 app: Add cli options to print SPDK version
Change verbose mode parameter of fuzz test app from -v to -V to avoid
collisions with app framework parameters

Change-Id: Ibe562fa82408366aa487a19fad9a99e1e692b48c
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474005
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-15 18:25:31 +00:00
Wojciech Malikowski
6c3f93c7fb lib/vmd: Documentation and user guide
Add VMD documentation to driver modules section.

Change-Id: I806058d897586780b8be73f90df3d0df152f6a84
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473526
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-15 18:21:23 +00:00
Shuhei Matsumoto
dc6d89b925 lib/scsi: Really wait for only outstanding tasks for LUN hotplug
One previous patch refined LUN hotplug process and updated the
comment but we still had checked not only outstanding tasks but
also pending tasks to be completed or aborted.  But, as written
in the comment, we can wait for only outstanding tasks now.

Management task is the highest priority and is pending only when
there is any outstanding management task, and the completion
callback of management task executes the first pending management
task.

The last patch changed us to abort all pending management tasks
after stopping new submission.

Hence we can do this change not only for IO task but also for
management task.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I66056f2a02af05d5bccaf6462c6f48c608cd0ca3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474032
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
8287ae7814 lib/scsi: Abort management tasks being queued when removing LUN
SCSI management task is for SCSI tasks which is submitted after it,
and all such SCSI tasks are aborted by the last patch when we
remove LUN dynamically. Hence we can abort all SCSI management tasks
being queued when removing LUN.

Add simple unit tests too.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9be3f910ab4bbb99cd399f71dc716a7c40f34fe5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
db91cfe045 lib/scsi: Stop submitting new task and abort pending tasks for LUN hotplug
Previously iSCSI LUN hotplug had critical bugs and we had no choice but
accept submitting new tasks as late as possible. We fixed the bug now and
we can stop submitting new task immediately after starting LUN removal
process.

By this change, no task is submitted to the LUN and previously queued
tasks have no chance to be kicked. Hence we execute them instead after
stopping new task submission.

This change simplifies LUN hotplug process and reproduce LUN hotplug
issues solidly if we don't have the fix.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9a33e6a217978b0863d15aaff3d35880dbdccfd4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
74ab9f9041 lib/scsi: Factor out submiting pending tasks in LUN into a function
This is a preparation to the next patch. If connections detect that
LUN is removed, we have no way to spdk_scsi_lun_execute_tasks()
to the pending tasks.  Hence the next patch will call the new
function scsi_lun_execute_tasks() in that case.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifdf630bd349c7d2099a6a14accc52f77c129f641
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473610
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
b386a3a9a2 lib/scsi: Assert if ref count of task goes negative
We didn't have any check if ref count of SCSI task doesn't go negative.
This addition will verify the idea to LUN hotplug issue by subsequent
patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I1d6ca9b9439bc7c4b701b65cc004cdf5cae16e91
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473585
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
3d8c12a16f lib/iscsi: Remove duplicated freeping primary task for write I/O
For large split write I/O, spdk_iscsi_task_cpl() does the same
operations for the error case as the normal path.  Hence remove
duplicated operation in spdk_iscsi_conn_free_pdu().  This fixes
the issue that the reference count of the primary write task
goes negative.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I169d8932821f2a1c8e1f153347cd3175f1291bf1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
e4cf639f7d lib/iscsi: Free primary after last subtask done for read I/O
For large split read I/O, the primary task have to be freed by the
last subtask. However, if LUN is removed in the middle of the split
read I/O sequence, the primary task is freed by the not last
subtask.  This had caused critical system failure by LUN hotplug.

This patch fixes that.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I78acaf054360254dffbdc282c2d0d8bb5868e5d4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473783
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-14 18:08:12 +00:00
Shuhei Matsumoto
0326031ecd lib/iscsi: Do not overwrite primary->lun even when LUN removal is notified
In spdk_iscsi_conn_handle_queued_datain_tasks(), we had overwritten
task->scsi.lun.  However, if the primary task is already submitted,
it cannot process IO completion correctly because task->scsi.lun is
NULL.  This patch fixes the issue.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia63f4c2e37b43477eaccbfd6dfea28fa357bde12
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473627
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-11-14 18:08:12 +00:00
Seth Howell
d115518ba0 test/nvmf: use nvmfappstart in bdevperf.
just a little cleanup.

This patch series is aimed at enabling reset handling in the spdk nvme
bdev module and demonstrating how it will work using a test in bdevperf.

Change-Id: I1f9108aa2e6f197c8e1c138f0e3205d78fa89dce
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473760
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-14 18:07:39 +00:00
dongx.yi
65cb0a108d scripts/ceph: Fix some compatible issues about ceph-14.*.*.
This is fix the issue on fedora30 about ceph-14.*.*.
Our ceph setup is a simple or demo case for rbd creation.
Make this script can run well on all versions.

Unify the filesystems about filestore and bluestore. That
all versions' logs seem similar.

We can trace the log through file:/var/log/ceph/ceph-mon.a.log.
It can record the whole processes.

osd.0 [v2:127.0.0.1:6800/2770008,v1:127.0.0.1:6801/2770008] boot
osdmap e5: 1 total, 1 up, 1 in

fix issue: #980.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: I594fdab791fdc6af6df739dacef4648e396e3d35
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473831
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-14 16:29:58 +00:00
Artur Paszkiewicz
0b4ca5225a module/raid: raid level specific start/stop callbacks
Add a function pointer to raid_bdev_module to be called before the bdev
is registered and a corresponding one to be called before the bdev is
unregistered. This allows setting up the bdev parameters and any custom
initialization/cleanup required for the raid module.

Change-Id: Ib9fe8f0365ca47f499a50630f582399e7bb9fd0f
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472714
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-14 16:25:55 +00:00
Artur Paszkiewicz
9d94e1f53e module/raid: don't declare variable inside 'for' statement
Fix all occurances in raid source files.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I1862b703e213ca133fb81e46dd9502b0b9d32d9f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474071
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-14 16:25:55 +00:00
Artur Paszkiewicz
94ea87541e module/raid: add asserts in raid_bdev_configure()
This function should only be called for raid_bdev in 'configuring'
state, so convert the 'if' statement that checks that to an assert. Also
add an assert for the number of discovered base bdevs.

Suggested-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ib2fdf2af64ff0e9b458af4070321d138508a3df8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473966
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-14 16:25:55 +00:00
Artur Paszkiewicz
182278c093 module/raid: function pointers for submit_request in raid_bdev_module
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I3186ea3450b91a73fa976d3d1d9cc67326daa10d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471088
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-14 16:25:55 +00:00
Artur Paszkiewicz
6ba11b194e module/raid: function for raid bdev_io completion
Use a dedicated function to complete the raid bdev_io.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I2ef0d19b31064b56c63866713f18deccafaeb8f7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-14 16:25:55 +00:00
Artur Paszkiewicz
ae70d6a4a6 module/raid: simplify raid_bdev_queue_io_wait()
Pass raid_io instead of bdev_io to raid_bdev_queue_io_wait() and its
callbacks to eliminate unnecessary type conversions. Pass bdev and
io_channel directly as parameters instead of passing a base device
index.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Iecbf351ec1598b29709e7ccff2efb1776faf11f9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471086
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-14 16:25:55 +00:00
Artur Paszkiewicz
235cc2c956 module/raid: assign some values to variables to improve readability
These will also be used in the next commit.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Iffd1a6e7fda2fc1688f3923777c085dcacc8613d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-14 16:25:55 +00:00