10524 Commits

Author SHA1 Message Date
Seth Howell
3911922005 nvme: remove redundant transport_qp_is_failed checks
The qpair state transport_qpair_is_failed is actually equivalent to
NVME_QPAIR_IS_CONNECTED in the qpair state machine.

There are a couple of places where we check against
transport_qp_is_failed and then immediately check to see if we are in
the connected state. If we are failed, or we are not in the connected
state we return the same value to the calling function.

Since the checks for transport_qpair_is_failed are not necessary, they
can be removed. As a result, there is no need to keep track of it and it
can be removed from the qpair structure.

Change-Id: I4aef5d20eb267bfd6118e5d1d088df05574d9ffd
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-09 13:55:41 +00:00
Seth Howell
3369105f31 test/lvol: fix shellcheck failures on master.
Change-Id: Ia3dd62384a34564092b352487042a1eb2908f9fc
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 20:10:57 +00:00
Karol Latecki
283890cb71 test/vhost: remove unused test options from manual.sh
fio.sh script is never used for vhost performance testing.
There are separate test scripts created just for this purpose.

Change-Id: I55e9b8c0d53100d1ce8077cf1758590bbe971dee
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-12-06 17:27:25 +00:00
Darek Stojaczyk
4256981d5d test/lvol: rewrite construct_lvol_bdev_using_name_positive to bash
Make it a part of existing test_construct_lvol_basic
test case. It's the same test, but the lvol is created
using lvs alias rather than uuid.

Change-Id: I909ded489eb441aa9fa212b11b2cbb91b5db5bbe
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459672
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-06 14:58:44 +00:00
Darek Stojaczyk
da90d41a40 test/lvol: rewrite construct_multi_logical_volumes_positive to bash
Change-Id: I9dfcde7094376df9fb545fc0d784ea2e5ff07e01
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459519
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-06 14:58:44 +00:00
Darek Stojaczyk
f54ce6fb91 test/lvol: rewrite construct_logical_volume_positive to bash
Change-Id: Idd734af0ffc912852f8185c16d8591dd8152b7dd
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-06 14:58:44 +00:00
Darek Stojaczyk
7fa9063148 test/lvol: start rewriting python tests to bash
There are multiple things wrong with current python tests:
 * they don't stop the execution on error
 * the output makes it difficult to understand what really
   happened inside the test
 * there is no easy way to reproduce a failure if there
   is one (besides running the same test script again)
 * they currently suffer from intermittent failures and
   there's no-one there to fix them
 * they stand out from the rest of spdk tests, which are
   written in bash

So we rewrite those tests to bash. They will use rpc.py
daemon to send RPC commands, so they won't take any more
time to run than python tests.

The tests are going to be split them into a few different
categories:
 * clones
 * snapshots
 * thin provisioning
 * tasting
 * renaming
 * resizing
 * all the dumb ones - construct, destruct, etc

Each file is a standalone test script, with common utility
functions located in test/lvol/common.sh. Each file tests
a single, specific feature, but under multiple conditions.
Each test case is implemented as a separate function, so
if you touch only one lvol feature, you can run only one
test script, and if e.g. only a later test case notoriously
breaks, you can comment out all the previous test case
invocations (up to ~10 lines) and focus only on that
failing one.

The new tests don't correspond 1:1 to the old python ones
- they now cover more. Whenever there was a negative test
to check if creating lvs on inexistent bdev failed, we'll
now also create a dummy bdev beforehand, so that lvs will
have more opportunity to do something it should not.
Some other test cases were squashed. A few negative tests
required a lot of setup just to try doing something
illegal and see if spdk crashed. We'll now do those illegal
operations in a single test case, giving lvol lib more
opportunity to break. Even if illegal operation did not
cause any segfault, is the lvolstore/lvol still usable?
E.g. if we try to create an lvol on an lvs that doesn't
have enough free clusters and it fails as expected, will
it be still possible to create a valid lvol afterwards?

Besides sending various RPC commands and checking their
return code, we'll also parse and compare various fields
in JSON RPC output from get_lvol_stores or get_bdevs RPC.
We'll use inline jq calls for that. Whenever something's
off, it will be clear which RPC returned invalid values
and what were the expected values even without having
detailed error prints.

The tests are designed to be as easy as possible to debug
whenever something goes wrong.

This patch removes one test case from python tests and
adds a corresponding test into the new test/lvol/lvol2.sh
file. The script will be renamed to just lvol.sh after
the existing lvol.sh (which starts all python tests) is
finally removed.

As for the bash script itself - each test case is run
through a run_test() function which verifies there were
no lvolstores, lvols, or bdevs left after the test case
has finished. Inside the particular tests we will still
check if the lvolstore removal at the end was successful,
but that's because we want to make sure it's gone e.g even
before we remove the underlying lvs' base bdev.

Change-Id: Iaa2bb656233e1c9f0c35093f190ac26c39e78623
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-06 14:58:44 +00:00
Konrad Sztyber
6722e57e02 lib/vmd: check requested size before padding VMD's base address
The address was padded before the size of requested area was checked,
which for areas bigger than available space resulted in unnecessarily
padding the remaining area.

Change-Id: Ic52a6e30c1cbe2526ac7f3effc20279cd91cadbc
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475832
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:57:01 +00:00
Maciej Wawryk
2951c1010e Change fio version
Whole autotest fails on VM Fedora31
GH #1081
Fio version update to fio-3.15

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I3b91c426050eb30af6b58434b6219090c61a48ba
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476893
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-06 14:50:59 +00:00
Shuhei Matsumoto
fd77140073 lib/iscsi: Remove the code for the deprecated CPU mask
CPU mask parameter was deprecated in v19.10.
If we remove the related code from the portal parser, we will be
able to use the parser for the iSCSI initiator to know the target
portal in iSCSI fuzz testing.  So let's remove the parser and its
test code here.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I72ad4364323abda0f0ed10519b56244cd0c7612e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476830
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-06 14:50:43 +00:00
paul luse
487eb66e24 module/crypto: enhance recent session optimization
When we moved to shared session a limitation of one crypto
volume was accidnetally introduced. Add a limit to keep the
session pool size down and print debug message when its hit.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic55efd554f37c4f44149edfbfe037420d8d61ce6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474264
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:49:40 +00:00
Jin Yu
f209b343b6 bdev: add the tsc trace
Add the tsc trace for the parent bdev_io.

Change-Id: Idf5c5df3953fa1697c384a4f83b591584dd30a80
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476588
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:47:56 +00:00
Jin Yu
c6e50f7f41 bdev: record the submit time
The reset cmd and parent of splited IO would not go into
the _bdev_io_submit() so they don't record the submitted
time. The timeout IO check needs the submitted time so we
add them in this patch.

Change-Id: Ic7217171ec878d4a6dfa80d6635957ca0186928b
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:47:56 +00:00
JinYu
56de1b0061 bdev: add bdev channel a link
Add a link to the bdev channel for linking all the bdev
IOs that were submitted to this channel so that we can
monitor each IO's consuming-time.

Change-Id: I1e425b2059f20fd7b158eb3d6b023ce8629e7a30
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:47:56 +00:00
Konrad Sztyber
860a075094 bdev/ocssd: read / write support
The patch adds zone address to Open Channel LBA translation as well as
initial support for read  and write commands.  Each IO command is
currently limited to a single zone (chunk).

Change-Id: I3ee6d58323871f0651ac1d5e8dda28eb6d687a95
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467149
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:45:05 +00:00
Konrad Sztyber
b9635c2f16 bdev/nvme: move nvme_io_channel to common.h
Other NVMe bdev modules (e.g. OCSSD) need the definition of the
nvme_io_channel to be able to send IO requests, so this structure has to
be defined in the common header.

Change-Id: I550d15d091078588c6c7ab824d883e049ec5a72c
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470019
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-06 14:45:05 +00:00
Konrad Sztyber
6bda87b324 bdev/ocssd: create / delete calls
Added a way to create and delete OCSSD bdevs on top of OC NVMe
controller.  The controller can be created using the regular NVMe bdev
RPC call.  For instance, the following (assuming 0000:00:04.0 is an
OC device):

 rpc.py bdev_nvme_attach_controller -b nvme0 -a 0000:00:04.0 -t pcie
 rpc.py bdev_ocssd_create -c nvme0 -b nvme0n1

creates Open Channel controller nvme0 and OCSSD bdev nvme0n1 on top of
it.  The bdevs can be deleted either by the bdev_ocssd_delete call or by
deleting whole NVMe controller, in which case all bdevs are destroyed.

Change-Id: I9f2f02103fc5570a53bd26479c8690be206829c3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:45:05 +00:00
Konrad Sztyber
a116324c88 bdev/ocssd: initial Open Channel bdev implementation
This patch adds initial Open Channel zoned bdev implementation.  The
bdev will allow to use the zoned API on top of OCSSD devices.

Added the ability to create the OCSSD NVMe controller.  The controller
is created using the regular NVMe's RPC when the controller is detected
to be Open Channel.

Change-Id: I31d271126dba4369ac2eaebd4cc7bdd460e5f808
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467147
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:45:05 +00:00
Changpeng Liu
3ef7f2b7ea nvmf: remove special xfer check for get/set features admin commands
There are some new features defined in NVMe 1.4 specification also need
the data buffer, but for now we only listed 3 features, we will check
the data length field anyway, so this piece of code can be removed.

Change-Id: I22204cf53077073434bb0ed73693bcf72883f084
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475953
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
bc13d02237 nvmf: move transport spdk_nvmf_*_req_get_xfer() function into the common nvmf library
Change-Id: I1619cc9b3feea1feb16282dc6c9cc8d5a380282c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
0c9057f031 nvme: replace cdw11 with specific union strucutre
Change-Id: I1152b5d6d5f8e3c2f96dcca1353d85a410924fb4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
38d982f5b4 nvme: add the definition of reservation notification mask/persistence features
Change-Id: Idfb95c8405bd19d7c1bf5114654184a0c75f80ee
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475817
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
974bb84e1f nvme: add cdw11 union data structure definition
Change-Id: I242b61dd1a0bfa66a051ad15f8edfefd8e31ef52
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
ff0a435c08 nvme: move the features definition before the spdk_nvme_cmd
We will add the cdw11 specific commands definition in the next patch,
all the features definition will be reused, so just move the code
to the right place, no code was added with this patch.

Change-Id: I6186523cf1875ab0576f19957ae4c31733e4a8a6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
5b54916b83 nvmf: split reservation acquire/release as different cases
Reservation Acquire Action(rqcqa) is used for reservation acquire command, since
we have replaced cdw10 with specific commands fields, so split the two cases to
make the code looks more clear.

Change-Id: Ib2a2458dd0fe8345f1c33341dab1c4fe3b82829f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476838
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
1fea1fccf5 nvme: replace cdw10 with specific union command
Change-Id: Ibdbc6d0356749795db26f58aa5c5e68739d5df5e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-06 14:43:41 +00:00
Changpeng Liu
bae1750bef nvme: add cdw10 union data structure
NVMe command dword 10 and dword 11 have multiple definitions for
different commands, add a union data structure here then we can
replace the hardcoded cdw10/cdw11 with command specific structure
in SPDK NVMe driver.  For now add dword 10 for the first step.

Change-Id: Ie2c58b3039cfa5843d44224d31a38850ff577ed0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-06 14:43:41 +00:00
paul luse
bcfb51b124 module/crypto: add UT coverage for error path in poller
Add coverage to the path where we dequeue multiple ops for a
bdev_io and one of them has failed. Confirm that the bdev_io
gets failed.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie864b63819f506da43fdcad960c26a48a01196c7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472417
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-05 13:06:35 +00:00
paul luse
dc8788d150 module/crypto: handle enqueue errors
A recent change in the CryptoDev API means that failure to enqueue
all attempted submissions does not automtically mean busy. We need
to check the status of the last submitted op and only retry if
busy, otherwise fail the IO.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I0873d07a430a08f5aee25581e47187ef60ba8542
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472400
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-05 13:06:35 +00:00
paul luse
c602bd81f6 module/crypto: change how we handle crypto device full condition
Previously we would sit in the submission routine and spin on the
poller and then retry in the event that we could not get CryptoDev
to take all of the ops that we had available.

The implementation was fine however a recent CryptoDev change
requires us to now check the status of attempted submissions
instead of assuming that the device was busy. If there was
another reason for the failure we don't want to retry or we'll
be stuck in an endless retry loop.

Changing the current device full handling to match what was
done in the compression vbdev makes it easier to handle the
new condition and also makes the two vbdevs more consistent
with regards to how they manage the DPDK framework API.

This patch only changes how we handle full conditions to put
the ops on a linked list and resubmit them the next tie the
poller runs naturally. A following patch will handle the
CryptoDev API change that instigated this change.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Idcb6e06b6826045f23e59b64eca051f3eee2b850
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472309
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-05 13:06:35 +00:00
Seth Howell
87dea05be4 test/nvmf: call nvmftestinit before checking IP.
The first IP is not configured until after nvmftestinit.

Change-Id: Icfc4ce40e357d36da2c50fefb3b967806966eb80
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476648
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-05 12:37:23 +00:00
Seth Howell
15fd9afbad test/nvmf: call killprocess only when nvmfpid exists
In nvmftestfini we try to kill the nvmf application, but this doesn't
get set by nvmftestinit. I think that nvmftestfini and nvmftestinit
should be symmetric options, or at the very least, one should be able to
call fini right after calling init without fini failing.

Change-Id: I14f274f940ec20c2d5f8d831fed3f6d6c4c6d2ca
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476687
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-05 12:37:23 +00:00
Jim Harris
5bd76a1098 rpc.py: expliclty print the exception's message
This worked fine as-is with Python 3.  But when
trying to run rpc.py with Python 2, it was noted
that simply doing print(ex) didn't print the message,
it needs to be print(ex.message).

Note that the similar code for printing the exception
message when running an RPC script was already doing
print(ex.message), so this change also makes the
code more consistent.

We aren't supporting or testing rpc.py with Python 2,
and there are some fairly significant mods required
to get it to work, but this small change at least
reduces that effort for those needing to implement
it and maintain it themselves.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0a6c94889dc631ef33835433f20f35ddc738aafb

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476633
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-05 12:23:30 +00:00
Jim Harris
8bc9798dc7 rpc.py: add new kv to existing params dict
When we get an error when calling an RPC, client.py
makes extensive use of the ** operator to add
the method and req_id to the params dict variable.

This doens't work with Python 2.  We don't really
support Python 2, but there are at least some folks
out there who make their own mods to get rpc.py
to work with it.  We can very easily implement this in
a way that is Python 2 friendly, so let's do that
to make it a tiny bit easier for those folks.

Incidentally, I do think the changes here also make
this part of the code a bit easier to read and
understand.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1846c80e21032ffba67128bee946b041a61d0621

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476632
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-05 12:23:30 +00:00
GangCao
cd433e7ef9 RPC: properly handle the semaphore in error case
This is to fix below issue:
https://github.com/spdk/spdk/issues/1076

The problem is that the semaphore is not increased and
the wait operation time out.

Change-Id: I56e950d0705663b35de9257f17c400c31636ee34
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476048
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-05 12:22:45 +00:00
Shuhei Matsumoto
b70eb775eb lib/iscsi: Delete unused match_digest_word() and make_digest_word()
In iSCSI library, macro MATCH_DIGEST_WORD() and MAKE_DIGEST_WORD()
have been used instead of match_digest_word() and make_digest_word().
The latter has not been used for a long time.  Hence remove them.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0fdb5c24a120a08fe06f825ee5e6c24ba64c0edf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
4dedb3b3ba lib/iscsi: Suppress error log for receiving PDU after logout
This log has been observed often after recent refinements but
this log does not mean any error. So use SPDK_DEBUGLOG instead
of SPDK_ERRLOG here.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3aa39ab3773ae83586c99699fd2473ca02c35eb9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
30235a0b2a lib/iscsi: Move up iscsi_op_login_check_session in a file
This patch series orders login related functions top down in iscsi.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I23ea7101030c6d2d3fbccba878bdf77d89b814cf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476415
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
d1833a9f52 lib/iscsi: Move up iscsi_op_login_check_target in a file
This patch series orders login related functions top down in iscsi.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I39431b213216b1e4e677f80de8c14c40dd7da150
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
27d6428076 lib/iscsi: Move up iscsi_op_login_session_type in a file
This patch series orders login related functions top down in iscsi.c.

Other than code movement, fix a typo.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib55fe86295f47c1c86bb99d5e3a6862f508bfcfa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476413
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
661fc79c0f lib/iscsi: Move up iscsi_op_login_initialize_port in a file
This patch series orders login related functions top down in iscsi.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I82fe06e3488a1c623c5fc875cf797790ee4ea48f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
4308b9119f lib/iscsi: Move up iscsi_op_login_store_incoming_params in a file
This patch series orders login related functions top down in iscsi.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2f7dd681ca430c810dac6fcea122f84a142152d7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476411
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
3054c3f108 lib/iscsi: Move up iscsi_op_login_rsp_init() in a file
This patch series orders login related functions top down in iscsi.c.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8e123177124b4526e9bbe9001293c6f668b9c3bb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476410
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
9e1fe1ba70 lib/iscsi: Assert if conn->data_out_cnt goes negative or left positive
conn->data_out_cnt does not control anything now but adding assert
for conn->data_out_cnt will be helpful at least to ensure that the
current SPDK works correctly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I31ee90769ce0555e64bd41c283e8b437326efebf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476409
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
b24bd0390a lib/iscsi: Simplify iscsi_conn_free_tasks()
iscsi_conn_free_task() is used only when exiting connection now.
Hence we can remove the parameter lun and simplify the function
and its unit tests.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6e7bf09672edca1f70c042ac58f098114d71ec78
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476115
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
27d79ee367 lib/iscsi: Wait until all PDUs are flushed for LUN hotplug
We ensure current pending tasks are aborted and no new task or PDU
is pending after LUN is unplugged now.  Then we have to ensure all
task completions and aborts are sent to initiator.

However, we had removed pending tasks without any notification to
initiator.  For this implementation, we are not surprised if initiator
waits for in-flight commands for a long time.

Due to recent refinements of LUN hotplug, we can wait safely that
all existing tasks complete or abort.

In _iscsi_conn_hotremove_lun(), after aborting R2Ts, start a poller
to wait until all PDUs for the LUN are flushed.  Then close the LUN.
We can safely free deferred PDUs for the LUN because they are
already flushed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I869b79a7c93d2e8a4a1577cc20d3b466548dfaaa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
081153a82c lib/iscsi: Rename iscsi_conn_remove_lun() by iscsi_conn_hotremove_lun()
We want to use poller for LUN hotplug in iSCSI library. According to
the naming used in SCSI library, rename iscsi_conn_remove_lun by
iscsi_conn_hotremove_lun.  The next patch will iscsi_conn_remove_lun.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I08bb8c92db23ac3adcde4f39c0e812f3d97430d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476114
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
6415d84113 lib/iscsi: Introduce struct spdk_iscsi_lun to refine LUN hotplug
Previously we had allocated temporary context, struct
_iscsi_conn_remove_ctx to process LUN hotplug.  However, we
had to consider out of memory during connection is active,
and we could not use any poller to process LUN hotplug because
LUN hotplug may conflict with connection exit, and this
possible conflict made us very difficult to use poller for LUN
hotplug.  Introducing struct spdk_iscsi_lun will resolve all
these issues.

Allocate struct spdk_iscsi_lun per LUN and store connection,
LUN, and LUN's descriptor into the struct.  Then use the struct
for LUN hotplug and free the struct after LUN is closed when
LUN is removed or connection exits.

struct spdk_iscsi_lun is similar with struct spdk_nvmf_ns in
NVMf library.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ice26330f3948070c96d2fb53b94941be3b467079
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
3f1e18cbbf lib/iscsi: Factor out operation to open each LUN for connection
This is a preparation to the subsequent patches.

Subsequent patches will introduce struct spdk_iscsi_lun
to refine LUN hotplug process and fix critical issues of it.

This change will make us easy to introduce struct spdk_iscsi_lun.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I75db59d88bb09ee2ea94e8c02e0e87003352850c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476112
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-12-05 12:21:09 +00:00
Shuhei Matsumoto
22adcd1487 lib/iscsi: Fix orphaned PDUs when freeing PDUs and tasks for LUN hotplug
In _iscsi_conn_free_tasks(), we had parsed conn->write_pdu_list
and then parsed conn->queued_datain_tasks.  However when we parsed
conn->write_pdu_list, if there was any task in conn->queued_datain_tasks,
some PDUs were inserted conn->write_pdu_list.  Hence after parsing
conn->write_pdu_list, new PDUs were in conn->write_pdu_list as orphan.
Then orphaned PDUs were freed later but LUN was already freed and
critical failure occurred.

This patch swaps the order of conn->queued_datain_tasks and
conn->write_pdu_list, and add comment to explain the change.

Additionally, this patch adds unit test which fails if it runs
without this fix.

Fixes issue #1030.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icb0ffbbbac70792a62939dc55a69df05d2ab9128
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-05 12:21:09 +00:00