14898 Commits

Author SHA1 Message Date
Shuhei Matsumoto
7a783b5b38 iscsi: Factor out getting data buffer from mempool into helper function
Wrap an operation to get a data buffer from mempool into a helper
function iscsi_datapool_get() and wrap an operation to put a data
buffer to mempool into a helper function iscsi_data_pool_put().

Use inline for both functions.

Besides, as a minor fix, remove duplicated file inclusion between
iscsi.c and iscsi.h.
7

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia3005dffaa93a6bca16f19bb467fb5b64ae1aad2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: sunshihao <sunshihao@huawei.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-12 09:49:31 +00:00
Shuhei Matsumoto
3de09f8ece lib/iscsi: Support data split case for CRC32C computation
Add crc32c to struct spdk_iscsi_pdu and initialize it by SPDK_CRC32C_INITIAL,
and then use it as the initial value of _iscsi_pdu_calc_data_digest().
Separate finalization of crc32c into _iscsi_pdu_finalize_data_digest().

Move the definition of related macro constants from iscsi.c to iscsi.h.

iscsi_pdu_calc_data_digest() is used for read too. So setting
pdu->valid_data_bytes before calling iscsi_pdu_calc_data_digest()
for read.

Data split will be supported only if DIF is disabled, and hence
DIF case is not changed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9d24f605fd0d452782e17695b613cd2f63d2e42f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6421
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-12 09:49:31 +00:00
Shuhei Matsumoto
acdeb53f4b lib/iscsi: Increase buffer size to 64KB at most when more Data-OUT PDUs follow
The following patches will want to aggregate multiple Data-OUT PDUs
into the same data buffer, but it will be 64KB at most.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I72eabbeae0b027c2fbff2a5837d180b06b0a1b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6418
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-12 09:49:31 +00:00
Shuhei Matsumoto
2ef97692bc lib/iscsi: Create subtask after reading data segment for Data-OUT PDU
The following patches will change the handler for Data-OUT PDU to
submit subtask only when 64KB data is read or F bit is set.

Previously, we had created a subtask when processing header and
before reading data segment.  Creating a subtask beforehand is not
convenient for the following changes.

Hence create a subtask after reading data segment.

If LUN is removed while processing the Data-OUT PDU, the corresponding
primary task will be terminated by iscsi_clear_all_transfer_task(),
and any subtask completion is not sent to initiator. Hence we can
reject the received Data-OUT PDU safely.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifb6d6988676080b458b31d12fef065f3c1de0cb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6415
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-12 09:49:31 +00:00
Jin Yu
59237d22b8 lib: add a hotplug lib for device
The hotplug lib can be used for pcie devices
such as nvme, virtio_blk and virtio scsi.

For the sigbus handler, there is only one in a
process and it should handle all the devices.

And align nvme to the hotplug lib

Add the ADD uevent support for allowing the
device hotplug.

Change-Id: I82cd3b4af38ca24cee8b041a215a85c4a69e60f7
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5653
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-04-12 09:48:21 +00:00
ChengqiangMeng
1c3bc9d64d rdma/common.c: Fix possible memory leaks
Signed-off-by: ChengqiangMeng <chengqiangx.meng@intel.com>
Change-Id: I134f3ca3dfb0c747b0ca8544068961a53470106f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7262
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-12 09:44:41 +00:00
yidong0635
35dfd3eafb memory: Fix memory leak of g_vtophys_map allocated failed.
if g_phys_ref_map allocated successfully and g_vtophys_map
allocated failed. It reports memory leaks. so add
spdk_mem_map_free(&g_phys_ref_map).

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I06149c98f158d1c679e5045e279beccbdea10e55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7249
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Chengqiang Meng <chengqiangx.meng@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-12 09:44:13 +00:00
Michal Berger
f40b0b1629 pgkdep/git: Upgrade ice driver to 1.4.11
The old 1.2.1 version doesn't compile against latest 5.11.x kernels
(e.g. due to recent changes in include/net/xdp.h) so upgrade to the
latest compatible release.

Also, move the url to sourceforge since ice is released more often
there under e1000 project.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I08b7beae922addeb877fe321679151b02bd253db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7011
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-12 09:43:31 +00:00
Michal Berger
0f8bff29f8 pkgpdep/git: Patch QAT to adjust for changes in recent kernels 5.11.x
Most notably, crypto/sha.h was split into two separate header files,
crypto/sha1.h and crypto/sha2.h.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I43ae61034f743bf4aae7f2f29919eec92d851f01
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-12 09:43:31 +00:00
Michal Berger
c09b139e61 test/nvmf: Use rxe_cfg to get rxe-enabled net devices
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iafc067f880525f25363db1adf9a94a9fc9215455
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6994
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:41:20 +00:00
Michal Berger
4d52dae32a scripts/rxe_cfg: Add env switch to skip printing status header
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ibc3149b5b9591826ff9aeed38457c2aa9ff747e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:41:20 +00:00
Michal Berger
e99cfdb6c9 scripts/rxe_cfg: Split collect_devices()
collect_devices() is split into two functions:

 - collect_net_devices(): Collect ethernet net devs from the
                          net class.
 - collect_rxe_devices(): Collect all rxe devices from the
                          infiniband class.

This is done in order to make handling of some conditions easier.
Case and point, in newer kernels, device/net link is not anymore
created for the soft roce devices, instead only ./parent attribute
is available. collect_rxe_devices() is adjusted to handle such
a condition.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idefa39c4a62c9e650a03e237f49940461e9782a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:41:20 +00:00
Monica Kenguva
132d80be05 rpc: Document scsi_get_devices
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Id7febeb8e4cad2f8f7e98366e38b6117afa7d7d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6916
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-12 09:40:19 +00:00
Monica Kenguva
039ab97c48 rpc: Document split bdev create and delete
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I3aba1fe4420830a06c9813d224417201e6ba594d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6753
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:39:29 +00:00
Ben Walker
2ab0506ef7 test/nvmf: Don't load kernel modules for custom transports
For custom transports, skip the kernel module loading and
unloading steps.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I193c62ffa0a9a5060b4bc504b39db3b6f660e1a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3584
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-10 03:49:40 +00:00
GangCao
01f45ecd8c NVMe: add the prefix of ctrlr identification for diagnosability
Introduce new utilities NVME_CTRLR_ERRLOG, NVME_CTRLR_WARNLOG and so
on to output the ctrlr's identification  at different log levels.

For RDMA and TCP, the subnqn will be output and for PCIe and custom,
the traddr will be ouptput.

Change-Id: I81a112463bf752999aa1fe4e0c867d88e09a2f64
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7057
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-09 17:29:21 +00:00
GangCao
415d664029 NVMe: fix the potential issue of dereference NULL pointer
Change-Id: Icb81239e64faa5e9c0daf361798b68e153effcee
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7154
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-09 17:29:21 +00:00
Ben Walker
e3e8fadca4 sock: Eliminate init parameter of sock_map_insert
This can be handled in a cleaner way by having the sock group
create/close operations take an extra reference.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Id701b6dd9a19b01cd40e0d95eb870aef977eea99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-09 17:15:57 +00:00
Ben Walker
4bcafd5b15 sock: Delay asking a socket about it's placement_id until after it is
added to a group

The process of adding a socket to a group may, in some scenarios, change
the placement id.

Change-Id: I879d9641099d86978ede5d5e2be1a72eda65a79b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-09 17:15:57 +00:00
Ben Walker
141a95aa36 sock: Placement Ids may only map to a single group
Change-Id: Ib917f2e95f93f4cb48d53e35f3273b0c42ba1a4f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7206
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-09 17:15:57 +00:00
Ben Walker
28b3889c8e sock: Use an enum for placement modes
Easier to read than integers.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ie9b8b16e1916b393a257e9ed0180ef9837f20cd2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7205
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2021-04-09 17:15:57 +00:00
Ben Walker
0254ef6d2b sock: Fix return code for adding a socket to a group twice
Also fix the comment. It's never going to make sense to add a socket
to a group twice.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Id4845b77114aef32bbe4ea0e53d2e1fde8e116f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7204
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-09 17:15:57 +00:00
Konrad Sztyber
c6b9113ba7 nvmf/tcp: use free() to release transport when create fails
Using nvmf_tcp_destroy() would destroy ttransport->lock, which hasn't
been initialized by that point yet.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie9ced97ef520236dddaa70453b6807e8382ce534
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7235
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-09 17:13:57 +00:00
Michal Berger
c84ecccca3 vhost/windows: Make sure host drive is properly cleaned up
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I53ec43c62f5ab7bef2719119252eec3c7c6b90d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7269
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-08 16:38:39 +00:00
Shuhei Matsumoto
0247a9945a thread: Add spdk_io_channel_get_io_device() to get io_device from io_channel
This will be useful as the same purpose as
spdk_io_channel_iter_get_io_device() and will be used in the
following patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id45f5980c65543703b91df2afeb47448232fe503
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7237
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
b3f998e58b bdev/nvme: Get current thread simply by spdk_get_thread() to set bio->orig_thread
These are not performance critical and we can get the current thread
simply by spdk_get_thread().

This change will make the following changes a little simpler.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ide79b1275ec0122f7f9812fe53c08a83d0aafeda
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7162
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
48cf3e9bbd bdev/nvme: Factor out deleting nvme_bdev_ctrlr into a helper function
This change will make the the upcoming changes a little easier.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I51776c8ab91a0dfd9150f17e567f8be795b9f4b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7046
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
61532e08ed bdev/nvme: Rename nvme_bdev_ctrlr_do_destruct() by nvme_bdev_ctrlr_unregister()
Probably nvme_bdev_ctrlr_unregister() is more appropriate name.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3a0d3622c084df909e5daeb548726cd9bd826ad8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7045
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
1a0144b7c4 bdev/nvme: Deleting OPAL and OCSSD after completing unregistration of io_device
Deleting OPAL device and OCSSD ctrlr after completing unregistration
of io_device will be safer and be helpful for the upcoming patches
to introduce subsystem.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7eb11d5c19bf50903496a7fb936c77aa05089046
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6889
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
68a8502f1c bdev/nvme: Remove nvme_ns->ref and use nvme_ns->populated and nvme_ns->bdev instead
As said in the previous patches, nvme_ns->ref is 2 at most, and
first is for populating namespace and second is for nvme_bdev.
On the other hand, nvme_ns->populated is for populating namespace
and nvme_ns->bdev is for nvme_bdev.

Preparation was done by the preceding patches. Let's remove nvme_ns->ref
and use nvme_ns->populated and nvme_ns->bdev instead.

We have unit tests for both normal case and shutdown case now.
So regression will be avoided.

These changes will be helpful for the following patches to support
multipath.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I279ca21a41d6ee2c07bbbeb62866e28423fcc6d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7097
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
2bed9983e2 bdev/nvme: Merge clearing nvme_ns->populated into nvme_ctrlr_depopulate_namespace_done()
Merge setting nvme_ns->populated to false into
nvme_ctrlr_depopulate_namespace_done(). This will simplify the
further changes.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4bb8347b085ec5855690f6dbde243269cedee706
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7136
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
b1d6c2f3cb bdev/ocssd: Add ocssd_ns->depopulate_pending to complete pending depopulation
The following patches will remove nvme_ns->ref and use nvme_ns->populated
and nvme_ns->bdev instead because nvms_ns->ref is two at most and
each count of nvme_ns->ref corresponds to nvme_ns->populated or
nvme_ns->bdev.

To do that, we need to ensure nvme_ns->populated is cleared after
spdk_bdev_unregister() is called, otherwise nvme_bdev_ctrlr_destruct()
is called twice.

However OCSSD namespace had used nvme_ns->populated to free resource
after getting log page completes.

To keep such deferral, add the depopulate_pending flag to struct bdev_ocssd_ns
and use it. Then clear nvme_ns->populated in bdev_ocssd_free_namespace()
after spdk_bdev_unregister().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I93199dbd504145bd1e5ea59f5914422c6c5bc938
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7102
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
f26d6c73e1 bdev/nvme: Set nvme_ns->populated to true just after nvme_ns is available
A few patches including this patch will remove nvme_ns->ref by
nvme_ns->populated and nvme_ns->bdev and remove nvme_ns->ref.

In this patch, set nvme_ns->populated to true when nvme_ns->ref is
incremented, i.e., nvme_ns is available. nvme_ctrlr_populate_namespace_done()
clears nvme_ns->populated if creating bdev failed by memset().
And add assert to nvme_ctrlr_depopulate_namespace_done() to ensure
nvme_ns->populated is false when nvme_ctrlr_depopulated_namespace_done()
is called.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I536933abb516c0a29231149ab564dc8b84245b10
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7096
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
8cc81fa07a bdev/nvme: Inline nvme_bdev_ns_detach()
The nvme_ns->ref was added to nvme_bdev_ns to support both multipath
and OCSSD range. OCSSD range was to split a namespace into multiple
partitions.

However, OCSSD range was not used and removed to simplify multipath.

When namespace is not split, nvme_ns->ref is 2 at the maximum. First is
for populating namespace and second is for nvme_bdev. nvme_ns->populated
is for populating namespace and nvme_ns->bdev is for nvme_bdev.
Hence we can use nvme_ns->populated and nvme_ns->bdev directly instead of
nvme_ns->ref.

A few patches from this remove nvme_ns->ref and use nvme_ns->populated
and nvme_ns->bdev instead.

dThis patch inlines nvme_bdev_ns_detach() into the callers and remove it.

The following patches will adjust the locations to update nvme_ns->populated
and nvme_ns->bdev and then remove nvme_ns->ref.

Removing nvme_ns->ref will be helpful to associate multiple namespaces
into a single nvme_bdev for multipath.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I87581d39769681adc5e1fdf0b33680e31d958e47
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7095
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
2b8661d6ac bdev/nvme: Move nvme_ctrlr_depopulate_namespace_done() to common.c
Moving nvme_ctrlr_depopulate_namespace_done() to common.c removes the
stub from bdev_ocssd_ut.c and will simplify the upcoming changes.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia94a88ca77518c7df08b52aa9298cdea5997ca1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7135
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
610dd86b8c bdev/nvme: Inline nvme_bdev_ns_to_bdev() and remove it
nvme_bdev_ns_to_bdev() was introduced because nvme_ns->bdev may be
removed to support multipath. However, it is unlikely now.

Hence it is easier to read to stop using nvme_bdev_ns_to_bdev() and
access nvme_ns->bdev directly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iaf35455218761bafd03060a21110ad1c4098b1db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7098
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
45d8309e52 bdev/nvme: Factor out deleting secondary trid into a helper function
Factor out deleting secondary trid from bdev_nvme_delete() into a
helper function bdev_nvme_delete_secondary_trid().

This will make the following changes simpler.

Besides, fix a typo, the case should be not 1B but 2B.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iba21efa0d8036ed15d2743a2548df05e866089d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7123
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
08e2210ace bdev/nvme: Merge callback and spdk_nvme_detach() into bdev_nvme_add_trid()
This change separates failover case more clearly, and will make
the following changes simpler.

Merge spdk_nvme_detach() and populate_namespaces_cb() into
bdev_nvme_add_secondary_trid(). Then change the return type of
bdev_nvme_add_secondary_trid() to void and move the comment to the
head of it.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0eb706728cf1b9ad7031eb66e11880a47e2ba767
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7044
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Shuhei Matsumoto
0e3de45def bdev/nvme: Refactor add_trid() and rename it by add_secondary_trid()
Clean up bdev_nvme_add_trid() by factoring out comparison of namespaces
and comparison of trids into helper functions, bdev_nvme_compare_namespaces()
and bdev_nvme_compare_trids(), respectively.

Then rename bdev_nvme_add_trid() by bdev_nvme_add_secondary_trid().

Rename is for clarification and clean-up is for the next patch.
The next patch will merge spdk_nvme_detach() and populate_namespaces_cb()
into the renamed bdev_nvme_secondary_trid(). Clean-up makes the
next patch simpler.

One note is that checking if the type of trid is not PCIe is done
by holding mutex now to prioritize clean up.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idc8652329dd2c721d101a724ec1a57a66c4174a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7094
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-04-08 16:10:26 +00:00
Karol Latecki
7366e569fb build/fio: disable warning for fio clang-11 builds
Fio 3.19 fails to build with Clang 11 because of
implicit-const-int-float-conversion error.
Disable -Werror checking for now.

Fixes #1848

Change-Id: I0f3922d7af412de35d9f79f3f5413af146996bf3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-08 16:09:00 +00:00
Alexey Marchuk
c3853598b1 nvme/ut: Add test to cover failed split of SGL list
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Iff99d5402ffaf3a526a2ac04539672d47a0a1ccb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7093
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-05 22:58:57 +00:00
Alexey Marchuk
c7bb68aa3e nvme: Handle errors returned by submit function
When a request is submitted, it may have incorrect iov
alignment that doesn't fit PRP requirements. In the
current version an internal function fails such a request
and returns a NULL pointer. This is mapped to -ENOMEM
error which is returned to generic bdev layer where
such a request is queued in a "nomem_io" queue and
later can be resubmitted. That is incorrect and such
a request must be completed immediately. To fail the
request, we need to differentiate between -ENOMEM and
other cases, so we pass a pointer to a result to
local nvme functions

Change-Id: I7120d49114d801497a71fca5a23b172732d088de
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7036
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-05 22:58:57 +00:00
Alexey Marchuk
a1f848b057 nvme: Add a function to handle error case in IO functions
This function will be used the next patch, current
behaviour remains unchanged

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie74c7395f67a08b0cac018eb5114f358a6b583cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-04-05 22:58:57 +00:00
Monica Kenguva
90db6167ce rpc: Document enable_vmd
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I04f58517ecae09e810d95d22bebc7215028cf016
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-02 08:23:05 +00:00
Michal Berger
7347f60bbc autotest: Use rpc_cmd() for rpc plugin-based tests
Also, create a common dir which will hold symlinks to all existing
plugins used in the tests. Location of the actual lib is not changed
so the relation to the given test suite is clearly preserved.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Icb70bbc61fbfa3325a357d5dd93f554ff132a3b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-02 08:22:42 +00:00
Michal Berger
5ddb210e61 scripts/rpc: Support loading plugins in --server mode
Put plugin parser into a separate function and use it to additionally
parse arguments passed to rpc.py via stdin.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I170d762ed9f5483d92b298f4804ee4e9f227a751
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7145
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-02 08:22:42 +00:00
Michal Berger
31f8d52d13 test/bdev: Switch to rpc_cmd()
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I01eb2e6b6da1e633278a1998ca8982ea8a132720
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7088
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-02 08:22:42 +00:00
Michal Berger
08d4dce28c test/common: Allow rpc_cmd() to execute multiple commands
This allows to pass to rpc_cmd() sets of commands via stdin. E.g.:

rpc_cmd <<-CMDS
	bdev_malloc_create -b Malloc0 32 512
	bdev_malloc_create -b Malloc1 32 512
	bdev_malloc_create -b Malloc2 32 512
CMDS

Since rpc.py is already running in a server mode, this is slightly
faster than grouping commands and running scripts/rpc.py directly.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I29fbcf3f2751400980d35b4de2cce2da1cd2bf2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-02 08:22:42 +00:00
Michal Berger
9aa3c063da test/common: Don't disable unit tests for scan|auto builds
Unittest job is now split into three smaller jobs, each of them
requires unit tests to be built.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ibc0dfbf6f7395a9dfe23242b532fb5cb1f8dd8f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-01 11:20:35 +00:00
Michal Berger
63330f68b4 autobuild: Ignore warnings comming from doxygen 1.8.20
This particular version reports plenty of false positives causing the
test to fail. Example:

nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ctrlr_alloc_io_qpair()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ctrlr_get_ns()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_read()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_readv()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_read_with_md()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_write()\ilinebr' not found
nvme.md:2: warning: @copybrief or @copydoc target 'spdk_nvme_ns_cmd_writev()\ilinebr' not found

This is potentially a regression in caused by the following issue:
https://github.com/doxygen/doxygen/issues/7948

The above seems to be resolved in latest releases, however, until
they are shipped as part of distros we support, similar mitigation is
in order.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I1e39c94f9f388de99e1106bcf15750e4b39384e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-04-01 11:18:53 +00:00