10899 Commits

Author SHA1 Message Date
Seth Howell
771d759312 lib/nvme: add spdk_nvme_transport_available_by_name
This new api function will enable us to work with custom transports.

This is needed to enable properly parsing and comparing custom transport
IDs that may all resolve to the same enum value.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I26aa3cb8f76f8273f564799d9b2af8041ea0d219
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
f038354efa lib/nvmf: enable pluggable NVMe-oF transports.
Change-Id: If1fd7d6c2385f42ca32dea0f8ecb528a60778d40
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-16 09:10:38 +00:00
Seth Howell
b397546e9d lib/nvmf: get_transport now takes a string.
This function previously accepted a trtype enum, but needs to be able
to accept a string to support custom transports.

Change-Id: I931aed30ca3be65468552ffa1bb1ef3f91275fda
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Seth Howell
5b3e6cd137 lib/nvmf: opts_init and transport_create use string now.
This will help enable pluggable NVMe-oF transports.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1947cc2e6e4ff078609f8bdbbdfefc5b110674c2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478753
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
2020-01-16 09:10:38 +00:00
Seth Howell
d4ea320bb5 lib/nvme: add trstring comparison to trid comparison.
Multiple custom transport types could share the same trtype variable. Add this additional check to compare them.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ief93d0d82aee754c9235cbab3c55827e57869fe9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478748
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Seth Howell
a050dcf21d lib/nvmf: add a default value to trtype enum.
For custom transports, we should use a range outside the spec value
to identify them.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I82b29c349e143b8906f79ce2de818def116a3fe4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-16 09:10:38 +00:00
Seth Howell
7ed0904b9b lib/nvme: update trid struct with trstring.
The trtype should be stored as both an enum and string. This is intended to
help pave the way for pluggable NVMe-oF transports.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I6af658d7a17c405e191ff401b80ab704c65497e7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-16 09:10:38 +00:00
Maciej Wawryk
6c2747491d Shellcheck - ignore SC2034 warnings
Add to pernamently excluded in shellcheck:
    SC2034: foo appears unused. Verify it or export it.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib63edb8cdf8ec718981d28d5acec75772b825a48
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478102
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-15 14:30:06 +00:00
Maciej Wawryk
19e48a66a4 test: Shellcheck - correct rule: Iterating over ls
Correct shellcheck rule SC2045: Iterating over ls output is fragile. Use globs.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I436878774d6f86c23b1c5cf5220da297053f03e2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-15 14:30:06 +00:00
Maciej Wawryk
a291cb0606 test/vhost: fix parted error
Vhost nightly test fails on Error: Partition have been written, but
we have been unable to inform the kernel of the change, probably
because it/they are in use.

Closes #1090

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iafd051e02b3dd32ede8a62c69c7079a0facade0c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476639
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-15 14:28:51 +00:00
Konrad Sztyber
aac65dcfd8 test/unit: remove vhost dependency from ftl tests
The tests were only built if CONFIG_VHOST was enabled, which doesn't
make sense, as they're not related to vhost in any way.

Change-Id: Idb151b3bdf8c5f2a90357e41099528b10a98f331
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481505
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-15 13:37:19 +00:00
Maciej Szwed
ca0eeaab21 unit/bdev: Unit tests for spdk_bdev_comparev_blocks.
These are unit tests for non emulated version of
spdk_bdev_comparev_blocks function.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I459ac614e1a03e42b9d584aaa907a1448a13d16e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478969
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
Maciej Szwed
3eda8bf6d3 unit/bdev: Extract common code for comparev tests
Upcoming patch will use most of the comparev test code
so let's reuse it.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ibc6230d38da224bfcf25f76fe1d834d48200bfd9

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479832
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
Maciej Szwed
bee042e431 unit/bdev: Add SPDK_BDEV_IO_TYPE_COMPARE to g_io_types_supported
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I509947d7b5a2139c9f721dc55e80d9eb8e744bd6

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478949
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
Maciej Szwed
77183c9c80 unit/bdev: Unit tests for spdk_bdev_comparev_blocks (emulated)
This patch adds test for spdk_bdev_comparev_blocks
function. For now it tests only emulated compare
operation. For native compare call will be created
separate patch.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Icf2297575f59ab4ee46ba1b40eff6729b3e95b1a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
Maciej Szwed
393afcb1d6 nvme: Unit test for spdk_nvme_ns_cmd_comparev_with_md
This is vectored version of test_nvme_ns_cmd_compare_with_md
test.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8af3a50ae764908024dc2758bb53b119b86edb2c

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
Maciej Szwed
32a961fc5d bdev: Add check for atomic compare and write unit
Compare and write fused operation num_blocks should
not exceed value of 'atomic compare and write unit'.
In case of NVMe native support we should read this
value from 'namespace atomic compare and write unit'
if set in namespace identify data, otherwise from
'atomic and write unit' field in controller identify
data. If bdev does not support this natively we should
set this value to 1.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ib1ea02dbf9d1eed476d9dd0114ea96b1376e0c45
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-15 12:25:26 +00:00
Maciej Szwed
61d6254d18 bdev/nvme: Add bdev_nvme_comparev_and_writev function
This function will be used for NVMe which supports
fused commands natively.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I5c41d98e1830ea9e14a521ccb06c6e9284857eb6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477026
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-15 12:25:26 +00:00
Wojciech Malikowski
8ffec69bff bdev/nvme: ftl_managed flag is no longer needed
bdev_ftl is virtual bdev now and is not associated
with nvme_bdev anymore so ftl_managed flag could be
removed.

Change-Id: I720e05aed9c36a9d36079276fbd27fe9ad70c0c0
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478614
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
79fc6ed2f8 CHANGELOG.md: ftl interface update
Flash translation layer library interfaces updates
related with witch switch to bdev_zone API.

Change-Id: I0a6cc58b7b526ea6de7bc3f324806429659fa9bc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481591
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
f2c8083ef3 lib/ftl: Functional tests
This patch adjust functional tests to
new FTL stack.

Change-Id: I0b8ff08c61793d7f1685942031aecf9cc88d1f24
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468738
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
151e37cfaa bdev/ftl: Examine config support
Bdev FTL is dependent on base bdev and cache
bdev. Allow for examine config if dependent
bdevs are not ready during bdev FTL creation.

Change-Id: I917994d7015f3b74a29ccd066f0c6989ad3c1c4e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471375
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
e2e6254651 bdev/ftl: Remove NVMe dependencies
This patch changes FTL bdev to vritual bdev.

Change-Id: I7b96af56053874b670a76b910a846837396119d9
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479703
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
843f296e2e lib/ftl: Replace Open Channel dependencies with zone bdev API
This patch replaces NVMe Open Channel API usage
inside FTL library with corresponding zone bdev
API calls. This include following calls:
 - spdk_nvme_ctrlr_cmd_get_log_page -> spdk_bdev_get_zone_info
 - spdk_nvme_ocssd_ns_cmd_vector_reset -> spdk_bdev_zone_management
 - spdk_nvme_ns_cmd_read -> spdk_bdev_read_blocks
 - spdk_nvme_ns_cmd_write_with_md -> spdk_bdev_write_blocks

Change-Id: I1b5a6863d9ce72f4af1cfbb0e449fc1a5b638144
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479702
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Wojciech Malikowski
0178395e2c test/ftl: Temporarily disable FTL functional tests
Next patch in this chain will break FTL functional
test so disable it temporary until FTL refactor will
be finished.

Change-Id: I69168209163784e13fd5f56db9daa80b6709344f
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-15 12:24:44 +00:00
Tomasz Zawadzki
1437b25472 lib/blob: make sizes of pages array consistent
Just to make all sizes consistent and less error prone.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id0a21bbd45954a0f2317e0eefd3725f1542ef04f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479961
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 17:13:15 +00:00
Tomasz Zawadzki
eba7f9f5ea lib/blob: make sizes of cluster array consistent
Fixed size of check in _spdk_bs_snapshot_newblob_open_cpl().

Rest are just to make all consistent and more error prone.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5a23a7795f1e598c1cfd6d17ce37b367f2f34df8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479960
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 17:13:15 +00:00
Tomasz Zawadzki
4b8db27b2a lib/blob: add _spdk_bs_md_page_to_lba() function internal to blobstore
The _spdk_bs_page_to_lba() [without 'md'] is only for translating the
pages on the blobstore to lba they are at. Those pages start at
the begining of the device and cover all of it. Thus simple
math is enough to translate those.
It is used to calculate lba_count for set of pages as well.

Meanwhile there are 'md_pages' which are the same pages as for
the above, but their count start at bs->md_start.
Which is right after super_block and couple pages for bit masks.

This patch creates new _spdk_bs_md_page_to_lba() that is more
explicit in what page number is passed. Hopefully avoiding
confusion when reading which page number refers to which
'type' of page.

Exception to the that is _spdk_bs_dump_read_md_page(), where
blobstore is not actually loaded (md_start from super block
is not copied to bs structure).

Additionaly providing assert to catch errors on debug builds.
Making the check in _spdk_blob_load_cpl() for max_md_lba obsolete.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I66bbca55b5ca3d6794c462d50177e6037ddbefa6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479017
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 17:13:15 +00:00
Tomasz Zawadzki
72f6c8086a ut/blob: check for clone count in blob_snapshot UT
Previously this UT was only checking the snapshot counts.
This was correct, but since the blobs in this tests are
clones of each other - it would make sense to check it as well.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id9dec7f0a292cbb52da8196bfc77679fc0f49d8b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-14 17:13:15 +00:00
Maciej Szwed
91dac0f4d0 bdev: Add num_retries field in bdev_io
Added new field in bdev_io structure for tracking
number of IO retries. It will be used in future patches.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8e002e93f54c9ce39c7af0dd3a1960e6aea93580

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479828
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-14 07:51:45 +00:00
Jim Harris
ae43c81a30 bdev: add spdk_bdev_comparev_and_writev_blocks
We will only support a vectored variant of
compare-and-write for now.

This does no locking for now.  Ii will be added
in a separate patch.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5bd075c912de60090e19cf8fced19c4879fcc900
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475941
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-14 07:51:45 +00:00
Michael Haeuptle
d9afeb150e lib/nvmf: preparation for custom admin cmd handler
This commit exposes some internal functions and enums
in preparation for the custom admin cmd handler functionality.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: Iec15c1f3d9cba5db267f6e43f3d929cf382ca8f4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-14 07:51:18 +00:00
Jim Harris
2a2b7296ee bdev: do not allow overlapped locked ranges
We can't allow overlapped locked ranges - otherwise
two different channels could be deadlocked.

So add a pending_locked_ranges to the bdev.  When we
start a lock operation, check if the new range overlaps
one that's already locked.  If so, put it on the pending
list.  When an unlock operation completes, we will
check if any pending ranges can now be locked.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2e3113216a195887b954533495ff200df14fadc1

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-14 07:50:26 +00:00
Jim Harris
ebd1a4f76c bdev: inherit locked ranges for new channels
Keep a mutex protected list of the active locked ranges
in the bdev itself.  This is only accessed when a new
channel is created, so that it can be populated with
the currently locked ranges.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id68311b46ad4983b6bc9b0e1a8664d121a7e9f8e

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477871
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:50:26 +00:00
Jim Harris
b90b7ce477 bdev: queue new IO that overlap a locked region
Add an io_locked TAILQ to each channel, which hold
IO that will write to a currently locked region.

Also add a new step to the locking process per channel.
Each channel needs to wait until all existing outstanding
writes to the newly locked range have been completed.

Only the channel that locked an LBA range may submit
write I/O to it.  It must use the same cb_arg for the
write I/O as the cb_arg used when locking the LBA range.
This ensures that only the specific I/O operations needing
the lock will bypass the lock.

When a range is unlocked, we will just blindly try to
resubmit all IO in the io_locked tailq.  This could be
made more efficient in the future, but we don't expect
this path to occur very often, so going for simplicity
in the first pass.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibdc992144dfaffe7c05471a5b3c020cedd8cdfc3

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478226
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-14 07:50:26 +00:00
Jim Harris
d84a88c1a0 bdev: add base infrastructure for locked lba ranges
This adds new internal APIs bdev_lock_lba_range and
bdev_unlock_lba_range.  To start, these APIs will
manage dissemination of lock/unlock requests to all
existing channels for a given bdev.  This does not
yet interact at all with any I/O sent to the channel.

Future patches will check new I/O to see if they
are trying to write to a range that is locked.  Future
patches will also ensure we do not have overlapping
ranges active at once.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6d5b1cc84b41a7adc2a3c5791c766bb77376581f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-14 07:50:26 +00:00
Jim Harris
b87080efa2 bdev: add lba_range and overlapping checks
This will be used by upcoming patches for implementing
LBA range locks.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifa4ad8dcc0d09ccf20d35f010fcae19dcc17abc9

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478224
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2020-01-14 07:50:26 +00:00
Seth Howell
9e118adeca test/autorun_post: fix ubsan reporting.
Change-Id: I7bfe785b355ec925d28d8c306395733945a7b5d0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
a571eb03e9 test: move report_test_completion into run_test
This also requires us to change the create_test_list function to rely on
the run_test function for creating the canonical test list.

Change-Id: Ib35e7752935a3ac83de2702b6dfbd42539027f6a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476962
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
2f476a638f autobuild.sh: simplify and unify test suite.
We have added a lot of flags to this script over time to try and shorten
parts of it that take a long time or modify the way we make and
whatnot.

In the test framework today we really have two settings, we either want
to run the entire autobuild package with all the bells and whistles, or
we just want to make the code and get on with the rest of the tests. I
believe this change can save between 1 and 3 minutes on each of the
functional test suites.

Change-Id: I7519e8320aa16b57f09f633f866dc36cb494aa80
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Seth Howell
cbc1e069ee configure: allow cuncurrent build of RBD and reduce
This was disabled in configure because at one time there was an ISA-L
dependency break between the RBD libraries and our own submodule.
It seems that the dependency break was fixed a while ago, but the check
to not allow one to build with the other was never removed.
Unfortunately, I don't have the details on what that fix was.

Either way, this compilation works now.

Change-Id: Ic249791549dae36c7279114d67e77be94e842ddf
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479726
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-14 07:49:08 +00:00
Pawel Kaminski
91dc22bae6 test: shellcheck - move rule SC1083 to permanent exclude list
SC1083: This {/} is literal. Check expression (missing ;/\n?) or quote it.
This rule points to places that are part of command
and cannot be fixed:
1. git archive HEAD^{tree}
2. vpp (e.g. dpdk { no-pci })

Change-Id: I30117913f964415cdaed94d29b1209d119e4d5a4
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479986
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-14 07:48:03 +00:00
Pawel Kaminski
7200ad8199 test: shellcheck - move rule SC2046 to permanent exclude list
SC2046: Quote this to prevent word splitting.

Change-Id: Iaafcf823c72a7dcf1b6eab893e14ea85e2c67ccf
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-14 07:48:03 +00:00
Pawel Kaminski
9c47dd8e09 test: shellcheck - move rule SC2086 to permanent exclude list
SC2086: Double quote to prevent globbing and word splitting.

Change-Id: Ia427dbb3b5d5ad20bcc09a3a455b80d0f69fb583
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477266
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-14 07:48:03 +00:00
Pawel Kaminski
8d4ecf1e22 test/vhost: Fix. vm_run takes vm numbers as separate arguments.
In patch fdc82d3 'vm_run $@' was changed to 'vm_run "$@"' but
arguments like 'vm_run "0 1"' should be passed as
'vm_run 0 1'.

Change-Id: I2a51a31fc0b0b55079963b11587be55d9bf86a87
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-13 12:45:04 +00:00
Pawel Kaminski
6df3f23f22 test: Shellcheck - apply rule SC2097 and SC2098
SC2097: This assignment is only seen by the forked process.
SC2098: This expansion will not see the mentioned assignment.

Apply these two rules together as they point to the same code.

Change-Id: I61020e8fd4b6db2b06ae8134d1004290a9bef9e7
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-13 12:45:04 +00:00
Changpeng Liu
1ca017e0df vhost_user: add VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD definition
We will enable vhost live recovery and packed ring feature in next
few days, however most of the code is in DPDK vhost library, so we
will not enable this feature for our internal vhost library, but
there are many users still use it, so we will not support the new
features with internal vhost library but will maintain it until
we can drop it in future, this is the first patch to do it, another
patch will be submitted until the packed ring patches being merged.

Change-Id: I50fa0314fd64fa5ee1f06a78e1495e2d0d50a0d0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479502
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-13 12:43:16 +00:00
Changpeng Liu
f508f50a5e vhost: disable Waddress-of-packed-member for internal vhost library
When using internal vhost library, there is a warning information:
taking address of packed member of class or structure may result in
an unaligned pointer value [-Werror,-Waddress-of-packed-member].

DPDK disable this check by default, so we will do it same for
internal vhost library.

Change-Id: I6fb12d5f5339ccb03bb85196f9c100d0521e4620
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479501
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-13 12:43:16 +00:00
Changpeng Liu
2020418f2b vhost_nvme: fix one compilation issue
Commit 1a1cbdf30 changed the struct spdk_vhost_dev to have static
cpumask variable which broke the complilation.  This patch will
fix that.

Change-Id: I8d0ef90f491e4fc4f0fae6faff2909fd3bd68053
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479500
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-13 12:43:16 +00:00
Kozlowski Mateusz
db126b3ee1 lib/app: Change signal handler to use spdk_thread_send_critical_msg
Fixes deadlock when closing application with SIGTERM during call of
spdk_thread_send_msg.

Fixes #1105

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: If191550f9681a30674fc9c0704289fcdb70b33cb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478291
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-13 12:42:49 +00:00