When we have a really large disk, setting up a nested lvol across the
entire drive can actually be quite time consuming. When trying to run
this test on an 800G p3700 I was actually getting timeouts on the rpcs
when creating the nested lvolstore. By only taking the first 20G of the
drive to set up the stores, we end up with a much quicker test and
conveniently get rid of the timeouts I was facing.
Change-Id: Ifd43c21c66bc77b615da9b0f421e95337b4c7019
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
With all the pieces in place we can finally remove
the legacy cross thread messages from vhost.
We replace spdk_vhost_allocate_reactor() with
spdk_vhost_get_poll_group(). The returned poll_group
has to be passed to spdk_vhost_session_send_event(),
where it will be assigned to the session. After the
session it started, that poll group will be used for
all the internal vhost cross-thread messaging.
Change-Id: I17f13d3cc6e2b64e4b614c3ceb1eddb31056669b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch adds UT code to test if the bug in nvmf_tcp_pdu_set_data_buf
is fixed by the last patch.
Change-Id: I0b504ba37652de8300c85eea2f85218c837dd503
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455624
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Using malloc'ed string for string in iSCSI target has caused
scan-build error.
Define maximum port number of portal to be 32 and use fixed size string
whose size is 33 for spdk_iscsi_portal_grp::port.
This change will reduce the potential malloc failure.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie1fcdbd45ce000a9c1c53761195697555b8d030a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459709
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Using malloc'ed string for string in iSCSI target has caused
scan-build error.
Define maximum IP address of portal to be 256 and use fixed size string
whose size is 257 for spdk_iscsi_portal_grp::host.
This change will reduce the potential malloc failure.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iceeae94e250ea426f72ff72355a213606308da51
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459708
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Using malloc'ed string for string in iSCSI target has caused
scan-build error.
Maximum size of target name is already defined to be MAX_TARGET_NAME,
and hence use fixed size string whose size is MAX_TARGET_NAME + 1 for
spdk_iscsi_tgt_node::name.
Change psdk_iscsi_tgt_node::alias together.
This change will reduce the potential malloc failure.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iac4cd6e9d60173ddeb68ca21ce712126c13bc3c4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459707
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Using malloc'ed string for string in iSCSI target has caused
scan-build error.
Maximum size of initiator address is already defined to be
MAX_INITIATOR_ADDR, and hence use fixed size string whose size
is MAX_INITIATOR_ADDR + 1 for spdk_iscsi_initiator_mask::mask.
This change will reduce the potential malloc failure.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic39e08986c9377800ce58a1cb5b8401c6b71cf96
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459706
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Using malloc'ed string for string in iSCSI target has caused
scan-build error.
Maximum size of initiator name is already defined to be
MAX_INITIATOR_NAME, and hence use fixed size string whose size
is MAX_INITIATOR_NAME + 1 for spdk_iscsi_initiator_name::name.
This will also reduce the potential malloc failure.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic6bc172125fc6c9c0896499704d2a9b522106da0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Here is the an example to describe existing issue:
There is a Write request with 64KiB data length, and this IO is cross the IO
boundary. We assume that the parent IO will have 2 children requests, one is
33KiB length, the other one is 31KiB. Here is the view of parent iovs, the
first 33KiB length data has 33 iovs:
iov.[0].iov_length = 1024;
.
.
iov.[31].iov_length = 256;
iov.[32].iov_length = 768;
.
.
iov.[64].iov_length = 1024;
In function _spdk_bdev_io_split(), then you can see that for the 33KiB length
child request, exiting code will run out of child child_iov space and return
error due to last one data buffer is not block size aligned.
Here we can rewind the existing offset to last block size aligned buffer to
avoid the error case, for backend which need aligned data buffer such as
AIO backend, the request will go through spdk_bdev_io_get_buf() again to
do the data copy, otherwise for those backend devices such as NVMe with
hardware SGL support, 256 data segment is fine for them.
Change-Id: I96ebdf29829d86f9b38fab28a7406eedc9fa44ef
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: I009926f41465298e3072a3cca1452e9808095c9b
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459764
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: Ic830ec69149dcb80f9bf5ac7bd95ecc93d4b6f74
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459763
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: Ib4d6fad99979d2e8ab61e18feb04661f388eb4c1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459762
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: If0bf702dcdc19bfaec44fb8867205f29f562097e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459640
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added check if band that is added to reloc
have any valid blocks. Return immediately
if there is no valid blocks.
Change-Id: I2bce088e0ad71479c6899fff96845397d12e2e92
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457625
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In case ANM event occurs on open band there
can be situation that reloc will try to read
block on which there is ongoing write.
This is happening because lba valid map is
updated before write submission to allow sent
consistent metadata to disk before all user
writes are completed.
Added write offset to the each chunk and
add check to reloc if particular ppa is written
on that chunk.
Change-Id: Ic95a06e69381d2152a86984b65a0975afaff955d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458056
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In case ANM event occurs on open band
reloc need to be able to process such
event.
If band is not in closed sate do not
alloc lba map for it and do not set
it to free state after relocation.
Change-Id: I2f4a5770fef08271d222936ca19f3cc98e5e5be1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457612
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Open bands need to have lba map segments
set to cached state to prevent read lba
map from disk during relocation events.
Change-Id: Ib4f1ed19131fad174c1d2f70e4c02e83701e2a0a
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457853
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Compilation Warning on fedora30.
iscsi_ut.c:184:31: warning: The left operand of '==' is a garbage value
CU_ASSERT(rsph->status_class == ISCSI_CLASS_INITIATOR_ERROR);
~~~~~~~~~~~~~~~~~~ ^
This is related to issue #822.
Change-Id: I221aa6601bae4a2d55bd557fcd1d001fbc616149
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459723
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Compilation Warning on fedora30.
dif_ut.c:151:24: warning: The left operand of '==' is a garbage value
return (iov->iov_base == iov_base && iov->iov_len == iov_len);
~~~~~~~~~~~~~ ^
This is related to issue #822.
Change-Id: Ie050c65d3747c551e2c26e84d3879742eaf86fd8
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459712
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When possible we should make sure that no silent failures or
skipped tests.
Since some of the tests use fio, autotest should fail
in those places when it is missing.
Change-Id: I97581e2df30a02445b560e3b88d03ce92986d0fd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456316
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Just use a for loop, then we can delete all malloc
devices.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I34fb06c0d49333ab1b5bce0ea2e2b7ced7c50eb2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458045
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This will be used to get extended LBA based range or length in NVMe/TCP
target later.
Change-Id: Id0f08bdaeea634dbc05b34a0f7914be21aef9aae
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458706
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add spdk_dif_update_crc32c_stream to update CRC32C by stream fashion.
spdk_dif_update_crc32c_stream utilizes the updated _dif_update_crc32c_split.
A minor bug was found in UT for spdk_dif_update_crc32c and is fixed
together in this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I92358e845e8e2e17c6f288aa718b947e71e6e1fb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment, and _dif_update_crc32c_split will have to
process partial data block, particularly the following types:
- start and end are both within a data block.
- start is within a data block, and end is at the end of a block
On the other hand, _dif_update_crc32c_split had assumed that passed
block is always a complete block.
This patch exposes offset_in_block, data_len, and guard as parameters
of _dif_update_crc32c_split() and make _dif_verify_split() process
the above two types of data block properly.
The next patch will utilize the updated _dif_update_crc32c_split to
add spdk_dif_update_crc32c_stream().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iee29377ad49d4f209673fffb4de4a23a54f31766
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458918
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Like other test, need to add SPDK_CU_ASSERT_FATAL(TAILQ_EMPTY(&g_bdev_io_queue)).
To avoid issue:
scsi_bdev_ut.c:956:2: warning: Address of stack memory associated with local
variable 'task' is still referred to by the global variable 'g_io_wait_queue'
upon returning to the caller. This will be a dangling reference
ut_put_task(&task);
^~~~~~~~~~~~~~~~~~
waiting exit from while loop in ut_bdev_io_flush.
This is related to issue #822.
Change-Id: If390f5076582a0149646f0c2a408c3bcd3b4fab6
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459534
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Tests that were thus far performed using posix net framework
can now be run with VPP. This patch adds network interface
configuration for VPP to work in iSCSI tests.
Some tests are disabled on purpose:
- IP Migration, RBD and NVMe-oF due to their tests lacking network
namespace support
- rpc_config adding/deleting IP, as VPP has separate utility for that
calsoft.sh doesn't handle TCP stream properly and fails decoding iSCSI
requests when are divided by TCP segmentation. This is very common
situation for VPP and causes that calsoft.sh never PASS.
Change-Id: I7c80427ca1675a1789ce7440796cc8d9956f1c9e
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/394174
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
NOTE:
Due to some changes in VPP two cherry-picks from newer version of VPP
(at least 19.04):
"sock api: allow to start client with no rx_thread"
commit: 97dcf5bd26ca6de580943f5d39681f0144782c3d
"dlmalloc: honor 8 byte alignment requests"
commit: f5dc9fbf814865b31b52b20f5bf959e9ff818b25
These commits are already merged in VPP 19.04.
This patch includes also workaround for VPP 19.01.1 to prevent closing
sessions to already closed applications after timeout. It causes
intermittent segfaults.
This is temporal solution and should be solved with next releases of
VPP, but is required now to create more stable VPP environment.
Change-Id: If4b45b7159819cfd836dd7d50f333dbab2b38eab
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456462
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This patch prepares environment to allow compile applications and
libraries with VPP.
Change-Id: Icc067c17b57932062afa5e6a67791bd58de3a2cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We have no guaranty that iscsiadm -m session shows all required iscsi
devices at some point of time after login. It may cause that not all
devices are used in further tests (e.g. in fio).
This patch allows to wait for a certain number of sessions after login.
Change-Id: I868cb8f1235dd04d88572bc8518e97de99a2c5e6
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458236
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Bdevperf up to now didn't use unregister callback when opening target
block deice. This is unsafe if test scenario involves target bdev
unregistering voluntarily, although with recent bdev layer update
not registering hotremove callback has become safer.
To be on the safe side, this patch registers a callback to transit
gone target into the I/O drain mode in case the target unregisters
during runtime.
Changes wrt previous versions of the patch:
- incorporated review comments
- updated callback functionality
- updated patch description.
Change-Id: I0dad2f4a7cf4acb22b363e23e4b670ca73d8c6f2
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447956
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add spdk_dif_verify_stream to verify DIF by stream fashion.
spdk_dif_verify_stream utilizes the updated _dif_verify_split.
spdk_dif_verify_stream is very similar with spdk_dif_generate_stream().
UT code demonstrates how it is realized.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1c5d197cf4c0bbc82c8e7f4fa45ddc0b94051058
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458330
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment, and _dif_verify_split will have to process
partial data block, particularly the following types:
- start and end are both within a data block.
- start is within a data block, and end is at the end of a block
On the other hand, _dif_verify_split had assumed that passed block
is always a complete block.
According to the refactoring done in the last patch, this patch
exposes offset_in_block, data_len, and guard as parameters of
_dif_verify_split() and make _dif_verify_split() process the
above two types of data block properly.
The next patch will utilize the updated _dif_verify_split to
add spdk_dif_verify_stream().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic371d3ccefbd5fe8147a948a624013be2702128e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458329
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
As have updated the NVMe reset case, the check_io() must be executed
after ctrlr reset. It's easier to reproduce #819, so we can set a
short time.
Change-Id: I9ba867a6c2aa1a3921bace2e77acd28a069d9016
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Address translation wasn't correct for >32 bit length packed address.
This commit fixes the issue and adds a corresponding unit test.
This patch fixes issue #774:
https://github.com/spdk/spdk/issues/774
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Idce67c47f2a9888f9e2ae2eadaf71ccc34e5c260
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457114
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Updated doc to include section on how to compile SPDK with link
time optimization (lto) gcc flag prior to running the benchmark.
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Change-Id: I1af4033144dd9f7594bcf40621acb7fb52018e7a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459330
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Scanbuild error on fedora30 reports:
warning: Array access (via field 'write_buf') results in a null pointer dereference
if (context->write_buf[i]) {
In deallocated_value.c, cleanup function be used at many places to deal with failed
cases even context->write_buf is NULL, so add context->write_buf pointer check before
array data. I think context->read_buf is the same.
This is related to issue #822.
Change-Id: I33c685fd732da820c1dfc861eb991b92b41caa29
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458736
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Missing dependency libudev-devel. If not installed then
./cofnigure fails with:
checking for libudev.h... no
configure: error:
udev headers required
Change-Id: I53c8f522cbc8eb1e6b461fd78c0bf3305bfb09b9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently we are missing cleanup routine for case when
power failure interrupts creating snapshot. This patch
add such routine.
For the case where we find blob with a parent snapshot ID
matching newly created snapshot we can finish whole process
during recovery by processing forward with setting snpashot
as read only, removing xattr and syncing. We should remove
snapshot only if there is no blob with parent pointing at
snapshot.
Fixes github issue #760
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2f0e298164e07a2b4dfa5367e8878facef640702
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is unit test for power failure event during snapshot
deletion. At the same time this is an example how to
use new power failure event functionality in unit tests.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9212392f665576fa16edd28c609199d0e02dc434
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454608
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Both clean and dirty shutdown tests were expanded to perform additional
writes after restore, to make sure write pointers have been correctly
re-initialised and don't cause data corruption.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I662cf196319a39fb1cab455f5a76571904c20215
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Add test which causes dirty shutdown of ftl and attempts to restore
it. Without persistent cache it expects partial recovery of data
and zeroed out blocks in found open bands.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I316cbbb47d0d03b2c20247df43ba789f2fd018db
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
This patch makes spdk_dif_set_md_interleave_iovs() and
spdk_dif_generate_stream() process unaligned start of data segment
properly by using ctx->data_offset.
Separating this patch into two may be required but this patch is
small and aggregating into a patch is good to test.
UT code demonstrates how it is realized.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idb5250aba4e12a34102e5ce067d725c685681177
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458142
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: Ib9454b1596dc9e0e35dae695f20d6e40dd2c1ab0
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Make it easier to reproduce issue #819 without setting long time.
To decrease the UT time.
Change-Id: I3dd0268a894113e1a12395618721afee6137588d
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Our test destination code of spdk is rsynced from host.
former link address of spdk "https://review.gerrithub.io/spdk/spdk"
is not very fast and always failed .
So changed to "https://github.com/spdk/spdk.git" ,this spdk just for
excuting start testing command.
Change-Id: I26b5c1556d8068becd5c5a94067cc64d9d32b5e6
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459223
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment, and _dif_generate_split will have to process
partial data block, particularly the following types:
- start and end are both within a data block.
- start is within a data block, and end is at the end of a block
According to the refactoring done in the last patch, this patch
exposes offset_in_block, data_len, and guard as parameters of
_dif_generate_split() and make _dif_generate_split() process
the above two types of data block properly.
The next patch will utilize the updated _dif_generate_split in
spdk_dif_generate_stream().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4211e65ead7fc256a40748412c670e46f83b1731
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment. spdk_dif_set_md_interleave_iovs() have allowed
reading data to have any alignment but had required data segment to
be a multiple of block size.
In other words, spdk_dif_set_md_interleave_iovs() had required that both
ctx->data_offset and (data_offset + data_len) must be a multiple of the
data block size.
This patch refines the algorithm to remove the latter requirement.
The update implies that spdk_dif_set_md_interleave_iovs support any
data buffer whose size is less than a single data block.
The update doesn't change parameters of spdk_dif_set_md_interleave_iovs
and existing UT should be passed.
This patch adds additional UT code to test these updates.
Change-Id: I88c7d2a80a8d92b54863b6ad1c3a9d2761a6195d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457542
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This patch adds mechanism to simulate power failure in
blobstore unit tests at any given moment in time.
This point in time is measured by number of IO operations
occured after triggering power failure with the number
of IO operations given as a parameter. Number of IO
operations parameter can be defined as any type IOs,
writes, reads, unmaps, writes zero or flushes.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I7be45331bb5110f6c16f01fd7c4e7984b2d1977d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This bug was found by code inspection. When PDU read restarts
after data segment, iov_offset must be reduced by data length.
However iov_offset had been reduced by buffer length by mistake.
Lack of UT code was the main reason of this bug. Hence add UT
code to test the fix of the bug together in this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I74f2f6ae8dca2e78a64bfdef8080c8031dfabb87
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458530
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Instead of doing some OS-specific magic to detect if the
spdk app has already initialized, just try to send it an
RPC. This serves mostly as cleanup, but also simplifies
test output in cases where the spdk app could not be fully
initialized. Previous waitforlisten implementation would
return as soon as the rpc subsystem was initialized, but
the app could have still failed on e.g. bdev initialization
later on. Since waitforlisten() returned success, the
script could continue its execution and try to execute
some RPCs. The logs from the SPDK app and the bash script
could easily mix, rendering the entire output not so clear
to the user.
To fix it, just try to send some RPC commands inside
waitforlisten().
Change-Id: I33eaf362e3cc645f8ea3ee22fd48db1643442129
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The test used waitforlisten() from autotest_common.sh to
wait for a socat instance and happened to work only by
a coincidence. As we can see in all the messages that
waitforlisten prints, that function is meant for SPDK
applications running an RPC server.
We're about to simplify waitforlisten a lot, so for now
just copy its current version to sock.sh and name it
waitfortcp(). Note that it already had to be stripped of
some rpc references.
Change-Id: Ibbe7cf67d20fbc277d407b20073154a96ddaaade
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458221
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Functions in this file already expect some variables
to be set by the calling script, so do the same with
$testdir and functions from autotest_common. We'll start
using `set -u` soon, so that variable expansion failure
will produce a clear error message (and will trigger ERR).
Change-Id: I3795af498639ad73a99cd03c6b5e72717fe5fd75
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458177
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
autotest_common.sh used to be sourced from
test/spdkcli/common.sh (and it still is - we'll clean
it up in a separate patch). Let's source that file in
each spdkcli test script, just like we do in every
other test script.
Change-Id: I2e021bcda7402ee0c5a4a5f45f921bb0142b6783
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This file is meant to be just sourced somewhere, so
don't include autotest_common.sh inside. Also, don't
hardcode the default pool file path. Expect
$default_pool_file to be defined already. We'll define
`set -u` soon, so that we'll get a clear error if it's
not defined.
Change-Id: Ie9bb86f8a65d05c9fb2d3e56b8682cab8dd569d6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This test defines a variable for "default_pool_file" path,
but keeps using the hardcoded path anyway. Fix that.
Change-Id: I767fdff1eb6a4638a0e028ca8f04bd94582664e7
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Replace them with `testdir` and `rootdir` that we use
in every other test. What was a BASE_DIR before is now
a testdir and what was a TEST_DIR is now a rootdir.
While here, also replace all "$rootdir/test/pmem"
occurences with "$testdir".
Change-Id: I9e5b867bbc481c9c9d6ff25378b6eda60690dd08
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It's always set in autotest_common.sh, there's no need
to set it again in each test script.
Change-Id: Ib14c4189c553dad54a3065c1a1d413a5fc5a5347
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Replace them with `testdir` and `rootdir` that we use
in every other test. What was a BASE_DIR before is now
a testdir and what was a TEST_DIR is now a rootdir.
While here, also move autotest_common.sh include to the
top of the file.
Change-Id: Ie15b7ec14bfb7536bcc7b3787eed930798d24c78
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
In fio.sh we used `set +e` for bdev deletion RPCs
and for `wait $fio_pid`. Disabling errors from those
RPCs doesn't make sense. We expect FIO to fail due
to bdev hotremoval, but the bdev hotremoval itself
must succeed.
Besides, it's easy to forget to undo `set +e` so
don't use it. For `wait` we can use the following
paradigm instead:
```
rc=0
wait $pid || rc=$?
```
The same applies to nmic.sh, where we expect some
RPC to fail.
Change-Id: I66ce3504a0c6b5da497759f0688b6c0ea6480b61
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457463
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
If bdev is configured to use separate buffers for metadata transfer,
but it doesn't support the write zeroes command, pass empty zeroed
buffer to make sure the metadata is cleared out as well.
Change-Id: If6f024266067e5764a28a276296f651d31da4792
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
With this change, bdevio tests are not ran immediately
after starting the application.
This will allow to configure new bdevs via RPC and
run tests on demand during script's execution.
Change-Id: I9740fca92c26cb2a39fbdc7cfb132ad53cec3787
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There was a gap where nbd tests could fail, but without
trap set the app would still be present.
When here removed trap from end of the script since,
no part of the script set it up.
Change-Id: I0c435c7cd603410c2cd7a034bf955ab424e413f0
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457615
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.
In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
4a. Add Clone1 entry to the Snapshot2 clones list
4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
clusters map
9a. If Snapshot2 is present switch parent ID and backing
bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Report issue on ubuntu18 :
"spdk/autobuild.sh: line 55: hash: scan-build: not found"
Check that ubuntu18 should install this package:
apt-get install -y clang-tools, to fit version of ubuntu18
and later.
Move clang-analyzer which for fedora to vm_setup.sh as well.
Change-Id: Ia702c492f8b0f64705c7c15ee57a861ca14521f9
signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
SQSIZE parameter validation in Connect command was broken because QID
field in qpair was used before intialization.
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I8a0b359937d661df3b9888e6084e7d0b4a9056ea
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In a test case of op_login_session_normal_test(), it is expected that
the tag of the session has any value different from the connection but
the tag of the session was not initialized.
Fixes#826
Reported-by: lorneli <lorneli@163.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I62f7a4bd301c5a74ad6ba10b1b448189f750b324
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458339
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This patch changes the meaning of the data_len parameter of
spdk_dif_set_md_interleave_iovs from `Expected data length of the payload`
to `Expected length of the newly read data in the extended LBA payload`.
This change will align the parameters of spdk_dif_set_md_intereleave_iovs
to of spdk_dif_generate_stream.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7f9c45e78be977625713acb79d2ae82d4375f419
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457543
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Previously RPC returned just number of cases that failed,
meanwhile return from the command was 0.
Now whenever there are failures, error is reported via RPC.
This will allow to use it in bash scripts with set -e used.
Change-Id: I34997dd5f08b561bc55f391287f9942861bda57a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457614
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There are some leaks in libiscsi that is external to SPDK.
With this patch, any leaks in libiscsi will be suppressed.
Please see examples of the leaks right now when running
bdevperf in initiator.sh test:
Direct leak of 176 byte(s) in 1 object(s) allocated from:
#0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
#1 0x7fd64c60edb4 in scsi_cdb_inquiry (/usr/lib64/iscsi/libiscsi.so.8+0x13db4)
Indirect leak of 256 byte(s) in 1 object(s) allocated from:
#0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
#1 0x7fd64c603a84 (/usr/lib64/iscsi/libiscsi.so.8+0x8a84)
Indirect leak of 52 byte(s) in 1 object(s) allocated from:
#0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
#1 0x7fd64c60d755 in scsi_malloc (/usr/lib64/iscsi/libiscsi.so.8+0x12755)
#2 0x7fd64c6097f8 (/usr/lib64/iscsi/libiscsi.so.8+0xe7f8)
Change-Id: I80ef23b1464841f683344c7aab99f1658a46cd36
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456766
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ASAN needs to be LD_PRELOADed before SPDK fio_plugin
in order to analyze its code.
Just adding that will report any issues in fio binary as
well as the fio_plugin.
To prevent known fio leaks from affecting the results,
a suppression list for LeakSanitizer (used in conjunction with ASAN).
At this time the suppression list contains known leaks
for fio 3.3. The list might need adjustments as fio
version is updated.
Side note. Even though it is possible to specify directory
to ignore ("leak:/usr/src/fio/"). Which in theory should
suppress any leaks in fio. It has side effect of hiding
SPDK leaks as well, since the fio_plugins leaks are
seen as coming from /usr/src/fio/ioengines.c.
See below for examples of each suppressed error:
Direct leak of 42 byte(s) in 4 object(s) allocated from:
#0 0x7f9d52f3e320 in strdup (/lib64/libasan.so.5+0x3b320)
#1 0x41f267 in get_new_job /usr/src/fio/init.c:490
Direct leak of 914936 byte(s) in 10397 object(s) allocated from:
#0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
#1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214
Direct leak of 608 byte(s) in 19 object(s) allocated from:
#0 0x7f74422e8ca8 in __interceptor_malloc (/lib64/libasan.so.5+0x10dca8)
#1 0x44c4e1 in add_to_dump_list /usr/src/fio/parse.c:1039
#2 0x44c4e1 in parse_option /usr/src/fio/parse.c:1098
Direct leak of 173 byte(s) in 20 object(s) allocated from:
#0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
#1 0x44b50d in __handle_option /usr/src/fio/parse.c:718
Indirect leak of 111925528 byte(s) in 1271881 object(s) allocated from:
#0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
#1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214
Indirect leak of 171 byte(s) in 19 object(s) allocated from:
#0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
#1 0x44c4ed in add_to_dump_list /usr/src/fio/parse.c:1040
#2 0x44c4ed in parse_option /usr/src/fio/parse.c:1098
Indirect leak of 167 byte(s) in 19 object(s) allocated from:
#0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
#1 0x44c502 in add_to_dump_list /usr/src/fio/parse.c:1042
#2 0x44c502 in parse_option /usr/src/fio/parse.c:1098
Change-Id: I9b5811993508421be50b12af160645c77ea93d7e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456315
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The lvol "tasting" done as a part of bdev examine is
done asynchronously in background, so we need to sleep
for a while before checking the list of created lvol
bdevs.
Fixes#801
Change-Id: Iab94ac5c66d329cd93955b53262fc03c6b3c9f3a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Make it able to run as standalone.
Change-Id: Ib8d5dcd300911978f3bd9192c00049a3617438f4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.
Change-Id: Id093b6c82d1f766ba6af13bed720977eceaa7ffc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch adds the UT regarding below function
iscsi_op_login_session_normal()
It covers the target related, connection related
and session related operations.
Change-Id: I570df82df850a0b359b2d57b4e1694d8cb16de37
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457925
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
stay the print parameter same with it in the deallocate_complete().
Change-Id: I21734f2b3097fd3a1006e390ab87b0578f0cbc2a
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458029
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Partprobe is a program that informs the operating system kernel
of partition table changes, by requesting that the operating system
re-read the partition table. For instance, if you create a new
partition on one of your disks using parted, you should run
partprobe afterwards to make the kernel aware of the new partition
configuration.
This is useful for issue #799.And it's compatible with other systems.
Change-Id: Icd4c85193bd9d9e6c2b32b8463e75c7a6ff06f34
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457735
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
NVMe bdev module support separate metadata now but NVMf subsystem
cannot process bdev with separate metadata yet.
Hence reject any bdev with separate metadata to be attached
explicitly by this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I793c6c5f61deb766d7bf427ff67ccc57a48974cf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457167
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Those tests are absolutely awful. For the last few days
they've been randomly failing way more often than usual
and we have no way of figuring out what the root cause is.
We know it's 'discover_bdevs' that hangs, but since it
doesn't produce any output (all redirected to null), we
can't get any relevant information out of it. The only
way to move forward is to rewrite this entire test from
scratch.
Change-Id: Ie874110008f82f3ca0dd62f733b0dac73160fbb8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457766
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We used `set +e` before bdev deletion RPCs and
`wait $fio_pid`. Disabling errors from those RPCs
doesn't make sense. We expect FIO to fail due to
bdev hotremoval, but the bdev hotremoval itself
must succeed.
Besides, it's easy to forget to undo `set +e` so
don't use it. For `wait` we can use the following
paradigm instead:
```
rc=0
wait $pid || rc=$?
```
Change-Id: I8716bd861eae85d7a8590d4bd26a65cfa10e0dc1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
For reservation feature in NVMoF, we can't support the persist through
power loss feature, now we will add the configuration file parameter
with Namespace, after users set the configuration file parameter with
one NS, then the PTPL feature can be enabled.
Change-Id: Id72699093f7e68318b9529f7bacc5c9804f7f86b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch uses the change by the last patch to initialize DIF
context in SCSI layer. Besides this patch changes the name of a
parameter from offset to data_offset to clarify the meaning.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I54bf1168ec5959432aa15dae0360c0640138b033
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Data offset are intended to correspond to DATAO in NVMe/TCP and
Buffer Offset in iSCSI.
Previously for iSCSI, buffer offset had been merged to start block
address, but passing buffer offset separately from start block address
clarifies the logic more.
On the other hand, for NVMe/TCP, passing DATAO separately from start
block address will be critically important because DATAO will bave any
alignment and will be necessary to use for not only reference tag
but also guard computation.
This patch adds data_offset to struct spdk_dif_ctx and adds it to the
parameters of spdk_dif_ctx_init(). ref_tag_offset is also added to struct
spdk_dif_ctx and it is computed by dividing data_offset by data_block_size
and is used to compute reference tag.
The next patch will use this change when getting DIF context in SCSI.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id0e12ca9b1dc75d0589787520feb0c2ee0f844a5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
DIF UT code had used numerics extensibly for complex DIF insert/strip
UTs. But numerics was error prone and difficult to follow the logic.
Hence this patch changes numerics to variables in DIF insert/strip UTs.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I40cb27167c068855dc5d1eb9e0f36818d39c73fa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
data_offset to spdk_dif_generate_stream() was wrong but the function
worked as expected because the wrong data_offset was in the same
data block as the correct data_offset.
But the subsequent patches will refine spdk_dif_generate_stream()
and this bug will cause UT failure. Hence fix the bug up front.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I45f2302deaf377fda0ec5454c61fb463231a39db
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
It's randomly times out on waitforlisten. See #815
Change-Id: Ib9a75d0a96756810cb25f2a1ddf72bc800ea4b5c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Read requested range of lba map during relocation
instead whole lba map.
To read partial lba map range single move now has
three states: read, read_lba_map and write.
Free queue and write queue were merged to a single
move queue.
Change-Id: I86839f2286d42d4debf87cea40091370e5283b15
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454747
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Keep track of read lba map segments to avoid
unnecessary traffic in case ANM events.
Lba map is divided on 4KB segments which can
store 512 lba entries.
In case multipe read request on same segment
keep pending request list and process it in
read_lba_map() completion callback.
Change-Id: I2661bdb716ab7c975140e0b37aebcb17aa23901d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453371
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is to solve below issue:
error 'Fio job '\''/migration-tc1.job'\'' does not exist'
Change-Id: I60dc6254d238878b03c5faf341b3201643286dfe
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This doesn't make difference right now, but would be
useful if we printed $BASH_SOURCE or $LINENO in xtrace
(via PS4). Those two will now point to the original
line where xtrace_restore was called rather than
always the single line inside autotest_common.sh.
Change-Id: Idf3ac8d00ad9610960678351014334013149b88d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Moving verifying test parameters to single function function.
This is start of series that will add RPC to start bdevperf tests,
allowing configuration beforehand.
In future verify_test_params() will be used to verify parameters
passed via RPC as well.
Change-Id: I0aff7d9fa9f99205b3559b0ea081aa3690013faa
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457223
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
For the nvme device, I/Os are completed asynchronously. So we
need to check the outstanding I/Os before putting IO channel
when we hot remove the device. We should be sure that all the
I/Os have been completed when we change the sgroup->state to
PAUSED, so that we can update the subsystem.
Fix#615#755
Change-Id: I0f727a7bd0734fa9be1193e1f574892ab3e68b55
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452038
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We're about to make SPDK vhost work against the upstream
rte_vhost from DPDK 19.05+ and vhost-nvme tests currently
stay in the way. vhost-nvme is only supported in our
modified rte_vhost fork and while it could be implemented
entirely in SPDK against the latest upstream rte_vhost
APIs, there are just no plans to do that now.
For that reason we disable all vhost-nvme tests.
Change-Id: I2d43bbe75dcab7535e7b1254ca75cb78b6d446fa
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Allow the user to select target bdev to be used by bdevperf. It's useful
when the config contains multiple bdevs, but only one is supposed to be
utilized by bdevperf directly.
Change-Id: I51aa645dcf60a4413057d86f68fe24442b280367
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
The target list is now updated inside bdevperf_construct_target. That
way, if the target is successfully created, it'll already be on the
target list once this function finishes.
Change-Id: I364c45005c28deadae37615b6ac85f3860fc5080
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Normal return for qemu checking , this should check before vhost/common.sh.
For qemu not installed on this machine. It may be a VM. Skipping nvmf_vhost test.
Otherwise, code will exit 1 before this normal return.
And in vhost/common.sh should with qemu checking.
Change-Id: I2285fffa6fc4384b4b172e5c443019e9ab391fa0
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456123
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Almost all I/O callbacks in ftl utilize data in ftl_io, which is
initialized as callback argument in ftl_io_init_internal. This patch
changes the behavior to always returning the ftl_io struct in addition
to an extra opaque buffer.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I611ab1b33575f599798a2bb65c231a724c852c7f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455831
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
disable FIO norandommap flag because it is incompatible with
verify which in context of async IO could cause verification errors.
More on norandommap:
https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-norandommap
Change-Id: I94392495a1f9f06360e3791fca31b88908d19dcb
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457001
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We check if get_ocf_bdevs after shutdown is the same as before,
which shows that state for all vbdevs wes correctly restored from metadata.
Change-Id: I9089642aeb840fc940b07908aa39c8193887432f
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455424
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Set serialize_overlap=1 to prevent data races.
This solves the issue of fio --verify fails in OCF tests
when persistent metadata is enabled.
Example:
https://dqtibwqq6s6ux.cloudfront.net/public_build/autotest-per-patch_33325.html
The serialize_overlap flag prevents from
creating undefined regions by doing simultaneus writes
(occurs on certain storages).
More on that:
https://fio.readthedocs.io/en/latest/fio_doc.html#cmdoption-arg-serialize-overlap
We should probably use this flag for all FIO --verify tests.
Change-Id: Ia4f73f9900835faf4f94018f1d148fb64f06dc8a
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456991
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Data block size was specified by mistake, and this patch changes
to use block size instead.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icfb1df40438ef82211279b0b7c79036283744361
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Keep lba map related fields in separate struct directly
in the band. Cleanup interfaces depended on ftl_md.
Lba map structure will be extended in next commit.
Change-Id: I1cfc2f2ff0c0e90bb63f39808780845673002e70
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453370
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Extended ftl_band_read_lba_map() interface to provide
range of lba map to read.
Added ftl_xfer_offset_from_ppa() function for
calculating offset of read portion of lba map
based on ppa address from IO. This offset is
used for coping read lba map portion to internal
ftl buffer.
Change-Id: I1c72a18c79eda8c33cd0b20ea36a5d9521a09d06
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449435
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Further patches in series will modify the suppression file,
so this patch now creates such file in /var/tmp/.
Meanwhile adding the known false positive to the list.
Any known false positives or leaks in external libs/execs
should be added by adding further entries to the suppression
file.
Change-Id: I7b78d900a86c6eca0b41240fb34dc4f7ad597079
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456622
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch adds functions in autotest_common.sh,
fio_bdev() and fio_nvme() for their respective plugins.
It simplifies its usage throughout the scripts.
Additionally will help with expanding the fio cmd
line with nessecary changes only in single place.
This will be used in next patch in series to
LD_PRELOAD ASAN before fio_plugin.
Note: Did not implement changes for perf scripts,
since they are usualy run separately and didn't want
to affect those.
Similarly didn't change vhost initiator tests,
because the exported directories do not work for
inside the VM. Will need to think of a way to change it
there as well.
Change-Id: Idf750ee7aa9d5e30efc0ce430d15fa45ceccb203
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456314
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
SPDK_SUCCESS is a remnant and all other SPDK libraries have used
just 0. So this patch removes SPDK_SUCCESS from iSCSI library and
uses 0 instead.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ie33fd26238411e994ea9ea0c898da1abb502cad6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456928
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Data digest computation should take extended LBA payload but
could not do yet.
This patch adds an new API spdk_dif_update_crc32c() to compute
CRC-32C value for extended LBA payload.
In the next patch, spdk_dif_update_crc32c will be used in iSCSI
target first.
Change-Id: I327f384bb7dfd8b68279b0acec0ee78a40264a26
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This sends an RPC to the vhost target with the given number.
Change-Id: Ifd74443087475242d3a65a0e8621dcd20ab2d1b0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456703
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
No need to prefix name with spdk.
Change-Id: I9b07a52f6c88d77e1dda2c63ec951c1847780cca
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456702
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
No need to prefix name with spdk.
Change-Id: I413f78fac427a0955db495bb926bf8d6bc3dfb2a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We typically don't prefix bash functions with spdk.
Change-Id: Icda57e42acb83f4aa51731ac62f8db7ee26e658a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456700
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Nothing used this anymore. Everything is json.
Change-Id: Iaa1239422e18436b3b8819ce734955683ec63735
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456699
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
DPDK rte_ring_enqueue_bulk() has free_space parameter to return
the amount of space in the ring after enqueue operation has finished.
This parameter can be used to wait when the ring is almost full and
wake up when there is enough space available in the ring.
Hence we add free_space to spdk_ring_enqueue() and spdk_ring_enqueue()
passes it to rte_ring_enqueue_bulk() simply.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b9d6a5a097cf6dc4b97dfda7442f2c4b0aed4d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Fix the following nightly failure:
> migration.sh: line 56: vhosttestinit: command not found
Change-Id: I56135c2a83de2455238eb7d40de973459019d63b
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456758
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This test sends identify controller command to the NVMe device.
Change-Id: Icc262a1a1fa52bafe5cb019151c3d196bd268673
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454609
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Use the new nvmfappstart to simplify this script a
bit and bring it inline with the other nvmf scripts
that have been updated recently.
Also remove the comments about aer with temperature
threshold. We use namespace notifications to test aer
now, so there's no need to try to resurrect the
temperature-based tests.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9c66ed4c9456acb39d6c64d55153304eb6e07100
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456706
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Change autotest.sh to pass --transport=rdma. But this
now allows someone to run nvmf.sh with --transport=tcp
to run all of the nvmf tests with the tcp transport!
Initial testing shows that bdevio.sh fails with tcp,
so only run this test with the rdma transport. Issue
#808 has been filed - once it's resolved this
restriction can be removed.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie50de4d67e504c84d6d8eebfe2b8c68b14f4ba2b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456698
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
While here, remove NVMF_TCP_IP_ADDRESS since we already
have NVMF_FIRST_TARGET_IP which serves the same purpose
when testing the TCP transport.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7cc4712cd9746377937e889127aa5a61566d8846
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We also no longer hardcode TEST_TRANSPORT=rdma. Users
of the individual test scripts must now explicitly
pass --transport=rdma. Support for tcp will be coming
in future patches.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I819d69cb0906eb27b692eb2755aca5085298d779
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456685
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Right now this is empty, but we'll add to it later as
part of parameterizing this script for different transports.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icf0f67ce4dd886031c37986b352129d3c556bd92
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456684
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This prepares for using scripts for tcp testing as well.
Note that this patch just hardcoded TEST_TRANSPORT to
'rdma' for now. An upcoming patch will require the
caller to pass --transport=rdma instead.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I085fdf51910aaf960959c71c73a187be41fd7ee8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A lot of places were still hardcoding 4420 directly.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iff8a549addeb05e441fe3b035a84a0f8a6bf2205
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456666
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This script was developed in parallel to some of the
script infrastructure improvements. So fix it up to
match the more recent changes - including leveraging
the common setup of the NVMF_FIRST_TARGET_IP, and removing
the parameters to nvmftestinit/nvmftestfini.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6cc4ef30325961822014e2835e575a950c53ecc0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456665
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Waits up to 2 seconds for a bdev to arrive, if not returns error.
In the next patch in this series this is used and seems like a good
general function.
Change-Id: I3402f34f3dad3d7373973dc785520a5c4e58cd14
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456091
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
we use this statement:
while (g_file->length_flushed != buf_length) {} in
file_length function.
It means that in this test case, length_flushed are
accessed by two different threads, so better to
use another new variable length_flused with volatile
before the variable definition. Then our ut will not hang.
Change-Id: I6152a4ba3f27f0fad1c8c2baa71324a36a2fb9e8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456580
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There's no need to set this every time we allocate
a request.
While here, fix a typo near where we needed to modify
the unit test to remove the qpair assertion.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8af41a6c483415950f625d1ed2ef46088b75a622
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Use separate buffers for metadata transfer if bdev supports it.
Change-Id: Ie0fa3d3c1f5b14f99c13f2d6b5b22edc216c6d64
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
vring notification mechanism is transport-specific. At present, vhost
dataplane code in `lib/vhost/vhost.c` triggers guest notifications with
`eventfd_write()` system call. But this is an AF_UNIX specific
notification mechanism. This patch replaces `eventfd_write()` with the
existing generic `rte_vhost_vring_call()` function that is part of
DPDK's librte_vhost public API.
`rte_vhost_vring_call()` takes a vring_idx as an argument to associate
the `struct spdk_vhost_virtqueue` instance with the relevant `struct
vhost_virtqueue` instance. We introduce a new `vring_idx` field in
`struct spdk_vhost_virtqueue` to enable this association. This field is
initialized in `start_device()`. In addition, a stub for
`rte_vhost_vring_call()` is added in the vhost unit test file.
SPDK's internal `rte_vhost` copy will not be updated in order to support
the virtio-vhost-user transport. However, an `rte_vhost_vring_call()`
function is introduced in SPDK's `rte_vhost` in order to have a solid
API. This function is just a wrapper of `eventfd_write()`.
Change-Id: Ic93e25cd3f06e92f04766521bc850f1ee80b8ec8
Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Keep an array of iovecs tied to each IO. Internal IOs can have
iov_cnt > 1 without having to allocate additional memory. User's iovec
can now be modified too (e.g. when splitting the request).
Change-Id: Iec73c6dad59acdc331a1461fd7feac085138a8de
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
There's no point in synchronously returning an error from ftl_io_write
/ ftl_io_read as they're also reported in the IO's completion callback.
All errors are now reported through io->status.
Change-Id: I4adf4e13221b63715625276042e1172cd63b8f9b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455521
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Some of the internal IOs split the contiguous data buffer into multiple
iovecs to separate the requests. It's unnecessary and can be replaced
by checking the position within the data buffer.
Change-Id: I9255ea0072fee6c4e6a7dca21e4008780bc45610
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This script was getting a bit unwieldy. adding argparse will make it
easier to add more options in the future or to set default values and
validate arguments going forward.
Change-Id: I1ffdbdf2082287ceb8a88cd3eb6ecf9bbd6c9e11
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455724
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
This isn't needed, and it actually results in warnings
when linking with LTO enabled.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e7dcd5d3f738fb3e53e35a1fff154a0e1ab06d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456238
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Existing code in spdk_bdev_write_zeroes_blocks() will call spdk_bdev_free_io()
for the error case, which will cause assertion because the bdev_io isn't
submitted to the backend yet, so we will check the condtion first to
avoid the error case.
Change-Id: If27d78217f709a3315e74c00869d345abd6b9a69
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This module simply sits on another virtual bdev
and adds a simulated average and p99 latency to that drive.
Change-Id: Ie9fc91e27585fd0636cb7dc845cb41744bf24625
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
When a new python script wanted to use RPC client or calls
from RPC lib from ./spdk/scripts/* it had to be created
within that directory.
By expanding PYTHONPATH in autotest_common, now any python
scripts can use that library.
First example of this is located in:
./spdk/test/bdev/bdevio/tests.py
Change-Id: Idf3e5891c3815a84c70525ab9338100acbfa4a43
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455219
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
New RPC added specifically to the bdevio app.
It will launch all tests on all targets by default.
When optional parameter for bdev name provided is supplied
tests will be ran against just that bdev.
After finishing, the application will await another RPC.
stop_init_thread() and __run_ut_thread now pass json request.
Added separate test.py to not pollute rpc.py with custom RPC
just for the bdevio application.
Added '-w' argument to the bdevio app, so that instead
of performing the tests immidietly - it awaits RPC.
After a lot of changes to blockdev.sh, we might end up
removing this argument and just use it as default.
Change-Id: I82e52352bdf8082c1712caa223ad5ab78aa4e7fa
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455200
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
All the SCSI commands have two states: allowed and conflict,
based on different reservation types and I_T nexus, the SCSI
module will check each commands before sending to the backend
device.
Change-Id: Ib05cece1c237873360f85c68d139362200d2d47e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
NOTE: pulling the vbdevcompress change into its own patch...
While enabling reduce testing in autotest a few issues were found
in autotest in that some options were being set based on the existence
of a header without checking the test flag, if one existed. Then at
the end of adding options, specific test flags were checked for 0 and
the configure updated to --without. So, to make this easier to read,
the options are organized in sections and where there is a header
dependency as well as a test flag, both are chcekced at the same time.
Change-Id: I8cdc6914906500bbac63528eb541e2b235feb797
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455137
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add the unit test of the nvme ctrlr init delay to increase
the code coverage.
Change-Id: I0d31055fafc1ad13682496163e15de0a5f1e90e3
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
After iSCSI login sysfs is not updated immediately, but when session is
initialized. It can take a while and causes that sometimes scripts
cannot write to sysfs.
Change-Id: Ie4fa512704f022041bb3d9ab02dbb4210ddd04a0
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
rootdir is the pattern used in the other tests
Change-Id: Idaedf8a3d9aeed54a9113a38986f29fa04355d9a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454797
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The vhost tests create some temporary files as they run outside
of the SPDK repostiroy. Clean these up.
Change-Id: Ic964fc6672b8a4ea088ace1b11c712eb70f7be08
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454933
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Ubuntu18 integrates librxe to rdma-core, libibverbs-dev no longer ships infiniband/driver.h.
Don't compile librxe on ubuntu18, install package rdma-core instead. ubuntu16 keeps the old
method. Otherwise, there's no NIC can be found for SoftRoCE failed.
Change-Id: Ib639b96a4229c79f2b27fda7b981d7a805f808cb
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455336
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Before simplifying test suite execution it was needed
to loop around function actually running the tests.
So the test functions now can directly contain whatever
that was doing, without intermidiate function between.
Change-Id: Ic4b648d8d26d6be7a3bf1efb1b1ee3f2615ee5fa
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455672
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
The QoS test no longer sets arbitrary QoS limits and
can no longer run into cases where disk is not able to
even reach that limit. We can reenable it for per-patch
testing now.
Change-Id: If05b1306c38889840c071a165b00c44295610950
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454670
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
check_qos_works_well() has been split into two separate
functions: run_fio() and verify_qos_limits().
Change-Id: I8c112ab93aed9f16d80c9a0a3f61280c44bf6539
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451888
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We used to set an arbitrary qos limit which in some
cases happened to be higher than the actual disk
capabilities. Even though we had an explicit check
for that and we skipped the entire qos test suite
if the device was too slow, the disk performance could
vary and be just enough to pass that initial check,
but then slow down and fail in the middle of the test
suite. If the bdev maxes out at 21MB/s on one run, it
may just as well do 19MB/s on another. That is exactly
the case causing intermittent failures on our CI.
We fix it by removing the arbitrary qos limit and
setting it to a % of the maximum disk performance
instead. This lets us e.g. remove the code for skipping
the entire test suite when the disk is too slow. We
definitely don't want to skip any tests.
Change-Id: I6de8a183c00bab64484b4ddb12df1dedfbed23f8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451887
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Make the old name a deprecated alias.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibbf50676e0d989b67121e465fc140f94faec46ed
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
These are used when running test scripts in iso mode at
your desk.
Change-Id: I1876771d8a94c9db57636c143f7bf9d6f98bd3aa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
With this patch each target has separate suite,
tests are ran just once for all setup suites.
Suite init/fini functions were added that move
current g_current_io_target ahead for next suite.
Results what works for particular bdev are of more use
rather than combination for all bdevs.
This will come useful when we want to perform RPC for only
select targets.
In case of CUnit library error, application is stopped without
returning number of failures. This is because at that point
its numbers are not reliable.
Change-Id: Ie82b6e33374841bf829da09a16a401fddd1fd81c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
We flush a cache buffer once it's filled. When the write
for that cache buffer has completed, we look to see if
there's more data to flush. Currently if there's *any*
more data to flush, we will flush it, even if it's not
a full buffer.
That can hurt performance though. Ideally we only want
to flush a partial buffer if there's been an explicit
sync operation that requires that partial buffer to be
flushed. Otherwise we will end up writing the partial
buffer to disk, and then come back and write that data
again later when the buffer is full.
Add a new unit test to test for this condition. This
patch breaks one of the existing unit tests which was
designed specifically around a RocksDB failure condition.
Change that file_length unit test to now write exactly
one CACHE_BUFFER, which still tests the general logic
making sure that we don't confuse the amount of data
flushed with the value written to the file's length
xattr.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I83795fb45afe854b38648d0e0c1a7928219307a2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455698
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Data can get implicitly flushed as cache buffers are filled. But
the length xattr is only written in response to a sync or close
operation. So we cannot just look at the amount of data flushed,
and ignore the sync operation if all of the data written has been
flushed - we still need to write the length xattr.
This also adds a unit test which reproduces the original problem.
Fixes issue #297.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icca6ef4d1544f72e9bc31c4ee77d26b4b7f0cce4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
We don't provide windows executable for this test.
Change-Id: Iedaafba06662f776c102528f8073b6c3d56bb323
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450406
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Recently, we moved the check for running an installed version of DPDK
above the point where we export the SPDK_* test flags. This resulted in
SPDK_RUN_INSTALLED_DPDK being undefined when trying to check fi we
should run against an installed DPDK. This just caused autotest_common
to print an error message every time it was loaded. I don't think that
it actually caused any errant behavior but this gets rid of the error
message. It's also probably a good idea to keep these definitions as
close to the top of the file as possible.
Change-Id: I5aadbe5c925ecf1ac92926b75c8c043aab73b36b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For the latest TLC NAND, one write buffer unit (rwb batch)
needs to be spread over three PUs instead of being allocated
to a single PU for better sequential read performance
since the optimal write size(ws_opt) of 3D TLC NAND is
3 times bigger than the optimal read size(rs_opt).
I added num_interleave_units in 'struct spdk_ftl_conf'
to configure the number of interleaving units per ws_opt.
If num_interleave_units is set as 1, the whole of the ws_opt
blocks are placed sequentially around each PU.
If num_interleave_units is set as N, the 1/N of the ws_opt
blocks are staggered. So consecutively numbered blocks
are separated by ws_opt / num_interleave_units.
The sequential read performance is improved from 1.9GiB/s
up to 2.97GiB/S with this patch on our system. No performance
degradation is observed on sequential writes or
4KB random reads/writes.
Please refer to the Trello card for more details.
https://trello.com/c/Osol93ZU
Change-Id: I371e72067b278ef43c3ac87a3d9ce9010d3fcb15
Signed-off-by: Claire J. In <claire.in@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450976
Reviewed-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In some cases user may want to flag blob for removal
then do some operations (before removing it) and while
it happens there might be power failure. In such cases
we should remove this blob on next blobstore load.
Example of such usage is delete snapshot functionality
that will be introduced in upcoming patch.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I85f396b73762d2665ba8aec62528bb224acace74
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
To establish a persistent reservation the application client shall first
register an I_T nexus with the device server. An application client registers
with a logical unit by issuing a PERSISTENT RESERVE OUT command with REGISTER
service action or REGISTER AND IGNORE EXISTING KEY service action.
Specify Initiator Ports (SPEC_I_PT) bit and All Target Ports (ALL_TG_PT) bit
are not supported for now, the registrants belong to the I_T nexus who sends
the command.
Also Activate Persist Through Power Loss (APTPL) bit will be supported in
following patches.
Change-Id: If057a764a4bfa73017f98048c94b697dbfa4b4a1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/436088
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Good suggestion from Darek - let's just always
parse common script args from autotest_common.sh.
These arguments follow common arg naming conventions
(i.e. --iso) so there's no harm just doing this for
any test that sources autotest_common.sh. This has
the nice effect of not requiring scripts to
explicitly call this function.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id89b68c22557a5a771be407873d0e57843f0d05a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455552
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Modify FIO test to use all cache modes that we support,
including WriteBack
New test config uses Nvme bdevs instead of mallocs because
memory is an issue when testing OCF
Change-Id: I3abec9605b61791f8ebaaaf08b88a011a50d3f26
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tests for checking read-only option no longer fail.
Removing rootdir variable and sourcing commands as
*_vm.sh scripts are designed to be run as standalone
inside spawned VMs.
Change-Id: I491ebf8e2b305539e3fabdc6219ea53693ad2883
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455203
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
iscsi test scripts can now take two arguments -
"iso" and then the sock type (posix or vpp). They
need to be in that specific order too. nvmf test
scripts also support "iso" and we want to add
the transport type (rdma or tcp) as well. Even further
out, we may want to use a sock type for nvmf, i.e.
tcp transport with vpp.
We also have the iscsi_tgt fio_remove_nvme.sh test
that does both iscsi and nvmf.
So to make this all work a bit nicer, add a new
function called parse_common_script_args that
will take the command line arguments to a script
and set the appropriate variables, including defaults
when a specific parameter isn't specified. We will
use getopt-like behavior for this also, instead of
enforcing a specific parameter order. Then a script
could be called like this:
test/nvmf/target/shutdown.sh --iso --transport=tcp --sock=vpp
Individual test scripts then just need to do this
after sourcing autotest_common.sh:
parse_common_script_args $@
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifb8d7666384991482a2d425e26ffa7525b9ac15a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Apart from writing the data to OCSSD, mirror the latest two bands of
data on the persistent write buffer cache. Currently the data is only
sent there, further patches will add metadata support, shutdown recovery
and L2P updates.
Change-Id: Ief05d0c23fa0e25bd6085e0ce3e1528d6736d174
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450266
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Previously it was explcitly set to NULL, so that
the application didn't even setup up the socket.
Change-Id: I2174fe0ff5790efd6578807f17702978cd9cf451
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455197
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Adding app_rpc and bdev_rpc allowing every kind of RPC
interaction with the app.
Change-Id: I738095686dc8ad61101a998b473df63cddc490a2
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455196
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Installing this package "open-isns-utils" always with pop-up dialogs,
we don't want to these dialog, so add --no-install-suggests
--no-install-recommends to prevent it.
Use sudo apt-get remove --purge * to simulate.
Change-Id: I0b892b9e0c88c82ab9461a92e71dc0d9823ecaf9
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adding these tests identified a small fix in the code under test
that is also included here. The compressdev 'produced' field
is unsigned and reduce is expecting a negative errno in the
callback.
Change-Id: I28ab11ee3ef54768a9d6ccd26282cf7dd022be43
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454806
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
mk/nvme.libtest.mk includes mk/spdk.common.mk, but all
of the Makefiles including mk/nvme.libtest.mk were
also including mk/spdk.common.mk unnecessarily. So
remove the spdk.common.mk include from all of the
offending Makefiles.
This was relatively harmless, although it would cause
weird things like CFLAGS and LDFLAGS getting duplicated
when building.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie60637db3c19a2ead097562b2adf6573dbe27472
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This avoids conflict with the nvme perf tool. PGO
gets confused during building - we may have data for
nvme/perf which it then tries to use when building
ioat/perf. Renaming the ioat perf tool fixes that
problem.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1084d56d671e44027ea05f453075a723f067580
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455320
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There's a bug in Linux 5.1 (possibly 5.0 too) where
the kernel initiator driver crashes if it can't get
one queue per CPU. This will get fixed eventually,
but for now we need to remove the cases where we
restrict the number of queues per controller so that
we can test on newer kernels.
Even on cases where we're testing the SPDK initiator,
there's no real need to restrict the number of queue
pairs.
The kernel will eventually get fixed, but we should
be testing with default behavior anyways (the kernel
wants lots of queues). We'll also want to add some
regression tests to make sure the kernel doesn't
break again. But that will all come later.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9979e6d94456e075688b822b042936b63e518a4a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454819
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Don't do it at runtime, do it just once on script initialization.
Change-Id: Idb345fc0f72d3a41072d830b11a520584ec8b321
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451886
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Do a single RPC instead of two, cache the result,
parse it twice.
Change-Id: Ib0ae19da5cb6b6db7bf1a46c3960104d1975afdd
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451885
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When parent IO was splitted into several children requests, SPDK
may return parent completion callback with error status before
all the children requests are finished.
Change-Id: I63221a0ae1a5925a7fcd9744b4f5d8079c641252
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Lightweight threads may now be exited by calling
spdk_thread_exit() within the thread. The framework
polling the thread can release the resources associated
with that lightweight thread by calling
spdk_thread_destroy().
Change-Id: I6586b9d22556b3874fb113ce5402c6b1f371786e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Rather than revert this patch, just modify the path to the common script
to enable it to pass in the test pool.
Change-Id: I33f5d89e1e118df4546f45237b55173ad30fad24
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
