This patch prepares environment to allow compile applications and
libraries with VPP.
Change-Id: Icc067c17b57932062afa5e6a67791bd58de3a2cc
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We have no guaranty that iscsiadm -m session shows all required iscsi
devices at some point of time after login. It may cause that not all
devices are used in further tests (e.g. in fio).
This patch allows to wait for a certain number of sessions after login.
Change-Id: I868cb8f1235dd04d88572bc8518e97de99a2c5e6
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458236
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Bdevperf up to now didn't use unregister callback when opening target
block deice. This is unsafe if test scenario involves target bdev
unregistering voluntarily, although with recent bdev layer update
not registering hotremove callback has become safer.
To be on the safe side, this patch registers a callback to transit
gone target into the I/O drain mode in case the target unregisters
during runtime.
Changes wrt previous versions of the patch:
- incorporated review comments
- updated callback functionality
- updated patch description.
Change-Id: I0dad2f4a7cf4acb22b363e23e4b670ca73d8c6f2
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447956
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_dma_*malloc() is about to be deprecated.
Change-Id: I6c308ee546c28c479ceb903bc1749bf5209dc6fe
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <uma.willpower@gmail.com>
spdk_dif_generate_stream() and spdk_dif_verify_stream() are very
similar. Factoring out the common part into a function will improve
the maintainability and do in this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I16ecd0860c75037d9182298d7513749dfe8e9b56
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458376
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add spdk_dif_verify_stream to verify DIF by stream fashion.
spdk_dif_verify_stream utilizes the updated _dif_verify_split.
spdk_dif_verify_stream is very similar with spdk_dif_generate_stream().
UT code demonstrates how it is realized.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1c5d197cf4c0bbc82c8e7f4fa45ddc0b94051058
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458330
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment, and _dif_verify_split will have to process
partial data block, particularly the following types:
- start and end are both within a data block.
- start is within a data block, and end is at the end of a block
On the other hand, _dif_verify_split had assumed that passed block
is always a complete block.
According to the refactoring done in the last patch, this patch
exposes offset_in_block, data_len, and guard as parameters of
_dif_verify_split() and make _dif_verify_split() process the
above two types of data block properly.
The next patch will utilize the updated _dif_verify_split to
add spdk_dif_verify_stream().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic371d3ccefbd5fe8147a948a624013be2702128e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458329
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment, and _dif_verify_split will have to process
partial data block, particularly the following types:
- start and end are both within a data block.
- start is within a data block, and end is at the end of a block
On the other hand, _dif_verify_split had assumed that passed block
is always a complete block.
To process the above types, separating guard computation, DIF copy
and skipping metadata field, and DIF verification into three parts
will be helpful and is done in this patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic4f1765e01507efa812dfaf7a8018666c6346f8e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458328
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Some of the errors were silent, making it hard to pinpoint the exact
failing call. This patch adds SPDK_ERRLOGs for each error path.
Change-Id: I71be6c97cab916ac52314e5f4e4d63358877bd96
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Send LBA along with the data block when mirroring writes to the
non-volatile cache. The metadata buffer is retrieved from the metadata
pool, so the maximum number of concurrent requests is limited to
nv_cache.max_request_cnt, while the number of blocks in a single request
is limited by nv_cache.max_requets_size.
Change-Id: If260302d16039183fb0fe073ef7419947532cfab
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458093
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Initialize the memory pool for storing metadata (LBAs) when writing data
to the non-volatile cache. The mempool's object count and size can be
configured via nv_cache.max_request_cnt / nv_cache.max_request_size
respectively.
Change-Id: I376df9a75be13d4b29ba475f350edf402c868d48
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
As have updated the NVMe reset case, the check_io() must be executed
after ctrlr reset. It's easier to reproduce #819, so we can set a
short time.
Change-Id: I9ba867a6c2aa1a3921bace2e77acd28a069d9016
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457761
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Address translation wasn't correct for >32 bit length packed address.
This commit fixes the issue and adds a corresponding unit test.
This patch fixes issue #774:
https://github.com/spdk/spdk/issues/774
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Idce67c47f2a9888f9e2ae2eadaf71ccc34e5c260
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457114
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Updated doc to include section on how to compile SPDK with link
time optimization (lto) gcc flag prior to running the benchmark.
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Change-Id: I1af4033144dd9f7594bcf40621acb7fb52018e7a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459330
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added spdk_pci_get_first_device() and
spdk_pci_get_next_device() to iterate
over all devices on g_pci_devices list.
Change-Id: I65079fb3e274195707dee64bc1fb8b4b72d07352
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Put the locks inside cleanup_pci_devices().
This serves as cleanup.
Change-Id: I040b28006e5584d1f33af26b63cafedbafe04fdb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458934
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
The global pci tailq is no longer modified on the dpdk
thread, so on the spdk thread we can access it safely
without any lock. The code is slightly more readable
then.
This shows that cleanup_pci_devices() is always wrapped
with lock/unlock. We'll put the locks inside this
function in the next patch.
Change-Id: Ia4d386b78a87078761df0a3b953bfc4ff44102f8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458933
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
To safely access the global pci device list on an spdk
thread, we'll need not to modify this list on any other
thread. When device gets hotplugged on a dpdk thread,
it will be now inserted into a new global tailq that
can be accessed only under g_pci_mutex. Then any
subsequently called public pci function will add it to
the regular device tailq.
Change-Id: I9cb9d6b24fd731641fd764d0da71bedab38824c9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
To safely access the global pci device list on an spdk
thread, we'll need not to modify this list on any other
thread. When device gets hotremoved on a dpdk thread,
it will now set a new per-device `removed` flag. Then
any subsequently called public pci function will remove
it from the list.
Change-Id: I0f16237617e0bea75b322ab402407780616424c3
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458931
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
For VMD driver we'll need to introduce some way of
iterating over all spdk pci device objects and we would
like to achieve that with simple spdk_pci_get_first_dev()/get_next_dev()
APIs. To make it thread safe though, we would have to
expose some public pci mutex to be locked around the
iteration and we don't want to do that, so we'll make
PCI APIs usable from only a single thread - this will
prevent any pci devices from being removed inbetween
subsequent get_first/get_next calls.
We currently have the following players accessing pci
device state:
1) public APIs, obviously (on any thread right now)
2) VFIO hotremove callback (dpdk interrupt thread)
3) rte_eal_alarm for detaching rte_pci_devices (dpdk
interrupt thread)
4) DPDK hotplug IPC (dpdk interrupt thread)
There is g_pci_mutex providing the thread safety, but
even today it doesn't protect #3 and #4, making the
entire pci layer prone to data corruption.
To make #3 and #4 safe, we would have to lock inside
device init/fini callbacks (spdk_pci_device_init/fini),
but those are called directly inside the public device
attach/detach functions which already lock.
So now, with the decision to drop thread safety from
public pci APIs, we narrow down the locks inside public
functions and introduce locks inside those lower-level
init/fini callbacks.
Change-Id: I5dcbc9cdcbab65ee76cd3c42890f596069ec9a8a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Scanbuild error on fedora30 reports:
warning: Array access (via field 'write_buf') results in a null pointer dereference
if (context->write_buf[i]) {
In deallocated_value.c, cleanup function be used at many places to deal with failed
cases even context->write_buf is NULL, so add context->write_buf pointer check before
array data. I think context->read_buf is the same.
This is related to issue #822.
Change-Id: I33c685fd732da820c1dfc861eb991b92b41caa29
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458736
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Missing dependency libudev-devel. If not installed then
./cofnigure fails with:
checking for libudev.h... no
configure: error:
udev headers required
Change-Id: I53c8f522cbc8eb1e6b461fd78c0bf3305bfb09b9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459323
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Converting to the struct virtio_req is useless.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I141268314d28cf87bdef529808c8e18bd1b41c9d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459360
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently we are missing cleanup routine for case when
power failure interrupts creating snapshot. This patch
add such routine.
For the case where we find blob with a parent snapshot ID
matching newly created snapshot we can finish whole process
during recovery by processing forward with setting snpashot
as read only, removing xattr and syncing. We should remove
snapshot only if there is no blob with parent pointing at
snapshot.
Fixes github issue #760
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2f0e298164e07a2b4dfa5367e8878facef640702
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is unit test for power failure event during snapshot
deletion. At the same time this is an example how to
use new power failure event functionality in unit tests.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9212392f665576fa16edd28c609199d0e02dc434
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454608
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
By default QAT will be selected if available however a new RPC
can be used to either auto-select (default) or specify either
ISAL or QAT.
Change-Id: I37cf7640bbd8cef455583e1eccb8adb59cc419d8
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456693
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Fixed issue when restoring from a dirty shutdown - sometimes end md
wasn't erased after a band was prepared from writing when a shutdown
happened. This resulted in inconsistency between the new head md and
old tail md, which was technically valid. Band sequence numbers would
then be reused, causing a failure on any subsequent restore.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic3e968be02bb814d6c85f0a3279403fe99337b86
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Both clean and dirty shutdown tests were expanded to perform additional
writes after restore, to make sure write pointers have been correctly
re-initialised and don't cause data corruption.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I662cf196319a39fb1cab455f5a76571904c20215
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Add test which causes dirty shutdown of ftl and attempts to restore
it. Without persistent cache it expects partial recovery of data
and zeroed out blocks in found open bands.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I316cbbb47d0d03b2c20247df43ba789f2fd018db
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Changed to use 4k alignment in dirty shutdown I/Os. Otherwise the
scatter gather lists used in QEMU for underlying file/block device
would use an extra entry (e.g. 17 for 16 sector writes), and eventually
some I/Os would write to offset 0 in underlying file, corrupting head
metadata.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: If8c88ce708529b094a09c8ee952912cc22cd53b9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458090
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Band's lba_map needs to be set to NULL before restore completes, as
it's not allocated on a per band basis and instead uses a pool from
restore struct itself. Without the fix initializing a band for writing
would hit an assert during proper allocation in ftl_band_alloc_lba_map.
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Icff4f54cbe722cb6030b9dfd55726b9b0d6c1e27
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458422
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch makes spdk_dif_set_md_interleave_iovs() and
spdk_dif_generate_stream() process unaligned start of data segment
properly by using ctx->data_offset.
Separating this patch into two may be required but this patch is
small and aggregating into a patch is good to test.
UT code demonstrates how it is realized.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idb5250aba4e12a34102e5ce067d725c685681177
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458142
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
To process unaligned data segment properly when a whole data buffer
is splitted into multiple data segments and each data segment has
any alignment, we have to update only data offset of DIF context
according to the progress.
Hence this patch adds an new API spdk_dif_ctx_set_data_offset().
The API will be used in the next patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I346ab583518b80792ea40d34cf0c8536ecc3d904
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458141
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: Ic725c8e750e6505e5cee8e8889f3cc8c0342eb51
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
VMD section with Enable flag set to true
need to be defined in config file to
enumerate devices behind VMD.
Change-Id: Ib9454b1596dc9e0e35dae695f20d6e40dd2c1ab0
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There're outstanding requests in spdk_jsonrpc_parse_request which caused by
connection close.
There are methods to call spdk_jsonrpc_server_conn_close, including
spdk_jsonrpc_server_conn_remove and spdk_jsonrpc_server_shutdown,
Some rpc methods call these functions to terminate connections ,that leads to
memory leaks.
Try to free outstanding requests after deciding to terminate a connection.
And do this follwing with close(conn->sockfd).
Fix issue #784, and can resolve other similar memory leaks about this.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Icd287bd0c5670ee8ec32750b999f82b0fa89cf84
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
By now (5.1 is released), the Linux kernel initiator supports the
success optimization and further, the version that doesn't support
it (5.0) was EOL-ed. As such, lets open it up @ spdk by default.
Doing so provides a notable performance improvement: running perf with
iodepth of 64, randread, two threads and block size of 512 bytes for 60s
("-q 64 -w randread -o 512 -c 0x5000 -t 60") over the VMA socket acceleration
library and null backing store, we got 730K IOPS with the success
optimization vs 550K without it.
IOPS MiB/s Average min max
549274.10 268.20 232.99 93.23 3256354.96
728117.57 355.53 175.76 85.93 14632.16
To allow for interop with older kernel initiators, we added
a config knob under which the success optimization can be
enabled or disabled.
Change-Id: Ia4c79f607f82c3563523ae3e07a67eac95b56dbb
Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Make it easier to reproduce issue #819 without setting long time.
To decrease the UT time.
Change-Id: I3dd0268a894113e1a12395618721afee6137588d
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457914
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
If users set the persist through power loss configuation file,
that means the Namespace has the capability to support ptpl
feature, here we added a ptpl_activated flag to indicate that
the users enable the feature or not. Users can use Set features
or Reservation Register commands to change the value.
Change-Id: Iae3fd44085c5be5bf9574e49efa567e8212dee20
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Our test destination code of spdk is rsynced from host.
former link address of spdk "https://review.gerrithub.io/spdk/spdk"
is not very fast and always failed .
So changed to "https://github.com/spdk/spdk.git" ,this spdk just for
excuting start testing command.
Change-Id: I26b5c1556d8068becd5c5a94067cc64d9d32b5e6
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459223
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Vhost testing crashed from Nightly testing, because a member
access within null pointer of type 'struct ibv_send_wr'.
Change-Id: If8f34f23864883ea73516d2d1fe3b30137c04316
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458913
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Removing Fedora 26 and 27 which are long past their EOL date.
Adding Fedora 29 and 30.
Change-Id: I35bc8b7a5ac693b7a5af0e6abb35869b18ed5777
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457924
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When JSON config was used, app layer was calling the
app start callback twice - once from internally-sent
"start_subsystem_init" RPC, and once from the app layer
itself.
In case of JSON configs, the callback from within the
RPC was actually called prematurely, as the real RPC
server was still starting in the background at that
point. We still need to start the app from that RPC in
case of `--wait-for-listen` option, but for JSON configs
it doesn't make sense. Just ignore it now and rely on
json config load completion callback to start the app.
Fixes#816
Change-Id: Ib54d624f3167137216c910b2d947bbd1dc5023b1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458351
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If reading the JSON config file has failed, we entered
spdk_app_json_config_load_done(-ERRNO) and tried to
close a client connection that was never initiated,
which resulted in NULL dereference.
To fix it, just check if client_conn != NULL before
attempting to close it.
Change-Id: I7340567c45e795f77110c2914e94ba83fa8d1bff
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458350
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
free resources when error was triggered in nvme_manage
Change-Id: Ibdc020ffb5abd3d614fe85e54788b1818ef5af1e
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458342
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Added new VMD subsystem to enumerate
devices behind VMD when event framewrok
is used.
To enable VMD, user need to provide Enable
flag via config file.
Change-Id: I89bfe22b127c00d358dac7336ffb44b0c0f426ea
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458443
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Warning: taking address of packed member of 'struct spdk_nvme_health_information_page'
and 'spdk_nvme_ctrlr_data' may result in an unaligned pointer value [-Waddress-of-packed-member]
So add __attribute__((aligned)) following with __attribute__((packed)) to avoid this kind of
GCC9 warning.
This related to issue #822.
Change-Id: I6117c1e50a137d29cd60ebbad5c15d9093d21670
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458535
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
NVMe/TCP target may split a whole data payload into multiple H2C
or C2H PDUs with any alignment. Hence to insert or strip DIF correctly
to the split H2C or C2H PDUs, we have to bring the interim guard
value of the last partial data block of the current H2C or C2H
PDU to the first partial data block of the next H2C or C2H PDU.
So we add last_guard to struct spdk_dif_ctx and use it in
spdk_dif_generate_stream().
API spdk_dif_generate_stream() is not changed and UT code should
pass without any change.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I12636c5ac7f619483402538faff4339a16c0e6b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
For NVMe/TCP target, data segments which correspond to H2C or C2H PDU
will have any alignment, and _dif_generate_split will have to process
partial data block, particularly the following types:
- start and end are both within a data block.
- start is within a data block, and end is at the end of a block
According to the refactoring done in the last patch, this patch
exposes offset_in_block, data_len, and guard as parameters of
_dif_generate_split() and make _dif_generate_split() process
the above two types of data block properly.
The next patch will utilize the updated _dif_generate_split in
spdk_dif_generate_stream().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4211e65ead7fc256a40748412c670e46f83b1731
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457544
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
