2073 Commits

Author SHA1 Message Date
Liu Xiaodong
8aaa707981 util/base64: add base64 lib and unit tests
RPC method (nvme-passthrough-cmd) needs base64 to
transfer binary buffer of nvme-cmd through Json text.

Change-Id: I50b9d0efca97da45d6d4fd2a1b53b74811947191
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417805
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-19 00:50:54 +00:00
Pawel Kaminski
46412580ee test/doc: Remove snapshot and clone test plan.
Move logical volume tasting test plan into script.

Change-Id: I2da9ab9348a01df86489c81b6debea4f564cc318
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/419383
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-17 20:57:27 +00:00
Tomasz Kulasek
d3c6335bc9 blobstore: no copy write in thin-provisioning
This patch prevents to copy cluster data when there is not
backing blob to improve cluster allocation performance
in thin provisioned blobs.

Change-Id: Ie766d2e5274daa74c2b13b2198a20205e3417467
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/417938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-17 17:44:13 +00:00
Tomasz Zawadzki
6fc44a7aea blob: fix unallocated clusters to run-length encode
Before this patch when serializing extents,
unallocated clusters were treated as separate lba.
This caused metadata to grow without need.

Change-Id: I5d66466dda5f5e6d4d53f4ed5bd0bac18c74be96
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419180
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-17 17:43:57 +00:00
Shuhei Matsumoto
cefab5d647 iscsi: Support hot removal of LUN based on LUN open/close
When hot removal of a LUN is started, callback is called for each
iSCSI connection which accesses the LUN. Callback checks all transfer
tasks complete and then close the LUN.

If the connection clears all transfer tasks before getting all responses
to them from the initiator, the initiator continues to retry data write.
Hence the connection have to wait until all transfer tasks complete.

Change-Id: Iad9063673cfedbd78758890d55a4254512e4fca4
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-17 17:43:28 +00:00
Shuhei Matsumoto
fec7c88e45 scsi: Enforce to open LUN and then alloc IO channel by using descriptor
Use a descriptor got by opening the LUN to allocate IO channel of the LUN.
This requires opening the LUN is done before allocating IO channel of the
LUN.

Use the descriptor to free IO channel of the LUN too.

Additionally, assert is added to the close LUN function to check if
IO channel is freed before the last close LUN function is called.

Change-Id: Iafb2f9ce790fff25801ea45b3286f3e26943807b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-17 17:43:28 +00:00
Pawel Niedzwiecki
767202e3ac test/json: Tests for iscsi subsystems.
Change-Id: I7053dca9cc6f2ef09566ba2bf8338f7f4c92ebf0
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/416745
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-17 17:42:06 +00:00
Ziye Yang
1e2b25ec20 bdevperf: fix the reset testing failure issue.
Root cause: we did not check the status of the bdev_io
in bdevperf_verify_write_complete function.

So the fix is that: if the status is failed, we should
not do the read check again, but just complete this I/O.

Change-Id: I3dc796abe47c3f2e7eeb2b6eb0ebc0338122479d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/419338
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-17 01:30:40 +00:00
Kunal Sablok
41586b0f1d bdev: add raid bdev module
Raid module:
============
- SPDK raid bdev module is a new bdev module which is
  responsible for striping various NVMe devices and expose the raid bdev
  to bdev layer which would enhance the performance and capacity.
- It can support theoretically 256 base devices (currently it is being
  tested max upto 8 base devices)
- Multiple strip sizes like 32KB, 64KB, 128KB, 256KB, 512KB etc is
  supported. Most of the current testing is focused on 64KB strip size.
- New RPC commands like "create raid bdev", "destroy raid bdev" and "get raid bdevs"
  are introduced to configure raid bdev dynamically in a running
SPDK system.
- Currently raid bdev configuration parameters are persisted in the
  current SPDK configuration file for across reboot support. DDF will be
introduced later.

High level testing done:
=======================
- Raid bdev is created with 8 base NVMe devices via configuration
  file and is exposed to initiator via existing methods. Initiator is
able to see a single NVMe namespace with capacity equal to sum of the
minimum capacities of 8 devices. Initiator was able to run raw
read/write workload, file system workload etc (tested with XFS file
system workload).
- Multiple raid bdevs are also created and exposed to initiator and
  tested with file system and other workloads for read/write IO.
- LVS / LVOL are created over raid bdev and exposed to initiator.
  Testing was done for raw read/write workloads and XFS file system
workloads.
- RPC testing is done where on the running SPDK system raid bdevs
  are created out of NVMe base devices. These raid bdevs (and LVOLs
over raid bdevs) are then exposed to initiator and IO workload was
tested for raw read/write and XFS file system workload.
- RPC testing is done for delete raid bdevs where all raid bdevs
  are deleted in running SPDK system.
- RPC testing is done for get raid bdevs where existing list of
  raid bdev names is printed (it can be all raid bdevs or only
online or only configuring or only offline).
- RPC testing is done where raid bdevs and underlying NVMe devices
  relationship was returned in JSON RPC commands

Change-Id: I10ae1266f8f2cca3c106e4df8c1c0993ddf435d8
Signed-off-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-on: https://review.gerrithub.io/410484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-16 20:50:40 +00:00
Shuhei Matsumoto
c5b00933ce iscsi: Fix the issue that queued iSCSI tasks are not migrated when clearing tasks
When multiple LUNs are attached and spdk_clear_all_transfer_tasks() is called with
a LUN, all tasks whose target is the LUN are removed and there will be room to start
queued tasks for other LUNs. However currently no migration is done.

Add UT code to verify the fix too.

Change-Id: I082d370ab86a46e5b4a74a16293a572fae663add
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418765
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-07-16 04:36:11 +00:00
Pawel Niedzwiecki
07926e1b55 test/nvmf: Do not use config files
Use rpc methods instead config file in nvmf tests

Change-Id: I75354cae95e1dd5d0e08f5c694b369fb270b92a4
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/419006
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-13 22:40:06 +00:00
Karol Latecki
f7d5bf0cef test/vhost: add script for performance benchmarking
Following script allows user to run tests which
follow test cases used in SPDK Vhost Benchamrk tests.

Change-Id: I4e69b13d86165fcf37c678b095c2a11817476465
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/416527
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-13 22:33:19 +00:00
Pawel Wodkowski
fdd6dbc902 vhost: add socket path in info dump
This also save CWD on init so socket path is no longer relative.

Change-Id: I303401fe0340f0bc2ea5e4ba468361f68ea84c3d
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419067
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-12 23:54:34 +00:00
Paul Luse
459e899095 bdevperf: don't try to free targets if head is NULL
In the event that something went wrong earluer, we shouldn't
try and touch memory that was never allocated.

This is at least a partial fix for issue 363.

Change-Id: I5174c845fb89f6d386877758fdc99178704ff73c
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/419039
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-12 22:09:27 +00:00
Shuhei Matsumoto
9166bb3c51 ut/iscsi: Add UT for clear_all_transfer_tasks (Delete all or LUN specific tasks)
Change-Id: I4694b2fdfafa64e7d2621e5b004f46a7072173e7
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418761
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-11 21:00:41 +00:00
Shuhei Matsumoto
d02e91b34f ut/iscsi: Add UT for spdk_del_transfer_task (Delete task whose R2T matches task tag)
Change-Id: I6de3fc53d7a771c702f013bfaf7bd425f1ab01e5
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418760
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-11 21:00:41 +00:00
Shuhei Matsumoto
fc01195e1c ut/iscsi: Add UT for spdk_get_transfer_task (Get task whose R2T matches TTT)
Change-Id: I7bc44aeffd4eeb1cc90babf0c6789a2bcee6bb59
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-11 21:00:41 +00:00
Shuhei Matsumoto
9265e1ebbe ut/iscsi: Add UT for spdk_add_transfer_task (Sending R2Ts)
Currently R2Ts per connection are managed by an array
outstanding_r2t_tasks and two tailqs active_r2t_tasks and
queued_r2t_tasks.

These are complex but there is no UT code for them.
Added UTs are not perfect but it may be helpful to keep the
quality and understand the implementation.

Change-Id: I2a0d19d8888e7d3beb5918a1b82b657f4775e01a
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418758
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-11 21:00:41 +00:00
Daniel Verkamp
fbb481c2c6 test: add SPDK_TEST_NVME_CLI autotest flag
As requested in GitHub issue #361, we should have a way to control
whether the SPDK-patched nvme-cli tests are executed.

Also add a report_test_completion call for the NVMe-oF nvme-cli test
so we can easily track whether it executed (the local PCIe nvme-cli
test already had a completion).

Change-Id: I70c26bcf2489ccfa563214a9723603aa705be7ab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418866
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Meneghini <johnm@netapp.com>
Reviewed-by: <ed.rodriguez@netapp.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 23:18:27 +00:00
Daniel Verkamp
56ad1cbea4 ut/lvol: remove bogus lvol_op_comp test
This is passing a struct lvol_task that is not nested inside a bdev_io
context, which is assumed by lvol_op_comp() in the actual lvol code.

There is no problem currently, since lvol_op_comp() happens to not
dereference the bdev_io except inside of mocked functions and inside a
SPDK_INFOLOG() that does not get executed in the unit tests right now,
but this would break if the lvol code ever actually used the bdev_io
pointer.

Change-Id: Ic3b67f4ecff47e5a98a5be8a83c2524573113aa9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418735
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-10 22:48:54 +00:00
Ed Rodriguez
2e0df7813e test: vm_setup.sh localization
Change-Id: Ief3a1fbde5d8eabefa87c39feefe3d496e3a5369
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/418236
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 22:47:16 +00:00
John Meneghini
806f43b26d test: improvements to vm_setup.sh and pkgdep.sh
Change-Id: Ie433edd7c72f84b62c17bacebd31d61ffa6f30ab
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/416052
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 22:47:16 +00:00
Paul Luse
58d8a4564b test: add ability to mock spdk_dma_malloc()
Used in crypto unit tests.

Change-Id: I7d7e31e9030c238cb539b9fcf0936e588c1a4211
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/418320
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-10 21:31:39 +00:00
Pawel Niedzwiecki
b2fd5b25b2 json/rpc: Tests for nvmf subsystem
Change-Id: I70044bfc01c77a367cf4634d245a58546273f40b
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/412477
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-10 21:18:18 +00:00
Tomasz Zawadzki
f0ec7bc6e7 scsi/bdev: use spdk_bdev_queue_io_wait()
New function was added in bdev layer to allow
handling spdk_bdev_io buffer exhaustion.

This patch adds that functionality to scsi bdev.

Change-Id: Ia6a5be871ae09a4d1166991925f0a44f3b355bdd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/417032
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-10 21:17:38 +00:00
Tomasz Zawadzki
d7a7ef613c scsi: spdk_bdev_scsi_reset() now returns void
SCSI reset function return code is not used,
so there is no need for it to have return code.

Change-Id: Ib647109c75b1546675d2601fa00004e1ef186024
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/418615
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-10 21:17:38 +00:00
Tomasz Zawadzki
6f9e81eea0 scsi/UT: add scsi_bdev IO operations UT
This patch adds UT for IO operations, as preparation for
next patch in series handling bdev_io buffer exhaustion.

Change-Id: I139cb8811453576efd1acddd15ad2a3672995f09
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/417452
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-10 21:17:38 +00:00
Chen Wang
87210c1391 test/nvme: add unit test cases for nvme_qpair.c
Add unit test for API function spdk_nvme_qpair_add_cmd_error_injection()
and spdk_nvme_qpair_remove_cmd_error_injection().

Change-Id: Ia1f0baf94c3ab1b9a03c6c2b4796346cbe120a62
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/416905
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-10 01:02:36 +00:00
Daniel Verkamp
1fcfc425bf test/nvmf: test InCapsuleDataSize 0 in filesystem.sh
Change-Id: I939aef31473b6a7acae879fb5b03a62154390cac
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418729
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-09 22:24:40 +00:00
Seth Howell
df8cbd66c7 test/unit: remove more scan-build errors
Change-Id: Ia0f304c20d736cebd43b29337e7c6f8918cff7ef
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418235
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-09 19:45:40 +00:00
Young Tack Jin
6d05186697 ocssd: add chunk info list as mptr of vector reset
This is verified on QEMU NVMe which merged the related patch.

Change-Id: I7c62d7a91350e0a877d1c3651796c38b55122f98
Signed-off-by: Young Tack Jin <youngtack.jin@circuitblvd.com>
Reviewed-on: https://review.gerrithub.io/417077
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Matias Bjørling <m@bjorling.me>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-09 18:54:22 +00:00
Seth Howell
09e3f4e3db nvmf: give qpair_disconnect an asynchronous api.
qpair_disconnect has previously presented an entirely synchronous API.
However, it relies on other asynchronous operations to complete its
task. By giving it an asynchronous API, we can avoid possible race
conditions. Patch 1 of several.

Change-Id: If9e26ee70ae5d6c0273750226b4408a8e4587e19
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417345
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:49:39 +00:00
Pawel Kaminski
57bce7953f test/json: Add rbd bdev test
Change-Id: I8d8b252bf826e6339810ccf27a4c68ed6a726af9
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/417603
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-06 22:48:57 +00:00
Daniel Verkamp
49e24df51c test: remove leftover spdk_cunit declaration
The function itself was removed, but the declaration in the header was
left behind.

Change-Id: Idbff42c197ef56e65fa92c6795306d9326381cf9
Fixes: ca1de9eb7593 ("test: remove spdk_cunit library")
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418231
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 22:47:56 +00:00
Daniel Verkamp
ad51b333e8 ut/json: add a few test cases to json_parse_ut
This brings the unit test line coverage from 94% to 97.2% for this file.

Change-Id: I257fdb0187f7cae256d6fccabbdaa6df9ce4e83b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/417369
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 22:47:18 +00:00
Daniel Verkamp
7d24e2a4cd env_dpdk: fix mem_map translation for sizes >2MB
Add a check to prevent spdk_mem_map_set_translation() or
spdk_mem_map_clear_translation() calls that start within the valid
address range but specify a size that would access parts of the mem map
outside of the valid region.

spdk_mem_map_translate() is safe without any extra checks since it only
accesses the first entry regardless of size, and the MASK_256TB check
catches out-of-range accesses to that entry.

Change-Id: Ie1437e57b5158363bb98a6b42a26fb41a089bbad
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418106
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-06 22:36:22 +00:00
Daniel Verkamp
7ac8b609b0 env_dpdk: fix mem map array sizes
The arrays for both the map_256tb and map_1gb structures were twice
as large as necessary; fix the sizes and add unit tests for the boundary
conditions to verify that the fix works.

Change-Id: I66bce463f234f54e69cf2a697db9f806d398ca1e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418105
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-06 22:36:22 +00:00
Seth Howell
88e26908fa test/unit: fix several scan-build errors.
These errors occur when scan-build is used with clang 6.0 as a backend.

Change-Id: I751a0fd3649ae500887a4dbda1d361104acb7c12
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418233
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-06 22:15:24 +00:00
Seth Howell
5b072d3739 test/config: add some comments to vm_setup
Just clarifying why and where some packages are used.

Change-Id: I36f17856a76ed1eb3d88054da2692beee9e569f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418230
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:15:24 +00:00
Tomasz Zawadzki
bcfd6d0fb4 scsi: only set SPDK_SCSI_STATUS_GOOD on bdev callback
Previously when IO was sent to bdev, even without callback yet,
status for task was set to SPDK_SCSI_STATUS_GOOD.

This patch changes it so that it is only set when callback actually
comesback from bdev via spdk_bdev_io_get_scsi_status().

Change-Id: I4a36ec345608257123e1036d31540f5f1090a23b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/417708
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 21:22:43 +00:00
Shuhei Matsumoto
c3b82474e0 ut/iscsi: Add unit test for residual count of SCSI Check Condition
Residual count must be ignored and sense data must be set instead
when Check Condition occurs.

Add test code to keep current implementation.

Change-Id: I8919a28481c81606e8803269d5ff8a4e3fcbf8c2
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418176
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 21:17:29 +00:00
Shuhei Matsumoto
3e66e72651 ut/iscsi: Add unit test for residual count of SCSI Request Sense
Current SPDK implements SCSI Request Sense command so that a SCSI
Data-In PDU and a SCSI Response PDU are returned and underflow are
set to both of them.

On the other hand, some implementation return only a SCSI Data-In
PDU and some implementation return only a SCSI Response PDU.

Adding test code will be helpful for every implementation.

Change-Id: Icccc2d6e334146f558f8d75215bab1362b2b68f0
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/418175
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 21:17:29 +00:00
Shuhei Matsumoto
a02ab95ebb iscsi: Bug fix for residual count when length of data transfer from SCSI was 0
This bug was caused by the following two changes:

iscsi: Remove duplication of the variable for write completion to bdev
https://review.gerrithub.io/#/c/393582/

iscsi: restore data_transferred accumulation for read
https://review.gerrithub.io/#/c/393713/

For write, bytes_completed is always equal to data_transferred. However,
for read, bytes_completed is based on expected data transfer length and
data_transferred is actual read size. Hence bytes_completed cannot be
used to calculate residual counts.

One of the reason why this bug cannot be found was that there was not any
test case when task->scsi.data_transferred is 0 and task->scsi.length is
not 0. Hence UT code is also added.

Same bug will occur for read sense data. Hence UT code for read sense data
is added in the next patch.

Change-Id: Ib1a283b769e5af0c2d05acb69f90948c5d658087
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/417960
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 21:17:29 +00:00
Piotr Pelplinski
9d258c75af bdev: split examine into two parts
During spdk_bdev_init, examine_config is called.
This call can claim bdev synchronously, based on
configuration. On spdk_bdev_start if none module
claimed bdev, examine_disk is called and can
perform I/O before claiming bdev.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I1448dd368cf3a24a5daccab387d7af7c3d231127
Reviewed-on: https://review.gerrithub.io/413913
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 21:12:53 +00:00
Seth Howell
1196deb504 test: fix scan-build errors in bdev unittests.
Specifically, the errors that occur in scan-build that uses clang-6.0 as
a backend.

Change-Id: I3922a4f65134cbd866fcb27ad9b7a75b6b7803bb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418229
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-06 20:34:01 +00:00
Daniel Verkamp
f1acfee360 test: make unittest.sh work from any directory
test/unit/unittest.sh would normally have to be invoked from the root of
the SPDK repository; tweak it slightly so that it can be executed from
any working directory (e.g. within test/unit).

Change-Id: I8fdbb7c0d618e2bb1490236212e54cbd43cccf6f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418217
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-06 18:43:10 +00:00
Jim Harris
ca1de9eb75 test: remove spdk_cunit library
There was a plan a long time ago to try to output all
unit test results in JSON format for post-processing.
It is used by only one unit test file currently, but the
results aren't used and there are no plans to use it in
the future or extend it to other unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I72c409090bbd7b8fa7ec307bbc97f97ccf2e397c

Reviewed-on: https://review.gerrithub.io/418112
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-06 18:35:03 +00:00
Jim Harris
56d35c5d13 test: forbid use of CU_ASSERT_FATAL
Static analyzers don't see CU_ASSERT_FATAL as truly fatal,
and will complain that code later in the function may try
to dereference a NULL pointer.  So we added SPDK_CU_ASSERT_FATAL
which should be used instead.

This still trips people up sometimes though - the static analyzer
complains and then the developer will add other checks that
pointers are not NULL.

So instead, forbid use of CU_ASSERT_FATAL through check_format.sh
and explain why.  While here, fix up all of the existing CU_ASSERT_FATAL
usages to either CU_ASSERT or SPDK_CU_ASSERT_FATAL.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7974c8c85ddb89ed1b7d882db3a2eb0882ea0217

Reviewed-on: https://review.gerrithub.io/418111
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 18:35:03 +00:00
Jim Harris
140eaaa084 bdev: submit queued IO after disabling QoS
There's no reason to abort IO that have been queued
due to QoS limits, when QoS is switched from enabled
to disabled.  Submit them to the bdev instead.

Fixes issue #357.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If5eafc53418ac686120e1d6a1da884b42cef845e

Reviewed-on: https://review.gerrithub.io/418128
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-06 18:34:45 +00:00
Pawel Kaminski
9aff546b7f test/json: Test for vhost subsystem.
Change-Id: I913d378fe3f324043e8e86c1df209ad19680c60b
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/410789
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-05 20:28:24 +00:00