4277 Commits

Author SHA1 Message Date
Dariusz Stojaczyk
761325e2fb fio_plugin: add proper I/O error handling
When an I/O failed to be issued, fio
used to freeze entirely. From the fio
perspective, that I/O was never being
completed. Fio expected us to complete
it during the I/O thread cleanup & shutdown.
(spdk_fio_getevents, line 578)

This patch makes the I/O issue callback
complete the I/O immediately on any
encountered error. Return codes are now
properly propagated with io_u->error
field. This also allows some additional
error messages to be printed.

Change-Id: I9fc3cb6c6a946b78fc8384701827a815103ed4c6
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/392534
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 19:35:36 -05:00
Pawel Niedzwiecki
3d28498af9 test/virtio: Perform unmap tests on unmap supporting devices
-perform rw tests on all devices
-perform unmap tests + rw tests to match number of jobs and queues

Change-Id: Ica59ea3667c1c3a639ec9cded0fd43ea9d83ef03
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/394074
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 19:35:36 -05:00
Shuhei Matsumoto
8f9028deb3 iscsi/rpc: Fix completeness of dump of iSCSI global params
To use JSON dump of iSCSI global parameters as configuration file,
at bootup, it must have completeness.

This patch fixes the following:
- DefaultTime2Wait was not contained.
- MinConnectionsPerCore was not contained.
- Converting only 0 to "None" for dump is not efficient.

Change-Id: I740e6938c216c5c62df6ee70b0ceac40fd0cac00
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/396100
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 18:32:37 -05:00
Pawel Niedzwiecki
d66b1e1c91 test/virtio: Move all nightly test cases to per-patch
With multiqueue all jobs run in parallel so
test execution time will not be extended

Change-Id: I6187ee0acf6958b137ba74dd34706538197ce966
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/394389
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 18:28:49 -05:00
Hailiang Wang
835e5bc225 autotest: blobstore_test (porting from nightly to autotest)
This patch is used to test blobstore automatically
which based on examples/blob/cli/blobcli.

Change-Id: I0309cf01d1561f309574a37aeaff8bfbc395f65c
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/386186
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-24 17:12:47 -05:00
Xiaodong Liu
24c7f62fed test/iscsi: check and avoid setting DataDigest
iscsiadm installed by some Fedora releases, like Fedora 27,
loses DataDigest parameter unintelligibly.

Change-Id: I6664146c6e0cb8933188b29166bedd087f4d38e1
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/396076
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 16:52:30 -05:00
Seth Howell
4151703690 test/nvmf: add null pointer check to subsystem test
Fedora 27 ships with Clang 5 which throws an erroneous error in the
add_ns subsystem test. It believes that subsystem.ns is a null pointer.
In order to circumvent this error, I add an assert to bail out of the
test if the pointer really is null and then only execute the final
assert if the pointer is not null. This way our tests will still pass
Scan-build tests on Fedora27.

Change-Id: I54d3fae485d56033ba5eb23b0aa323480ae6a6e4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/396051
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 16:50:42 -05:00
Changpeng Liu
53c24913f2 vhost_lib: enable virtio configuration space feature
New added vhost user messages: GET_CONFIG/SET_CONFIG are
used for get/set virtio device's configuration space, this
commit enable the new added vhost messages.

Change-Id: I5c3e3f8fb6ed55e99299323c39658765b1724bb8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/386545
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 16:48:48 -05:00
Daniel Verkamp
88da45bfd0 scsi: ensure bdlen is initialized before use
This fixes a false positive warning when building on GCC 7.2.1 with
CONFIG_COVERAGE=y.

bdlen is always initialized on the path where it can be used, but the
compiler seems to get confused when coverage is enabled, so zero out the
value at the top of the function.

Change-Id: Ifc13abff80124cad3d26286ffebf84f967141d13
Reported-by: John Meneghini <johnm@netapp.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/396244
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 16:45:49 -05:00
Dariusz Stojaczyk
8be76f8589 setup.sh: hide SKIP_PCI from public help
Instead, PCI_WHITELIST="none" can now be used.
SKIP_PCI continues to work, but is undocumented.

Change-Id: Id4a7959716f6bca92be0da761258c84f32aca8b5
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395946
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-24 16:45:09 -05:00
Dariusz Stojaczyk
2a68832d5c setup.sh: obey PCI_WHITELIST in ./setup.sh reset
It is now possible to reset only
particular PCI devices.

Change-Id: Ic45be5e7e16cf4736c742b14fc0d709788a3b3f1
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395945
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-24 16:45:09 -05:00
Piotr Pelplinski
074f4d7627 blob: set md_ro and data_ro fields only after persisting data
Currently, there is no possibility to save read only blob to disk.
This patch modifies behaviour so that read only flags are applied after syncing blob.
This is analogy to resize, set xattr and remove xattr operations.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Iffed601c78cb83231bb20e7ef05b73847dc3c95a
Reviewed-on: https://review.gerrithub.io/394243
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-24 15:56:15 -05:00
Ben Walker
be774bf664 nvmf: Check subsystem state when modifying listeners or hosts
Change-Id: Idadd4eb7574b005cce8b888b28c8c3b6993343a4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/395555
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-24 15:52:48 -05:00
Chen Wang
9e8415c688 autotest: vhost_filesystem_test (porting from nightly to autotest)
Add vhost-scsi and vhost-blk filesystem test cases with
lvol and nvme back-end in nightly test

Change-Id: Id2cccd9a45aaa995bef90415855835b93447f2e9
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/390923
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 15:51:03 -05:00
Tomasz Zawadzki
b6aaba0852 bdev: remove vbdevs during spdk_bdev_unregister()
spdk_vbdev_unregister() is part of internal bdev API,
yet bdev module that uses spdk_vbdev_register() directly
will not be removed correctly when using delete_bdev RPC.
spdk_vbdev_unregister() is now consolidated with
spdk_bdev_unregister().

This comes up when deleting lvol bdev, as it does not use
spdk_bdev_part_* functions.
base_bdev->vbdevs entry was not removed for bdev that lvs
is created on.

Additionally patch expands test to create lvol bdev,
after removing it using delete_bdev RPC.
With ASAN enabled this would report accessing
already freed memory previously.

Change-Id: I9547e83862e2daa50355d56a1c9f453aaa6cfdb8
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/395711
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 15:50:04 -05:00
Daniel Verkamp
4609215690 doc/nvmf: clean up and reorder the nvmf_tgt docs
- Remove outdated section about pre-release kernels; the Linux kernel
  support has been in released kernels for a while now.
- Switch to ./configure rather than `make CONFIG_...`.
- Put the target configuration into subsections, and move it earlier in
  the doc.
- Clean up some wording and style issues.

Change-Id: I86e79226a73c4e36ed8a440294957ea824f24f0b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/395869
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 15:40:04 -05:00
Jim Harris
22bdcdc39b iscsi: reuse spdk_iscsi_conn_stop_poller
Similar functionality is needed when migrating a connection
to its new core after login.  So make spdk_iscsi_conn_stop_poller
reusable for that case, and call it in the login migration path.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie014a2e50d6ed02165317deab85c943225953197

Reviewed-on: https://review.gerrithub.io/395857
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 15:39:05 -05:00
Jim Harris
20528e262d iscsi: simplify spdk_iscsi_conn_stop_poller()
All calls to this function now use spdk_env_get_current_core()
as lcore, so remove this parameter.

All calls to this function also use _spdk_iscsi_conn_free() as
the stop_fn, so remove that parameter as well - just call that
function directly from the callsites and eliminate the extra
event.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I639e63d26fbfcc054137d9b453be2927e2c1f625

Reviewed-on: https://review.gerrithub.io/395854
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 15:39:05 -05:00
Jim Harris
5aa90453cb iscsi: move spdk_iscsi_conn_get_migrate_event inline
This function is now only called from one place, and moving
this inline will simplify some future patches which will
consolidate the login and full-feature pollers.

No functional change.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7d4e6973882c7b579e341283648317e8e06d998a

Reviewed-on: https://review.gerrithub.io/395853
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-24 15:39:05 -05:00
Seth Howell
68b96260ed test/unit: replace sprintf call with snprintf call.
This fixes a Klocwork bug.

Change-Id: Ia42bf0f246c8e1b9aa211095a00b9d1ab8e7a085
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/396059
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-24 15:38:05 -05:00
Jim Harris
f570aa654a vhost: only split on 2MB boundaries when necessary
vhost I/O only need to be split on 2MB boundaries if
there is a break in the VM's memtable at that 2MB
boundary.

This should drastically reduce (if not eliminate)
the intermittent test pool failures seen recently.
virtio limits number of segments to 128, but this
2MB splitting could introduce additional segment
breaks which we do not allocate IOVs for.  In almost
all cases, there are no memtable breaks except at
low 2MB, so most of the extra segment breaks we are
adding are unnecessary.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I12d85c289ad80c7bb65e3d2030a2405092b19deb

Reviewed-on: https://review.gerrithub.io/396058
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2018-01-24 10:33:43 -05:00
Daniel Verkamp
e489ca69f6 setup.sh: add virtio device names to status output
Change-Id: I0f6e1d8df9480538e2dc39cf658396d4ecd14daa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/393724
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-23 23:59:12 -05:00
Dariusz Stojaczyk
3779dda48b setup.sh: change NVME_WHITELIST to PCI_WHITELIST
NVME_WHITELIST param remains to work, but is now
undocumented. PCI_WHITELIST will apply to all PCI
devices - NVMe, IO/AT and Virtio.

Change-Id: I782f48bea68079c63e4a2794e4a4eb8f9a7226c9
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395944
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-23 23:44:06 -05:00
Dariusz Stojaczyk
f8c1c71c2e setup.sh: support multiple hugetlb mountpoints
We used to crash when `mount` returned
multiple hugetlb mountpoints.

Change-Id: I75717c2c394204bdfec730edabb9ff6386047742
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395926
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-23 23:44:06 -05:00
Dariusz Stojaczyk
4b42897985 setup.sh: fix chown
Patch da7e3bb8 [1] disabled chown for vfio
groups and hugepage mountpoints. Chown code
was never being executed.

[1] da7e3bb8 ("setup.sh: replace username param with TARGET_USER env
variable")

Change-Id: Ie5c9ac42fca5b60c72ced501d4065842a5bef9cd
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/395925
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-23 23:44:06 -05:00
Maciej Szwed
68b8237cdd blob: assign iovcnt value in spdk_bs_user_op_alloc function
iovcnt value was set to 0 instead of being assigned from
input argument.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I0f7e89871357f5db9a5a34c801176bc5c7870021

Reviewed-on: https://review.gerrithub.io/395959
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 23:41:57 -05:00
Dariusz Stojaczyk
fb12bbecd2 virtio: move vdev->name allocation to generic virtio
Previously vdev->name was being allocated/freed
separately in virtio_pci and virtio_user backends.
Now it's all done in generic virtio library and
cleans up some code.

Change-Id: I810e976d09781c0c9b25c6f7fd957a83aad6c7b8
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394704
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Dariusz Stojaczyk
e6da08c28a virtio/pci: detach pci device on virtio-pci destroy
Fix memory leak on virtio-pci device destruct.

Change-Id: I7d33802bc15c15d6a2dbac3e83456c9cf23b6004
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394703
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Dariusz Stojaczyk
0d6a37c7d9 bdev/virtio/rpc: add RPC to attach virtio-pci device
`rpc.py construct_virtio_pci_scsi_bdev <bdf> <name>`

The RPC internals reuse the struct definition and callback
functions from virtio-user equivalents

Change-Id: I0c6c49d8481565a49ec9460a633696d27d55367a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394447
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 22:49:21 -05:00
Ben Walker
2bbc59fa79 nvmf: Fix bug when accessing realloc'd pointer
Caught by the latest scan-build.

Change-Id: I606a30447e5221c86e79accb9a2c1fe76e9eeee0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/396054
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 17:20:05 -05:00
Ziye Yang
b232bff575 nvme perf: fix the caculation for average, min, max latency
This patch is used to fix the time caculation for
average, min, max latency.

Also changes the float to double to make it more accurate.

Change-Id: Ia91278f04656787e2dcf94dd37d839bcf31d618d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/394674
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-23 17:01:23 -05:00
Jim Harris
83c81baee6 doc/vhost: updated user guide
Change-Id: Ie2d9a949d44a2f50523736ade83dfd66935a6385
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/391603
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-23 16:49:05 -05:00
Jim Harris
4d48d87a7f iscsi: remove idle connection handling
Some upcoming changes will effectively render this moot
anyways by adding an epoll/kqueue descriptor to poll
on in all cases (not just connections that have been
idle for 5ms).

The epoll/kqueue code was just ifdef'd out instead
of removed - some of this code will be useful
and reusable with minimal changes in the upcoming
patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0c354390537e6369cb3c32e78a59c300dec6d098

Reviewed-on: https://review.gerrithub.io/395553
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-22 23:22:17 -05:00
Jim Harris
8c2b8ca7d7 iscsi: put spdk_iscsi_conn_write_pdu() in lib/iscsi/conn.c
This function was originally named spdk_iscsi_write_pdu()
in lib/iscsi/iscsi.c.  Since this is an operation on a
connection, add "conn" to the name and move it to conn.c.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaad022907d43788108d5b2660306abcf5e94040d

Reviewed-on: https://review.gerrithub.io/395522
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 23:22:17 -05:00
Jim Harris
936eb59d8d iscsi: use return code to determine if all PDUs were flushed
This will enable some future enhancements where the main
iSCSI connection poller will not try to flush PDUs, we will
just start a separate poller to periodically flush PDUs in that
case.

This is all part of broader scheme to enable epoll() for
iSCSI.  Making these changes allows us to avoid using epoll()
to signal when TCP buffer space is available.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6fe9314e386673fcc87bb15580ef808838c55a7b

Reviewed-on: https://review.gerrithub.io/395521
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-22 23:22:17 -05:00
Jim Harris
902f8719f2 vhost: bump SPDK_VHOST_IOVS_MAX to 129
Currently we test up to 512KB I/O sizes, and this I/O
size could fragment to 129 IOVs if split on 4KB
boundaries if the buffer start is not 4KB aligned.

This is likely the cause of a recent slew of failures
in the test pool which coincide with updating to
fio 3.3.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I189157af578e75e025ca8e3420712739e604fca7

Reviewed-on: https://review.gerrithub.io/395872
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-22 23:19:43 -05:00
Jim Harris
b2503cb335 blob: add spdk_bs_user_op_t
This allows a channel's request_set resources to be
used for queuing I/O requests.  This is needed
for upcoming thin provisioning functionality,
where we must queue I/O requests that need to
allocate a cluster, if another cluster allocation
is in progress.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie8d3e799afc0b56bc95ba5ecab11253d8bc8608f
Reviewed-on: https://review.gerrithub.io/395037
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 16:00:09 -05:00
Jim Harris
4547f9bdf8 blob: split readv/writev as separate blob calls
Now all operations (both single buffer and iov-based
payloads) which span a cluster boundary get split into
separate blob calls for each cluster.

This will simplify upcoming patches that need to do
special operations if a cluster needs to be allocated.
This code can now be added to just the single cluster
operations and not have to worry about splits.  It
will also simplify the code that will eventually queue
requests which require a cluster allocation if an
allocation is already in progress.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I22a850e6e23e3b3be31183b28d01d57c163b25b1

Reviewed-on: https://review.gerrithub.io/395035
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-22 16:00:09 -05:00
Jim Harris
7ba04e589a blob: move batch creation into single/split non-iov functions
A future patch will queue an operation in
_spdk_blob_request_submit_op_single if the cluster is not allocated
and another allocation is already in progress.  If the queueing fails
because of no channel resources, we want to fail the callback and
need to do this before creating the batch.

This causes a bit of code duplication, but in the end should make the
code easier to read.

While here, pass spdk_blob instead of spdk_blob_data to
_spdk_blob_request_submit_op_single.  This simplifies a future patch
which will need the spdk_blob when queueing an operation.  It also
incidentally makes it consistent with _spdk_blob_request_submit_op_split.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib0ec0e5138eac5bf208fcde6676cd2a77a1a663f

Reviewed-on: https://review.gerrithub.io/395196
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-22 16:00:09 -05:00
Daniel Verkamp
273442b390 nvme/fio_plugin: update to fio 3.3
A previous commit updated the fio version for the bdev fio plugin, but
the NVMe fio plugin should also be updated.

Change-Id: I32fab18563b6b723a3e9b71589e3a3904196d07f
Fixes: 21ec62533d24 ("CHANGELOG: add fio 3.3 support")
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/395788
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-22 15:58:17 -05:00
Dariusz Stojaczyk
568cf46c5d bdev/virtio/rpc: cleanup construct_virtio_user_bdev code
Make it non vhost-user specific. The same
code will be later reused by upcoming
construct_virtio_pci_bdev RPC call.

Change-Id: I6964fedea2a1d52c9b06d5fd8cafe745e06df0ab
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394504
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 15:57:11 -05:00
Dariusz Stojaczyk
225ef46bc0 bdev/virtio: allow specifying a name for virtio-pci devices
The default name remains as sprintf("VirtioScsi%d", counter++).

Change-Id: I244c1389f8dfac16f61aaf1d610a7888dd55e56c
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394446
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-22 15:57:11 -05:00
Dariusz Stojaczyk
37fb62224a bdev/virtio: add API to attach particular PCI devices
This is just the API introduction. See subsequent
patches for its RPC usage.

Change-Id: Iadb7c9bf6a56ab4330c9f2215c6006a2935d208d
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394445
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-22 15:57:11 -05:00
Shuhei Matsumoto
632f0ced75 iscsi/rpc: Set processor affinity of connection through JSON-RPC
Currently setting cpumask to portal is possible only through
iSCSI.conf. This patch makes possible for any user to set cpumask
through JSON-RPC too.

The following are done in this patch:
- To keep compatibility, cpumask parameter handled as optional.
- Python test code is added.
- Current python script for JSON-RPC does not work correctly for
  IPv6 and the issue is fixed.

Change-Id: I42ef397ce95040a36db4430417a35e9e97527477
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/391728
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 15:54:48 -05:00
Jim Harris
7bae25dfcd iscsi: remove flush timeout
This concept was added very early in the SPDK iSCSI target
development process when there was a high focus on maximizing
throughput on a single iSCSI connection with 512-byte reads.

Realistically, in multi-connection environments focused on
predominantly 4KB (or more) workloads, this concept loses its
effectiveness - it is relatively rare that PDUs from multiple
I/O would coalesce within the default 8us flush timeout period.

There were no users of flush timeout in the SPDK tree - it was
not even documented in the iSCSI configuration file example and
there was no way to modify it at run time.  So leave the change
out of the CHANGELOG.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I08c3e959fb3945dc2c9cb89248305d0c88aa778f

Reviewed-on: https://review.gerrithub.io/395520
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-22 13:37:31 -05:00
Dariusz Stojaczyk
5f247660d7 setup.sh: add help message
```
./scripts/setup.sh help
```

Change-Id: I4804f032861f949be3e541456f1df55b64937909
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394807
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-19 15:52:59 -05:00
Dariusz Stojaczyk
da7e3bb880 setup.sh: replace username param with TARGET_USER env variable
Make it configurable the same way as other options.
setup.sh will configure hugepages and vfio for the
current user by default. To specify other user,
set TARGET_USER variable.

Change-Id: Ib98178a70a007bec761f7465689c2150e8844ffb
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394806
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-19 15:52:59 -05:00
xuhuagen
4041ab04fb Test: fix the syntax error issue in the nvme.sh
The nvme.sh will report error If more than one NVMe devices in the system.
Update the nvme.sh to fix this issue.

Change-Id: I9685394ec53eb036f7580a383619bca559f95c60
Signed-off-by: xuhuagen <huagenx.xu@intel.com>
Reviewed-on: https://review.gerrithub.io/394870
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-19 15:52:05 -05:00
Shuhei Matsumoto
f587197d21 iscsi&iscsi/ut: Unify cpumask decode between iSCSI.conf and JSON-RPC
Currently cpumask cannot be specified for each portal when it is
created by JSON-RPC and portal group creation is not unified
between iSCSI.conf and JSON-RPC.

This patch does the following:
 - cpumask string is decoded in spdk_iscsi_portal_create() which
   is common between iSCSI.conf and JSON-RPC.
 - parsing configline of portal is difficult to understand and
   hence it is refactored.
 - UT code is added.

 JSON-RPC will be added by the next patch.

Change-Id: I13b9989263fae5facff260de32a55ec99f5d5a06
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/392447
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-19 15:47:57 -05:00
Seth Howell
52bd445367 fio_jobs: add fsync_on_close to fio args
This is to help facilitate the update of the fio_ubuntu binary to fio-3.3

Change-Id: I4c896dc60255f821a5cf66af818396f210304f07
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/395394
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-19 15:37:08 -05:00