Commit Graph

2358 Commits

Author SHA1 Message Date
Daniel Verkamp
a02c0062bd nvme/host: use spec-compliant PSDT value 01b
Resolves GitHub issue #165.

Change-Id: If214bdfbe5b92faf2a6611d25e36c3c8876e3190
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362239
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-23 17:10:50 -04:00
Daniel Verkamp
b9bcc3531e bdev: make enum spdk_bdev_io_status private
The user should not see the bdev_io status directly; the NVMe and SCSI
error code wrappers provide the ability to translate to the desired
format regardless of what kind of error is stored inside the bdev_io.

Replace the spdk_bdev_io_completion_cb status parameter with a bool
simply indiciating whether the I/O completed successfully.

Change-Id: Iad18c2dac4374112c41b7a656154ed3ae1a68569
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362047
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-23 16:30:10 -04:00
Daniel Verkamp
54cfee2e84 iscsi: replace magic numbers with SCSI constants
Change-Id: I9b33b038a019576d162c868e2021db1c6589b4f4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362061
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-05-23 16:29:34 -04:00
GangCao
afbb5ceec2 nvme: remove NVME_IO_TRACKERS to use hardware and user specified size
Change-Id: Ie6946343314a62e4416d8cd7dadd98c20a1c1e3a
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/361700
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-23 16:01:52 -04:00
Cunyin Chang
7b5a229c5f test/hotplug: increase the test time for hotplug
This will make sure the hotplug instance not exit before all the hot insert and
remove events done, the setup.sh for hot insert events will use several seconds
sometimes.

Change-Id: I4ad61937760529547cdcf7701f3c3d9033121e45
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/362131
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-23 15:52:25 -04:00
Jing Xia
dc8acbf8d5 test/blobfs: Modify the value of g_fserrno
Change-Id: I120f2f6ffe49d0f4bf6b92892e26228010c9c132
Signed-off-by: Jing Xia <jingx.y.xia@intel.com>
Reviewed-on: https://review.gerrithub.io/362100
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: <sys_sgsw@intel.com>
2017-05-23 13:46:09 -04:00
Daniel Verkamp
b23bc69cfa travis: switch to using the dpdk submodule
Change-Id: I96cb0bc89e793126984899a4ded4e3a1dee56107
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362045
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 22:17:55 -04:00
Ben Walker
305cb239d2 io_channel: Remove unique flag
This is no longer used anywhere. For the places where we previously
used it, we've since found alternate solutions that do not
require it.

Change-Id: I738a80b95ef50348ce1c14969a3812b0a625b3fd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/362064
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
267a4e1ebd blobfs: Add separate I/O target for md
This isn't strictly necessary, but follow the
pattern for sync and I/O devices.

Change-Id: I34b1bd6f9da06008e419f55ee9810022f5fed3d5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/361674
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
a3ab7610f6 blobfs: Use a separate io_device for sync channels
This allows us to no longer use the unique=true flag as well
as assign sync channels different queue depths.

Change-Id: I0b3129a3139bf39d745c2a4dc1d9f402c0a8faa8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/361673
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
3b683c188e blobfs: Use a separate io_device for io channels
This allows us to have a different queue depth
on I/O channels than on metadata/sync channels

Change-Id: I30dfc95af550082efffec4b12ed6c4cd4b8c11cf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/361672
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
345f3d7ddb blobfs: No longer use unique_ctx when creating channels
This temporarily increases the size of the sync channel
to 512 (like all of the other channels). This will be
fixed in a later patch.

Change-Id: I679f1cb7c9f7afdad0acf9a3a26f9632f60cc510
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/361670
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Ben Walker
60e8fb4979 blob: Use a separate md and I/O target
This allows us to allocate different size channels and
not require the unique flag.

Change-Id: I4b1ffd244b60e9e9ab06f9ab4da8161ab57e1169
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/361668
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-05-22 19:27:17 -04:00
Jim Harris
fd2936651f Update dpdk submodule to d426cbab2:
.gitignore: ignore build/ directory

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie6284914ac8c4cee209ef0b5313385d6c02086b7

Reviewed-on: https://review.gerrithub.io/361817
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: <sys_sgsw@intel.com>
2017-05-22 15:31:15 -04:00
Changpeng Liu
30f3e4af42 vhost: correct the example AIO section of vhost configuration
Change-Id: I3143dc12b145ee76bdd23436e4bea17d5f4c12f8
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/361718
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Tested-by: <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-22 12:30:13 -04:00
Daniel Verkamp
5f3841f0e3 nvmf: allow NQN to be 223 bytes + null terminator
An ECN to the NVMe 1.3 specification has clarified that the NQN may
contain 223 bytes before the null terminator.  Make all of our NQN
length checks consistently enforce this behavior.

Change-Id: Iebfd57d11abea64964c7a6ad9d886e40efa243c3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-18 14:36:39 -07:00
Ben Walker
4eda4fd245 Fix incorrect free in blob/blobfs
The channel memory isn't allocated by these
libraries, so they can't free it.

Change-Id: I30909fa4e77bc5a41b45230f04ba5fe75b172dbf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-18 14:36:11 -07:00
Ben Walker
4a3182b891 blob: Add a separate config parameter for channel ops
Separate the maximum metadata operations from the
maximum channel operations.

Change-Id: I1bbd440ab094a2a2e19c9a5b71724ac91ba88e42
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-18 14:36:11 -07:00
Ben Walker
bea2e2308f blob: Remove per-channel queue size configuration
This will need to be configured globally for all channels.

Change-Id: I773252f220373617f8d09d1f24243db8095cf8a4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-05-18 14:36:11 -07:00
Karol Latecki
69d3eba0f3 scripts/setup.sh: add hugepage group write permission
Adding group write permission is needed in case of using vfio
driver and using libvirtd for vm provisioning to avoid
permission errors.

Change-Id: I5348e74c65bd8639071c1a2a164f28849dde9678
Signed-off-by: Karol Latecki <karolx.latecki@intel.com>
2017-05-18 14:04:52 -07:00
Daniel Verkamp
f66a0c9e34 scsi: pass management function as a parameter
Specify the function as a parameter to spdk_scsi_dev_queue_mgmt_task.
This makes the API clearer by making it explicit that the management
function is required for management tasks.

Change-Id: I92d893aadb6faebea81dd79729894d2c2fddf088
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-18 13:47:42 -07:00
Daniel Verkamp
1e7e6a618b scsi: automatically set spdk_scsi_task::type field
The type of a task is known due to the function used to submit it:
- spdk_scsi_dev_queue_task() for normal SCSI command tasks
- spdk_scsi_dev_queue_mgmt_task() for management tasks

Change-Id: I183a1f89ab85f3fce1de2491e77d95d4b147fd72
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-18 13:47:42 -07:00
Daniel Verkamp
de5ffac4ec scsi: factor out read/write LBA range check
The SCSI read and write functions use the same LBA range check, so move
it up to spdk_bdev_scsi_readwrite().

spdk_bdev_scsi_read() previously passed task->transfer_len / blen rather
than xfer_len to the LBA range checking function, but this should
actually be using the transfer length field from the CDB (which is the
xfer_len parameter).

Change-Id: I8285abf936a18a0baf7cc25709945637e4e6b87d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-18 13:47:42 -07:00
Daniel Verkamp
103351befd unittest.sh: limit coverage to libraries
Change-Id: I53d719173c5818e78c7ea07a97da053f5aed21fc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-18 11:10:08 -07:00
Jim Harris
af781ee003 bdev/nvme: print errno on spdk_nvme_ns_cmd submission failure
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I785bfd663a2921f5e4c0630bce7906a928f26376
2017-05-18 10:23:43 -07:00
paul luse
6fba6ff448 unittests: add local UT coverage
Collect coverage info when running UT locally when lcov
is available.

Change-Id: I34a8b60056775a25cd4a3906d59c4a42e7cc845f
Signed-off-by: paul luse <paul.e.luse@intel.com>
2017-05-18 09:48:23 -07:00
Cunyin Chang
32b4ab71a2 bdev/error: Add new bdev/error to inject errors in bdev layer.
Change-Id: I3c68cb5911b3662dbb35f551dc291d5ebdd84bf6
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2017-05-18 07:56:14 -07:00
Dariusz Stojaczyk
679e2831bd vhost: added rpc commands to remove vhost controllers and devices
Added new rpc commands together with underlying vhost API and tests.

Change-Id: Ib9c6a530d0909193ea5115aaac4920c44f39613c
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-18 01:01:46 -07:00
Pawel Wodkowski
82d26c4f20 vhost: track VFIO mappings
If more than one controller is added to guest the same memory table is
set multiple times making VFIO registration/unregistration failures

Change-Id: Ib55c38e292495e549c070beb0234e73f269e63d5
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2017-05-18 00:47:55 -07:00
Jim Harris
f2c8d0dc11 test/iscsi_tgt: exit rbd.sh if CEPH_DIR not defined
This allows the iscsi_tgt tests to be run more easily
in isolation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1fa87e8b02f6e6ed62c75f07471d93ce84da30c2
2017-05-17 15:41:56 -07:00
Paul Luse
b499ae64ad doc: update readme
Clarrifications for DPDK submodule

Change-Id: I7374c1f904f1c5c46c46c299af72816d96a770ff
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
2017-05-17 13:42:32 -07:00
Dariusz Stojaczyk
709d5453a3 vhost: obey VIRTIO_F_NOTIFY_ON_EMPTY
It'll trigger interrupts even if VRING_AVAIL_F_NO_INTERRUPT is present.
Also delegate notifying check to separate function vq_should_notify. It
will be expanded even more once we implement VIRTIO_RING_F_EVENT_IDX
support.

Change-Id: Ic41fd18c9c005c6266b109b0eda4a6dfaef5f06a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-17 10:24:48 -07:00
Dariusz Stojaczyk
7463bf6d2a vhost: add negotiated_features field in spdk_vhost_dev
Added new struct field and an rte_vhost_get_negotiated_features() call.

Change-Id: I5937026720b5341c7377c27fee682996b51e836e
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-05-17 10:24:48 -07:00
Jim Harris
737b1b571b build: add dpdk as a submodule
This will allow a git clone to default to an SPDK-specific
version of the DPDK repository.  Users can still override
to use a separate DPDK repository/installation with the
--with-dpdk configure script options.

While here, remove gzip option for the git-archive operations
in autopackage.sh.  We need to add a git-archive for the DPDK
submodule if we are using it, and compressing at -9 adds a lot
of unnecessary time.  Since we are not archiving these packages,
there is no need to compress them.  Also explicitly disable
coverage and ubsan for the autopackage build, since this build
is only to test compilation and is not actually used for any
test execution.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3cf8a2ed984003a175cdece6542636ede8cb2479
2017-05-17 09:49:27 -07:00
Jim Harris
8de75f8107 nvme/perf: add software-based latency tracking
The latency tracking is done with ranges of bucket arrays.
The bucket for any given I/O is determined solely by TSC
deltas - any translation to microseconds is only done after
the test is finished and statistics are printed.

Each range has a number of buckets determined by a
NUM_BUCKETS_PER_RANGE value which is currently set to 128.
The buckets in ranges 0 and 1 each map to one specific TSC
delta.  The buckets in subsequent ranges each map to twice
as many TSC deltas as buckets in the previous range:

Range 0:  1 TSC each - 128 buckets cover deltas    0 to  127
Range 1:  1 TSC each - 128 buckets cover deltas  128 to  255
Range 2:  2 TSC each - 128 buckets cover deltas  256 to  511
Range 3:  4 TSC each - 128 buckets cover deltas  512 to 1023
Range 4:  8 TSC each - 128 buckets cover deltas 1024 to 2047
Range 5: 16 TSC each - 128 buckets cover deltas 2048 to 4095
etc.

While here, change some variable names and usage
messages to differentiate between the existing latency
tracking via vendor-specific NVMe log pages on Intel
NVMe SSDs, and the newly added latency tracking done
in software.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I299f1c1f6dbfa7ea0e73085f7a685e71fc687a2b
2017-05-17 09:49:27 -07:00
Daniel Verkamp
a591161cb2 bdev: make struct spdk_bdev contents private
Change-Id: If203e82f8cd10d5998a565ad490ef11e2916687f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
eaf9019738 nvmf: use bdev name getter
Change-Id: Id0094871bb18f90908f3a9d96f47ddd264cea5bd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
9d1e05d2f2 bdev: add getter for write cache enable status
Change-Id: I93656deda8956c77e9565ee1f271795642e62ac9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
0d56499d3a scsi_bdev: use max unmap descriptor getter
Change-Id: Id67b158c36b2b879849b18909bc9bb9a5273f8c7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
76e67baf28 blob_bdev: use bdev block size getter
Change-Id: Id22f904fdb71798ef2369415d38f468b06d7ac07
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
ed362f1f35 test/bdevio: remove gencnt manipulation
The bdev generation count is an internal implementation detail;
applications should not be reading or writing it.

Change-Id: Ic4455b7b72dc80babbc410420c89f52ef5d77e6d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Daniel Verkamp
5bb041adcf bdev: rearrange spdk_bdev_io
Move some of the fields in spdk_bdev_io (the layout of which is no
longer public API) to put related fields together and improve struct
packing.

Change-Id: I0ca5d9441b00c65703c35391d125059eaa4bcee3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-17 09:45:42 -07:00
Jing Xia
928df4a448 test/blobfs: Rename blobfs test file name
Change-Id: I584c667aff5a0cdfee26b958a2b58c6ed41d45f7
Signed-off-by: Jing Xia <jingx.y.xia@intel.com>
2017-05-17 08:21:59 -07:00
Daniel Verkamp
133dc1e6ae iscsi: clean up iSCSI/SCSI task conversions
Add a helper function to find the iSCSI task given its nested SCSI task
structure, and use it to remove all casting between spdk_scsi_task and
spdk_iscsi_task.

Change-Id: Idc7c6d0a3b4d53041916d25a1bdecedfb56b94f9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-16 16:45:22 -07:00
GangCao
b347d551e8 nvme: detach the pci device with calling process's own devhandle
Change-Id: I2693b4bd29e0500379d5e399723aec7e44bffca3
Signed-off-by: GangCao <gang.cao@intel.com>
2017-05-16 10:35:29 -07:00
Ziye Yang
644678258f blobfs: move the location of next buffer check.
This patch address the issue:
https://github.com/spdk/spdk/issues/151.

For cache_append_no_cache in cache_ut testcase,
there is resource contention for buffer among two
threads in the following two functions.
Thread 0: cache_free_buffers
Thread1: __file_flush_done

When the thread1 execuctes __file_flush_done,
it calls the call back: __sem_post defined in
following statement in spdk_file_sync

_file_sync(file, channel, __sem_post, &channel->sem);

Thus Thread 0 will execute next function
cache_buffers, and it frees the buffer.

Then Thread 1 continues executing the remaining statements
in __file_flush_done with the assert function, and touches
the space already freed.

So it will be safe to move ahead the next buffer check.

Change-Id: Ic007b3481f4e3a17d47eeca5c9c802001949a5ab
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-05-16 10:51:26 +08:00
Jim Harris
4eafea0360 bdev: add a null bdev module
Also change the discovery/nvmf.sh test to use it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I56bce9a84bd46f13b6d4f34da81abf23413f2598
2017-05-15 14:00:37 -07:00
Daniel Verkamp
7455910d1b scsi_bdev: simplify scsi_readwrite helper function
Factor out the common data direction Check Condition logic from read
and write.

Change-Id: Ia2fbd5f69eefd826ebb5ab2a8e99aeb380f1b7ea
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 13:35:18 -07:00
Daniel Verkamp
8e379851ab scsi_bdev: clarify block count variable names
What was previously called "maxlba" was actually the block count
of the device, which is the maximum LBA plus one.

Change-Id: I5e141a3eb0c79b2aa70006739c5586cb50744d3d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 13:35:18 -07:00
Daniel Verkamp
658f816344 test/scsi_bdev: add read LBA range tests
Change-Id: If0cc8f970c20c4a398882183de9e7b066753f030
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-05-15 13:35:18 -07:00