8385 Commits

Author SHA1 Message Date
Ben Walker
dcb8ba7b8e thread: Add a cleanup function for threads
This reduces some repeated code. I could have made both paths
call spdk_thread_exit(), but I want to add some stronger checks
around the tls_thread there eventually that wouldn't be valid
in the other path.

Change-Id: I61e1484536d8aef05f120d9ed394312b41f8445a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453575
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:35:10 +00:00
Ben Walker
266dd568e6 nvmf: Remove use of events from nvmf application
This is all based on SPDK threads instead.

Change-Id: Idfd23d6541a548179be24b16540d62e5d7ea2a5c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:35:10 +00:00
Ben Walker
c2c0ece7a2 doc: Add concept page on submitting I/O to an NVMe device
To be referenced in an upcoming blog post.

Change-Id: Ib1669b3223257d1c1978e8664edf9b1e078918a3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:33:19 +00:00
Ben Walker
30af9d3798 doc: Finish overview page
This has been on the TODO list for a long time

Change-Id: Ibdd130f976e9f4436a9dc94919878e2ff187e873
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453452
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-09 04:33:19 +00:00
Ben Walker
68f9bbc719 doc: Rename directory_structure.md to overview.md
Change title, move to Concepts section. Otherwise
the document remains the same.

Change-Id: I15dfe541447a620e270d94df32c76b36082ef4be
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:33:19 +00:00
Darek Stojaczyk
d193d9873b autotest/common: fail if the requested test suite can't be run
Now that SPDK_TEST_* default to 0, we can decidedly say
that if the requested test suite can't be run on your
system due to unmet dependencies (e.g. missing system pkg),
you will get an error.

Previously we would silently disable that test - e.g. when
someone ran with SPDK_TEST_RBD=1 but had no librbd system pkg
on his system, the SPDK autotest would return success without
running any RBD tests. That was very counter-intuitive.

We still silently disable a lot of tests inside our particular
test scripts. Hopefully we'll stop doing that eventually.

Change-Id: Ib400ba772c095b4d65fa8893a0669f3d8c3984e6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
3674c1d3c3 configure: fail --with-asan if it's not available
Change-Id: I3e9209627125ae222559c98efa215d0a50320b32
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
02406e5654 configure: fail --enable-log-bt if it's not available
Change-Id: Id8c5eca81124f06b62087bd64fb8a20ae8c69d28
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453710
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
5cb1804002 configure: fail --with-iscsi-initiator if it's not available
Change-Id: If84bc3302c2d44a439a1028edfa0ceeaf3f32260
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453709
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
55bd0d3afc configure: fail --with-rbd if it's not available
Change-Id: I9767582835c0ef7ae1a341f60134bb74efbe3fd4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453708
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
ce71172650 configure: fail --with-pmdk if libpmemblk is not available
Change-Id: Ibbd84374dac6c2ac2eba30dd2664b8a4b56277a6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453707
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
eed0c6be16 configure: fail --with-rdma if it's not available
While here, also refactor IBV_WR_SEND_WITH_INV check
to use the new BUILD_CMD command.

Change-Id: Ieefe77d769f2b1369d204ec54b976fa11441af46
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453706
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
32b356cd04 configure: check IBV_WR_SEND_WITH_INV not only for Linux
If the user wants to build RDMA on BSD and IBV_WR_SEND_WITH_INV
is unavailable there he'll now get a warning message.

Change-Id: Ie6c6ce55d21de519bb371ceed44fb1e01d6dc878
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Darek Stojaczyk
c4a3b31278 configure: don't assume /usr/include/ is the header location
It doesn't have to be. In order to check if the system
header is available we'll now to try to compile a C program
that includes it.

Change-Id: Ie46c20dab5e96da746650bba1d5dbd0fc32ecec9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-09 04:32:23 +00:00
Jim Harris
788c16773b test/nvmf: add bdevio test
This tests a variety of bdev-related commands against
our NVMe-oF target.  The key ones are write_zeroes
and nvme_passthrough which until now weren't being
tested at all.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8a5cfb2e41c2a7e8fd940b61b43a80c39993f9a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452936
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2019-05-08 22:43:19 +00:00
Jim Harris
915270db68 bdev: make bdevs array for get_bdevs_iostat RPC
Fixes issue #775.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5e07084599c2363b64619f38bd826fe100217020

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 22:43:00 +00:00
Ben Walker
e13da83e4a nvmf: nvmf_tgt_host_trid stores a pg instead of a core
Change-Id: I53bb89f6999ab9bbc4bd8e960970454d60beebee
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452473
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:56:59 +00:00
Ben Walker
534240380b nvmf: Rename spdk_nvmf_get_next_core_rr to spdk_nvmf_get_next_pg
Make this return the next poll group instead of the next core.

Change-Id: Ie62dcbe0b3d869cc1a9b8f8b4ccce8f068ed4dfd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452472
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:56:59 +00:00
Ben Walker
97d35377c5 nvmf: Don't assume the first core is core 0 when scheduling poll groups
Change-Id: I883ffacca26be0d4fc0a3cfdac5a9c0391b7585e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 21:56:59 +00:00
Ben Walker
3555e24505 nvmf: Rename nvmf_tgt_get_qpair_core nvmf_tgt_get_pg
Begin shifting away from being core-centric to instead
be thread-centric.

Change-Id: I0664cdef6766c158f793dd25cc47f6bbd7d928ad
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 21:56:59 +00:00
Ben Walker
d34635f04c nvmf: Move nvmf_tgt_poll_group_add lower in file
This is only called from one spot, so move it near there.

Change-Id: I7e98ea9f95ba86e663566f6a1cbbc4f88d89a1d4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2019-05-08 21:56:59 +00:00
Ben Walker
67234b6ad4 event: Attempt to respect cpumask when scheduling threads
Individual threads can have their own CPU mask. Attempt to
respect that in the scheduler.

Change-Id: I2bd08d4249bdae32a459ed8770b88090346be5dc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452258
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:56:59 +00:00
Ben Walker
835d21a2d0 thread: Allow thread schedule callback to fail
Change-Id: Id391c2c03804dda0ad354d98dfe0a75fb4d3d324
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:56:59 +00:00
Darek Stojaczyk
a1d4dcdc85 vhost: don't use the second ctx param in spdk_event_allocate
Prepare to switch to spdk_thread_send_msg() which
accepts only one context parameter.

Change-Id: Iea3e8d1e715957d9b3fea12e969f29084a2948dc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452393
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 21:54:04 +00:00
Darek Stojaczyk
b643dcd1fd vhost: introduce poll groups
The goal is to remove legacy event messages from vhost.
The new message passing API accepts thread objects instead
of lcore numbers and poll groups are meant to simplify
the transition.

Eventually we'd like vhost to spawn its own threads and
do message passing only within those, but SPDK libraries
can't spawn their own threads just yet. As a stopgap, vhost
will now maintain a list of all available threads (in form
of "poll groups" to mimic nvmf) and will start pollers on
them using its own round robin scheduler.

This patch only adds the poll groups list, it doesn't
change any existing functionality.

Change-Id: I89cc5da5df3612827c6fc9015f03c94b5f4a10ad
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:54:04 +00:00
Seth Howell
03b9841dd9 test/iscsi: enable iso testing.
This paradigm has made it so much easier to test NVMe-oF locally. Extend
it to the iscsi tests as much as possible.

There are a few exceptions to the rule right now. Namely, the calsoft,
perf, pmem, and rpc tests which need to be slightly refactored before
they are ready for isolation running.

Change-Id: I3e92bd340c6a499dc77a56eb67bc7b074604746c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:48:30 +00:00
JinYu
743081c22b unit/nvme: add FC and TCP in trid trtype test case.
Add two trtype test and increase code coverage.

Change-Id: I59543dc6aa56529b05bb7ac5717d9ba7004b8b72
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453626
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:27:32 +00:00
yidong0635
62966da34c test/nvme: Move ASAN check out of for loop
Check whether ASAN is enabled up front, before entering the loop over all PCI devices.

Change-Id: Iede6a0466014745b77190b79fb8255e7b61bb260
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453512
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 21:27:05 +00:00
Dariusz Stojaczyk
e560d53c12 env: silence rte_vhost logs about each received message
rte_vhost currently logs every received vhost message as
a separate log entry. Every VM (re)boot results in over 100
lines of output printed for every queue in every vhost
controller. In our vhost initiator test suite we've got
over 5k lines of those in total. These logs do not help
in any debugging, so we disable them now. We'll still print
vhost messages of log level >= NOTICE.

Change-Id: Ief3f4230056478472412b13efc4da438b92b1d18
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/371691
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-08 21:25:48 +00:00
Alexey Marchuk
744adf39ee build: Fix ARM compilation
ARM system doesn't provide definition of PAGE_SIZE. The patch adds the definition of that symbol to the compiler flags for ARM system

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Change-Id: Iae0a528569aedaea7df95d9f0d9256cb23b9798b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453657
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 21:24:39 +00:00
Richael Zhuang
09a782dcff sock: fix a uninitialized-variable error
Valgrind reports error "epoll_ctl(event) points to uninitiazlized
byte(s)" when running unit test. Fix it by initializing variable event
to a known state.

Change-Id: I756687c1077eeb632c045c316a76714aa453b918
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453525
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 21:24:13 +00:00
paul luse
fa0005d41a test/json_config: fix issue with missing bdev
I'm guessing this happened during a refactor a while back, just
noticed it while looking at something else.  We create a PT bdev
on a malloc bdev that was never created.

Change-Id: Idda7448b908c184f31be3ac1dd076f1cdf3397ed
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452778
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 15:25:03 +00:00
Tomasz Zawadzki
9011f2401b test/raid: switch from config file to RPC
bdev_raid.sh was using configuration file (bdev.conf.in)
made for blockdev.sh. Changes in that config file affected
the raid tests.

Since the test actually only used two malloc devices as
raid bases, switching to RPC makes it simpler clearer
to understand what is used in tests. Additionally
making raid tests standalone from blockdev.sh.

Fixes issue #777

Change-Id: Ia4c5858852241a9fd8212690aefbf015c98d1b57
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 14:43:59 +00:00
Tomasz Zawadzki
a394f62868 test/raid: always call rpc_py with correct socket
Previously the socket was only used in single rpc call,
upcoming patch will use of the $rpc_py.
Moving socket to $rpc_py simplifies calling it in multiple
parts of the script.

Change-Id: Id1ba86740237c32a6f0259562c0b5c11d917bbcf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453643
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 14:43:59 +00:00
Karol Latecki
22c3befd3d scripts/rpc.py: fix pipe for whitespace separated list arguments
Ordinary python str.split() will by default split on each whitespace.
This includes arguments for rpc.py which expect a whitespace separated list,
eg. construct_raid_bdev -b "Bdev_1 Bdev_2".
shlex.split() prevents splitting on quoted arguments.

Change-Id: If37eb87cf82a161263f2b247baff5c8a77c43efc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453652
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 14:43:59 +00:00
Jim Harris
211cb4c250 test: don't call python explicitly
We use python3 now, not python.  This is important
because in next patch we will use a new 'aliases'
keyword argument in the argparse API that's only
available in Python 3.

While here, clean up some documentation that was
instructing users to call python explicitly instead
of just invoking scripts/rpc.py directly.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0c11e131cd56f7c994aa8fe21b4b00ae33489fdd

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 14:22:19 +00:00
Ben Walker
70e3aac333 doc: Improve title of memory management concept page
Change-Id: I4690fb85b7ce4349395a1c113f130d9ed30d7bcd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 12:26:29 +00:00
Ben Walker
1589ce9cfb doc: Improve title of vhost concepts page
Change-Id: I78e13fdea835882ba082caa2899c119215098073
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-08 12:26:29 +00:00
Darek Stojaczyk
005e18a881 autotest/common: silence hash output
We've got a bunch of `if hash` checks that generate
unnecessary output, e.g.:

$ if ! hash ceph; then [...]
/var/jenkins/workspace/Other_systems/centos7_autotest/
spdk/test/common/autotest_common.sh: line 28: hash: ceph: not found

We now silence that output in cases where we do expect
the binary to be unavailable and handle that case, but
we hold off in cases where `if hash` is just a sanity
check (e.g. `hash sgdisk` in part_dev_by_gpt() which
can make the entire function return immediately and do
nothing).

Change-Id: I7cbfada858aa86d57cbe182ea7f9b222debc8b88
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 10:58:51 +00:00
Jim Harris
fabd7fbb41 nvme: remove qpair_disable
This transport function is a complete nop now, so
remove it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5cc6ac75795a3cf5311f24e2ac293fb53d4b9f8c

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453487
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-08 01:44:20 +00:00
Jim Harris
783a2a20f1 nvme: add transport_qpair_abort_reqs
This will allow us to move more of the reset-related
functionality to the common layer, as part of enabling
resets for fabrics controllers.

The transport qpair_enable and qpair_fail functions
acted similarly - so those are both removed now and
replaced with this new qpair_abort_reqs function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9486630ad5b807239b0b5bcde50e8cfd313695d3

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-05-08 01:44:20 +00:00
Jim Harris
5d431efd6d nvme: move is_enabled logic to common layer
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idd938f255226256d864f70921ecd70c54769b9b2

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453485
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-05-08 01:44:20 +00:00
Jim Harris
4dd2fae3ad nvme: move qpair is_enabled flag to common layer
Upcoming patches will move the actual is_enabled
logic to the common layer as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9fd4d8712280295f57134ad66f8ccbfe9736d30a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-05-08 01:44:20 +00:00
Jim Harris
74aa552ef9 nvme: make helper function to abort outstanding err reqs
The nvme_qpair_disable functions will be going away in
an upcoming patch, so move this one bit of functionality
into a helper function in advance.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I61c2de535c2230b988d56dea13b00f39cb59dcfa

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-05-08 01:44:20 +00:00
Jim Harris
f366e261a6 nvme: abort aers at common layer
We submit AERs to all controllers - both pcie and
fabrics.  But currently we only manually abort the
aers when disabling the qpair for pcie.  Make this
common instead by creating a new transport function
for aborting aers.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1e926b61b8035488cdc6e8cb4336b373732f985e

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453482
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-05-08 01:44:20 +00:00
Jim Harris
a5c54a1f6a nvme/tcp: alloc requests when qpair is created
This allows us to reconnect the qpair in the future,
avoiding reallocating memory every time.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0b86d89d8ebee27125b6081fd3845718e1118d5a

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453481
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 01:44:20 +00:00
Jim Harris
14e67af3c5 nvme: rename reinit_io_qpair to reconnect_qpair
This better explains what the function is doing,
and makes the name more general so we can use it
for the adminq as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6b55761cb141a9a79cdef876be47995d8813b312

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 01:44:20 +00:00
Jim Harris
3a1b2ec262 nvme/rdma: alloc req/rsp during construct (not connect)
This moves us towards not freeing and reallocating
this memory if and when we reconnect the qpair.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic20d3c221442f6206d161760a8bfa7f9b8989d4c

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 01:44:20 +00:00
Jim Harris
6949c71dca nvme/rdma: separate req/rsp allocation from registration
This will simplify some upcoming changes to reconnect
a qpair.  In these cases we only need to re-register
the memory - we shouldn't have to allocate it again.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id8adff313f191fbf11d7502127a2b961f2ca2f6e

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-05-08 01:44:20 +00:00
Vitaliy Mysak
ac2b06c809 ocf: adopt examine to asynchronous construct
Make examine work with asynchronous bdev startup.

We need to count references of bdevs that are being examined,
  for the case when single cache bdev is referenced by multiple OCF
  vbdevs.

The construct function is not in fact asynchronous yet,
  but will be after adopting "trylock" API. Adopting trylock
  requires a lot of changes, so this patch is really to split
  implementation.

Change-Id: I6ac091f2dd48462e74aa89cf54acb0306c30673b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450033
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-05-07 19:29:41 +00:00