11141 Commits

Author SHA1 Message Date
Seth Howell
8d945f5256 test/blockdev: move configuration into the script.
Change-Id: I795e729f6e0497a56a0328bb308bc9d32f44922b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478241
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
bde783d460 autorun_post: fail if not all per-patch tests ran.
This will keep us honest with our testing strategy and help prevent us
from unintentionally skipping tests.

Change-Id: Ie9dda265b48814c5e8aafe0504205310b886eea9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477981
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
e0f9038893 test/blockdev.sh: change the way we run fio tests.
Previously, we conditionally ran one test in per patch, and another in
nightly tests. However, the only difference between these tests was the
configuration options sent to FIO. All other tests in the build pool
follow a pattern where they simply change the options and run the same
test in nightly and per-patch.
Following that pattern makes it easier to set a specific list of jobs
that must be run before a tet run is considered complete.

Change-Id: I6b2f49114155fdfa1d1e07faf45f03e64572eeb8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
df254d5a25 autorun_post: add a file for tests we don't run.
Change-Id: I8beac9959d08026ee984833d0e90ec17381a66e5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478240
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
ea781d6dbb test: report test domains for tests.
This will allow us to run test cases under multiple suites without
clogging up the logs too much, but it will also preserve information
about which test suites were run or (more importantly) not run.

Change-Id: I2434a54a0877ae36b9f84bfab8a62653ac1172f8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477367
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
4e3104d0e9 autorun_post: rewrite the aggregateCompletedTests function.
This whole function was just written poorly. Variables weren't named
correctly, little effort was made to use proper python programming
constructs, and there was a lot of redundant code in the function that
didn't do any good.

And I can say that because I wrote it.

Change-Id: I08576698d99c35c43348b61762791415f79eebf2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
9436ab59ba nvme/rdma: inline buffers for all host to ctrlr ops
Not inlining all host to controller operations breaks the target within
the context of fused commands. This issue was discovered when enabling
the compare-and-write fused command. Only the write command buffer was
being inlined which caused the write to jump the compare in the
transport specific state machine on the target side before our fused
command checks in the generic code.

Change-Id: I9e52ae6160e01ffd36d20429ffc8459491c729ef
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482001
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:41:47 +00:00
Changpeng Liu
2a8281fdfd thread: free message event after executing the callback successfully
We should check the thread's state at the end of message callback, or
we may leak the message memory in case the thread was set to exit state.

Change-Id: Ifb67c3b5c39440c411eca1d045c11e8aa6c514cc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482206
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-22 01:25:03 +00:00
Jacek Kalwas
de4bf95443 lib/nvmf: put explicit transport name on dump
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ie37c1d713f0e1b0767c4b40c1055b86d9de220af
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481954
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-21 23:16:57 +00:00
Jacek Kalwas
0651753ce8 lib/nvmf: introduce function to get transport name
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ide89666bfd856d42ca5cb535e8a29716f787ea3f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481953
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-21 23:16:57 +00:00
Jacek Kalwas
7a35a678b8 lib/nvme: extend trtype to str with CUSTOM
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I519bb6bf0e930e0cd977ef4b5133bbdd7ca8af86
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-21 23:16:57 +00:00
Mateusz Kozlowski
a6ff97602c bdev/zone: Zone append handling added to vbdev
Changed write function to be able to work in zone append mode.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I3af98f1d8e69eac6922e19addebabbc6ce847c63
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-21 08:43:31 +00:00
Mateusz Kozlowski
68c3a79a15 bdev/zone: Add finish zone handling to zoned vbdev
Handle finish zone management function to zoned block vbdev

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Iffe24572609e741a79b6bb92f72b56cede4906ca
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-21 08:43:31 +00:00
Mateusz Kozlowski
011cc5d61c bdev/zone: Add close zone handling
Handle close zone management function to zoned vbdev

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I001e25a157ba738724012ca366ef901016733944
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-21 08:43:31 +00:00
Mateusz Kozlowski
d8ccf6e7e5 bdev/zone: Read IO handling
Added handling of reads to a zone; blocks outside of a write pointer
are zeroed out.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Iaf60d6ff74caf9831998862e048446ccd0a0fa3f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468038
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-21 08:43:31 +00:00
Mateusz Kozlowski
ab1641eb30 bdev/zone: Write handling
Added handling of write commands, including zone state changes and write
pointer updates.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I576ca18b52474bb299c20296a7561f25e5afa85b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468037
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-21 08:43:31 +00:00
Mateusz Kozlowski
d530c103ab bdev/zone: Open and reset zone handling
Add handling of two of the zone management commands for vbdev.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I2868a96d15acbc2d13d18dea405b6d98cb4573d6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468039
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-01-21 08:43:31 +00:00
Jim Harris
eaa984801e bdev: break spdk_bdev_io_get_buf into multiple functions
This will allow for some reuse in future patches adding
support to get auxiliary buffers from the bdev memory
pools.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icd4e71d0e918cfe940c749e40cd1cc6c560e25a9

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482022
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:02:45 +00:00
Jim Harris
03171deee8 bdev: add bdev_io_get_buf_complete helper function
This will be helpful in future patches where auxiliary
buffer support is added, and we have a different
type of callback to invoke.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I81d2cf3057e706a0e68de87a20b24c194205bbd5

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482021
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:02:45 +00:00
Tomasz Zawadzki
7167f8d334 lib/blob: save sequence immidietly on bs_load/unload
Assigning seq to ctx was done very late in the process.
To keep future functions lean and without the seq,
it is assigned immidietly after starting.

Only functions in load path that require separate
seq argument are those passed directly to read/write
device operations.
Rest of them can just use spdk_bs_load_ctx.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2bd610dc4c7b4a7b0c3de92391922475c514326a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481899
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:02:00 +00:00
Tomasz Zawadzki
bbbe586b28 lib/blob: make passing ctx more explicit
No functional change is done in this patch.

Most of the functions already translate cb_arg to ctx and
use it, but then just pass cb_arg.
This will make it clear that it is ctx that is passed around.

Along with simplifying some of changes in next patch,
where arguments of functions will be cut down just to
the ctx.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If7d8ed38dc92175d867a2231ab2ebd4f2499efcd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482006
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:02:00 +00:00
Tomasz Zawadzki
994d4c38ba lib/blob: move generation of metadata into separate function
This patch creates new _spdk_blob_persist_generate_new_md()
function that is responsible for generation of new metadata
from current state of blob.

Functionality so far is unchanged.

This is preparation for later in the series where new
extent pages will be written out to disk before metadata pages.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I84158cb8316a881a6170ac37e151a60aaa9d7369
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479848
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:02:00 +00:00
Tomasz Kulasek
c879a8eed4 test/nvme: cuse nvme-cli check if oacs firmware supported
When firmware upload is not supported, fw-log may return
random data.

This patch compares an output of fw-log only if oacs.firmware
is set.

Change-Id: I74d7766ca6ebdf0a94f200a35e6215ed44ef1062
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481670
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-20 10:01:00 +00:00
Tomasz Kulasek
5700688366 test/nvme: check nvme-cli output for cuse device
This patch compares an output of nvme-cli for NVMe Linux
driver with SPDK NVMe CUSE implementation.

Some commands may fail or provide different outputs:
 1. smart-log -- data may change,
 2. list-ns, list-ctrl, get-feature -- depends on spec
    version of device (unknown or reserved field for spec 1.0),
 3. fw-log -- may return random data for fws if firmware upload
    is not supported -- this case is removed now and will be
    added in next patch

Change-Id: I47873f004efbe333e1de02010ba594baf253d093
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-20 10:01:00 +00:00
Maciej Wawryk
53147e0dc8 test: fix issue with changing device names in waitforblk function
During test before upgrading our vm's to Fedora31 there were issue with failing
nvmf test on function waitforblk. In some cases hardcoded nvme0n1 name was not
found in system. Here is fix that change searching by name to searching by nvme
serial.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic5ee70804652a057fa26b8cc004b5227262d2122
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 10:00:06 +00:00
Seth Howell
9c659fb125 test/fuzz fix end of test behavior.
The fuzz jobs should fail if the nvmf or vhost target crashed while they
were running. Otherwise they should pass. Some recent changes to the
way killprocess and nvmftestfini work were causing this to break. This
change fixes that.

Change-Id: Id14fd47c2d8c7d91358c5e8a69e53ec4cf33ddce
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:59:49 +00:00
Pawel Kaminski
20c5fc9dce test/lvol: Add test to check 'clear_method unmap' behaviour
Change-Id: Ib14022bdce2ba21cd3fdc4be5bf156359bef1573
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:57:16 +00:00
Pawel Kaminski
6fc7f49c16 test/lvol: Add test to check 'clear_method none' behaviour
Change-Id: I5e4581a8d9b100c965d14dc3e263084c4b729af6
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467967
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:57:16 +00:00
paul luse
ca667d064f lib/blob: read clear_method from per blob metadata
On blob load, read in the saved clear_method option. If
BLOB_CLEAR_WITH_DEFAULT was passed in, use the setting stored
in metadata previously.  If something other than the default was
specified, ignore stored value and used what was passed in. If
ignoring a stored value, print a warning.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia0c81fa0adc175dfaeb74c06e1ac91dc6b27e9ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:57:16 +00:00
paul luse
ea69d6d6cc lib/blob: store clear_method in per blob metadata
Accept a clear method option on blob create by adding clear_method
to the opts structure passed in to _spdk_bs_create_blob(). Store
these 2 bits in md_ro_flags so that earlier versions without an
understanding of these bits can not alter metadata.

The new metadata values will be used later in the series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5440645ca20b426778d13b2e544b65dc2b3b83c7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-20 09:57:16 +00:00
paul luse
d68459fcb3 lib/bdev: Add comment to bdev_io_put_buf() when unsetting bounce buffer
To clarify what seems like it might be unintentional.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id14b49e8b7bc2372ecb8dd7579e6797a395c2cf2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477363
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-20 09:57:16 +00:00
Michael Haeuptle
bf4db834b9 lib/nvmf: Functional test for custom admin cmd handler
This test verifies the custom identify handler functionality
by querying the NVMe drive and comparing serial and model
number from the NVMe drive with the corresponding info
from the NVMF identify result.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I9d4edb1bf32550cbf5fbd45cbfa85f9afc408fc0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481910
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:56:35 +00:00
Michael Haeuptle
c9874b91a7 lib/nvmf: custom admin handler config
This commit disables the custom identify handler by default.
The user has to explictly enable this handler via the set_nvmf_config
RPC or conf file.

Change-Id: I767816ba7639ebe78683993408ce6db02c7620fe
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479603
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:56:35 +00:00
Michael Haeuptle
e93902a03d lib/nvmf: custom admin cmd handler oacs support
This commit sets the optional admin command flags
in the identify structure based on whether handlers
for these optional admin commands are specified.

Change-Id: If4aa36a414b0811dafaadbc1094e6c2628d21b39
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479446
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:56:35 +00:00
Jim Harris
4df28960e2 nvme: add SPDK_NVME_OPC_GET_LBA_STATUS
This optional OPC was added to the NVMe 1.4 spec.
Also add its corresponding bit in OACS.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I61d7e7f0210b0617aaaf932e8fac060df79f7d6b

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482016
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:56:35 +00:00
Michael Haeuptle
3fa2205647 lib/nvmf: custom admin cmd handler
This commit provides the capability to install a
    custom admin command handler for NVMF.
    It can be used to implement or replace NVMe admin commands that
    are currently not handled by the NVMF subsystem.

    The handler implementation is pretty generic and the handler function
    has to figure out what to do with the command based on the bdevs
    that are configured for the subsystem.
    In cases where admin commands need to be forwarded to an NVMe bdev,
    the commit provides functions that allow access to the underlying bdev.

    There is an example handler in lib/nvmf/custom_cmd_hdlr.c.

Change-Id: I4f9d538c53669c176a836e8bdd379db0070a87dc
Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479167
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <jacek.kalwas@intel.com>
2020-01-20 09:56:35 +00:00
Alexey Marchuk
d0cbb46875 dpdk: update submodule to include compilation fix for ARM
Change-Id: I9eefffd4f653f7aa23af4f7a914ca2f99df1cbfc
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-20 09:56:20 +00:00
Jim Harris
5a04e473b2 test/nvme: fix shellcheck error
Fix perf/common.sh to remove the useless cat operation.
Fixes shellcheck SC2002 error.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2abafa5c238d45de63b174c7705e28bd93d75822

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482005
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 18:07:30 +00:00
Pawel Kaminski
0cd9241a2d test/json_config: Freebsd supports python3 now so use it
Change-Id: I7a03dddfbc455f9d9374ce864dc3c9fab157d81e
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481889
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 14:07:39 +00:00
Pawel Kaminski
50e5d88317 test: shellcheck - ignore SC2010
SC2010: Don't use ls | grep. Use a glob or a for loop
        with a condition to allow non-alphanumeric filenames.

Replacing e.g. "ls /dev/nbd* | grep -v p" would be complex
and problematic as flag -I for ls is not working when
looking for pattern.

Change-Id: I9e88359d247fe5dcf952a3388b0e7f35164768c1
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 14:07:39 +00:00
Pawel Kaminski
13c6c8db6e test: shellcheck - apply rule SC2002
SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.

Minor fix in test_converter: don't use python3 as
    config_converter.py is executable.

Change-Id: I250e893b48fa95e94778dcc31deb91e6ac0b39c0
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 14:07:39 +00:00
paul luse
989261d6c3 module/crypto: load balance QAT qp assignment
For the latest QAT devices there are 3 CPMs each with 16 VFs and
2 qp each VF. To load balance for multi-thread operations we want
to assign each new queue pair (QP) on a per CPM (processing module)
basis so this patch assigns the next QP, for QAT,  by taking the
last + 32 modulo the total number of QP. This will results in each new
channel getting a QP on the next CPM.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iea608ada68517b6f2faecd45701c7aae6d23a2d0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477082
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 14:07:12 +00:00
paul luse
1a6878db61 module/crypto: break out device/qp code into helper function
Mainly to make it way easier to write the UT in the next patch
in the series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I00b00d8a83edfad51f3945c49764ae6442fb6f52
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477332
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-17 14:07:12 +00:00
paul luse
5db021089d module/crypto: split global device/qp list into one per PMD
Previously one global list of device/qp combinations was used
regardless of PMD and when assigned, the device name was checked
to make sure a matching one was pulled from the list.

Later in this series a patch will make use of having different
lists as we discovered a decent way to load balance QAT but the
implementation with all PMDs on one list was too complex.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I54dfbd0206a881d126831ba27a4ae05cdc6f7c11
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477152
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-17 14:07:12 +00:00
Tomasz Zawadzki
3219bc9a80 lib/blob: separate blob load md parsing from loading back_bs_dev
In current version, immidietly following parsing all
metadata pages an action is taken inform of loading the back_bs_dev.

Patches later in the series will add more metadata in form of
extent pages, which have to be read separetly from usual
blob metadata pages.

This patch add separation between the two steps,
so later a device read can be put between.

Additionally, _spdk_blob_load_final() when no snapshot was present
passed bserrno which was always 0. This patch just sets 0 directly
there as no errors occured at that point.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4a77527f90bb1de12f972591067b7a50926f39c9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476427
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-17 10:00:19 +00:00
Seth Howell
7430431518 changelog: update with new nvmf changes.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ia6b9414efd0af467a7c664e8ec77b7af03c9e891
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478876
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-17 09:01:19 +00:00
Karol Latecki
d82c680d60 test/nvme-perf: group kernel mode setup into one block
Group all kernel conditionals for various ioengines in
one if block.

Change-Id: I14f736c1ab8fce13effeb9ec06731b3d14859941
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-17 09:00:57 +00:00
Karol Latecki
73059a2a53 test/nvme-perf: shorten results path
Part of results_dir variable was unnecesarily repeated.

Change-Id: I282e1f7389f79aeccf44b09f257ef33842be517b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-17 09:00:57 +00:00
Karol Latecki
18fbf2e15d test/nvme-perf: add nvme perf option
Allow running tests with nvme/perf. This will be needed
for comparison with bdevperf and/or fio nvme plugin.

Change-Id: Ie351d1b3c9d8eae430161779f9818c04d87e6ed7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479821
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-17 09:00:57 +00:00
Ben Walker
63a60a0c4c nvmf/tcp: Fix r2t completion callback
This was calling a callback for another function which
attempted to release the request. The code only worked because
in the r2t case the cb_arg was set to NULL, and that makes
the request free function do nothing.

Change-Id: Id9ec30ceb0eaa41deb67aa995da5d6f786d9b9f0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479903
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-17 09:00:08 +00:00