8832 Commits

Author SHA1 Message Date
Tomasz Zawadzki
7d6705f833 test/blockdev: run bdevio tests via RPC
With this change, bdevio tests are not ran immediately
after starting the application.
This will allow to configure new bdevs via RPC and
run tests on demand during script's execution.

Change-Id: I9740fca92c26cb2a39fbdc7cfb132ad53cec3787
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-19 03:39:04 +00:00
Tomasz Zawadzki
1599301b7b test/blockdev: add trap for bdev_svc process
There was a gap where nbd tests could fail, but without
trap set the app would still be present.

When here removed trap from end of the script since,
no part of the script set it up.

Change-Id: I0c435c7cd603410c2cd7a034bf955ab424e413f0
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457615
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-19 03:39:04 +00:00
Chunyang Hui
e3d21c7778 Opal: Optimize key creation and remove dev->dev_key
Change-Id: Iaf20c8ec0d208e03269406b62608d981d84cc48c
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-19 00:28:57 +00:00
Maciej Szwed
622127d7e1 blobstore: Make possible to remove snapshot if there is only one clone
Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.

In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
 4a. Add Clone1 entry to the Snapshot2 clones list
 4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
   for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
   clusters map
9a. If Snapshot2 is present switch parent ID and backing
    bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
    SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 13:19:32 +00:00
yidong0635
48cec0b127 vm_setup: fix scan-build not found on ubuntu18
Report issue on ubuntu18 :
"spdk/autobuild.sh: line 55: hash: scan-build: not found"

Check that ubuntu18 should install this package:
apt-get install -y clang-tools, to fit version of ubuntu18
and later.

Move clang-analyzer which for fedora to vm_setup.sh as well.

Change-Id: Ia702c492f8b0f64705c7c15ee57a861ca14521f9
signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 12:55:25 +00:00
James Bergsten
8785d5052d nvme: spdk_nvme_ctrlr_alloc_io_qpair extensions
Adds fields to structure spdk_nvme_io_qpair_opts.

These fields allow specifying the locations of memory buffers used
for the submission and/or completion queues.

By default, vaddr is set to NULL meaning SPDK will allocate the memory to be used.

If vaddr is NULL then paddr must be set to 0.

If vaddr is non-NULL, and paddr is zero, SPDK derives the physical
address for the NVMe device, in this case the memory must be registered.

If a paddr value is non-zero, SPDK uses the vaddr and paddr as passed.

SPDK assumes that the memory passed is both virtually and physically
contiguous.

If these fields are used, SPDK will NOT impose any restriction
on the number of elements in the queues.

The buffer sizes are in number of bytes, and are used to confirm
that the buffers are large enough to contain the appropriate queue.

These fields are only used by PCIe attached NVMe devices.  They
are presently ignored for other transports.

Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Change-Id: Ibfab3939eefe48109335f43a1167082dd4865e7c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454074
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 12:19:41 +00:00
Darek Stojaczyk
e58dfd5669 configure: silence gcc errors on rte_vhost checks
Redirect the entire gcc output to null for rte_vhost test
build - this gcc will be run even without any custom options
passed to `configure`, so don't bother the user with error
messages when he didn't do anything wrong.

Change-Id: I05136556ab81d78a6f27edd150b68f6e2b5cd8b5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457779
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 12:18:16 +00:00
Darek Stojaczyk
b18f19d8a7 configure: run rte_vhost gcc check explicitly with -Werror
We check rte_vhost version by trying to compile a program
using a public rte_vhost API function. We only compile the
program - without linking it - because rte_vhost has
a number of dependencies on other libs and we want to keep
the configure script simple. However, the function could
be still implicitly declared by the compiler, making the
compilation succeed when it should not.

To fix it, compile that test program with -Werror.

Change-Id: I0f187dfec909f7cd5cacb7b7a2fc21f5816c4632
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457778
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 12:18:16 +00:00
Evgeniy Kochetov
9e3d841d3e nvmf: Fix connect command SQ size validation for IO queues
SQSIZE parameter validation in Connect command was broken because QID
field in qpair was used before intialization.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I8a0b359937d661df3b9888e6084e7d0b4a9056ea
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 11:39:29 +00:00
Shuhei Matsumoto
d1a96e7cde iscsi: Fix UT failure due to uninitialized value detected by valgrind
In a test case of op_login_session_normal_test(), it is expected that
the tag of the session has any value different from the connection but
the tag of the session was not initialized.

  Fixes #826

Reported-by: lorneli <lorneli@163.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I62f7a4bd301c5a74ad6ba10b1b448189f750b324
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458339
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-18 06:55:44 +00:00
Shuhei Matsumoto
94f67dd5ae dif: Change data_len of spdk_dif_set_md_interleave_iovs to be remaining length
This patch changes the meaning of the data_len parameter of
spdk_dif_set_md_interleave_iovs from `Expected data length of the payload`
to `Expected length of the newly read data in the extended LBA payload`.

This change will align the parameters of spdk_dif_set_md_intereleave_iovs
to of spdk_dif_generate_stream.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7f9c45e78be977625713acb79d2ae82d4375f419
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457543
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 04:25:33 +00:00
Shuhei Matsumoto
df2164f6f4 dif: Merge single and multiple iovec cases of spdk_dif_generate_stream
As same as spdk_dif_set_md_interleave_iovs, no noticeable performance
difference was observed even if dif_generate_stream and
dif_generate_stream_split are merged into the latter.

spdk_dif_generate_stream will be updated in subsequent patches anyway.
Hence merge them into dif_generate_stream_split.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I77b588527bc7533dc38a36e450972758ce57d6bb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 04:25:33 +00:00
Shuhei Matsumoto
759a700f2a dif: Merge single and multiple iovecs cases of spdk_dif_set_md_interleave_iovs
Even if the code for single iovec case is removed, no performance
difference was observed when inserting/striping DIF for iSCSI target.

And the subsequent patches will add more complex operation and hence
simplifying the code before them will be valuable.

So this patch merges dif_set_md_interleave_iovs and
dif_set_md_interleave_iovs_split into the latter.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I784b9e04f0983ed91c7aa66bbc7ceaed4406e3ac
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-18 04:25:33 +00:00
paul luse
11bbb26e4f lib/reduce: use helper function to get correct length of logical map
When setting the offset into pmem for the chunk mamps, the wrong
value was used for the length of the logical map resulting in
map corruption after extended fio run.

Fixes issue #813

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb51cf8f38a04f05f69595ebd510ced7f1b608f3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-18 04:06:41 +00:00
paul luse
d571336b4b lib/reduce: add debug helper function
Prints key info about a volume and its PMEM. As the metadata is
mapped, there's no real reason to dump all this to a file, it
can be accessed via gdb using the addresses printed out.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2c7b8c32b6142ba0fded623a660b4d7d0c5b19bc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458117
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-18 04:06:41 +00:00
Wojciech Malikowski
cda27398e5 examples/nvme/hello_world: Optional VMD enumeration
New flag -V was added for VMD enabling.

Change-Id: I2d090a70b9d9b2bd996467a9b3da908f6eed1062
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457601
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:59:36 +00:00
Wojciech Malikowski
de6a68e031 examples/nvme/fio_plugin: Init VMD during spdk_fio_setup()
Try to enumerate VMD devices in fio_plugin.
New flag enable_vmd was added to fio config.

Change-Id: I5546665719e4ef2b169d403db8bf0398e834dbc4
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 03:59:36 +00:00
Wojciech Malikowski
8b29e8bc7f examples/nvme/identify: Optional VMD enumeration
New flag -V was added for VMD enabling.

Change-Id: I0a31994acd5fd677366e8d5e49ad9a3583670c2c
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456967
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 03:59:36 +00:00
Wojciech Malikowski
9d92b9d420 examples/nvme/perf: Optional VMD enumeration
New flag -V was added for VMD enabling.

Change-Id: Ie0feae1bdfce39ad246f57411d00587c6417d21d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450926
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 03:59:36 +00:00
paul luse
46523f33e0 log: passing user-defined log
Change-Id: I993e15a6e75029b0717960d5da31325e7f3522c1
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456407
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:56:50 +00:00
Tomasz Zawadzki
f87568a095 test/bdevio: respond with error via RPC on test failure
Previously RPC returned just number of cases that failed,
meanwhile return from the command was 0.

Now whenever there are failures, error is reported via RPC.
This will allow to use it in bash scripts with set -e used.

Change-Id: I34997dd5f08b561bc55f391287f9942861bda57a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457614
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:40:50 +00:00
Tomasz Zawadzki
1891f2e5c5 test/asan: supress memory leaks in external libiscsi
There are some leaks in libiscsi that is external to SPDK.
With this patch, any leaks in libiscsi will be suppressed.

Please see examples of the leaks right now when running
bdevperf in initiator.sh test:

Direct leak of 176 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c60edb4 in scsi_cdb_inquiry (/usr/lib64/iscsi/libiscsi.so.8+0x13db4)

Indirect leak of 256 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c603a84  (/usr/lib64/iscsi/libiscsi.so.8+0x8a84)

Indirect leak of 52 byte(s) in 1 object(s) allocated from:
    #0 0x7fd64dddfe50 in calloc (/lib64/libasan.so.5+0xeee50)
    #1 0x7fd64c60d755 in scsi_malloc (/usr/lib64/iscsi/libiscsi.so.8+0x12755)
    #2 0x7fd64c6097f8  (/usr/lib64/iscsi/libiscsi.so.8+0xe7f8)

Change-Id: I80ef23b1464841f683344c7aab99f1658a46cd36
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456766
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:40:50 +00:00
Tomasz Zawadzki
1bed9c1f1a test/asan: preload ASAN for fio
ASAN needs to be LD_PRELOADed before SPDK fio_plugin
in order to analyze its code.
Just adding that will report any issues in fio binary as
well as the fio_plugin.
To prevent known fio leaks from affecting the results,
a suppression list for LeakSanitizer (used in conjunction with ASAN).

At this time the suppression list contains known leaks
for fio 3.3. The list might need adjustments as fio
version is updated.

Side note. Even though it is possible to specify directory
to ignore ("leak:/usr/src/fio/"). Which in theory should
suppress any leaks in fio. It has side effect of hiding
SPDK leaks as well, since the fio_plugins leaks are
seen as coming from /usr/src/fio/ioengines.c.

See below for examples of each suppressed error:

Direct leak of 42 byte(s) in 4 object(s) allocated from:
    #0 0x7f9d52f3e320 in strdup (/lib64/libasan.so.5+0x3b320)
    #1 0x41f267 in get_new_job /usr/src/fio/init.c:490

Direct leak of 914936 byte(s) in 10397 object(s) allocated from:
    #0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
    #1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214

Direct leak of 608 byte(s) in 19 object(s) allocated from:
    #0 0x7f74422e8ca8 in __interceptor_malloc (/lib64/libasan.so.5+0x10dca8)
    #1 0x44c4e1 in add_to_dump_list /usr/src/fio/parse.c:1039
    #2 0x44c4e1 in parse_option /usr/src/fio/parse.c:1098

Direct leak of 173 byte(s) in 20 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44b50d in __handle_option /usr/src/fio/parse.c:718

Indirect leak of 111925528 byte(s) in 1271881 object(s) allocated from:
    #0 0x7f74422e8ea6 in __interceptor_calloc (/lib64/libasan.so.5+0x10dea6)
    #1 0x46402e in log_io_piece /usr/src/fio/iolog.c:214

Indirect leak of 171 byte(s) in 19 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44c4ed in add_to_dump_list /usr/src/fio/parse.c:1040
    #2 0x44c4ed in parse_option /usr/src/fio/parse.c:1098

Indirect leak of 167 byte(s) in 19 object(s) allocated from:
    #0 0x7f744227153d in strdup (/lib64/libasan.so.5+0x9653d)
    #1 0x44c502 in add_to_dump_list /usr/src/fio/parse.c:1042
    #2 0x44c502 in parse_option /usr/src/fio/parse.c:1098

Change-Id: I9b5811993508421be50b12af160645c77ea93d7e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456315
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:40:50 +00:00
James Bergsten
18b504afe7 CHANGELOG.md: Added information on NVMe additions that were recently merged.
Signed-off-by: James Bergsten <jamesx.bergsten@intel.com>
Change-Id: I629a6d1c94467d559fdbd4de4e6a397ec3e7a1d1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457861
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-18 03:30:45 +00:00
Darek Stojaczyk
036ca4acb6 test/lvol: wait for bdevs to be examined
The lvol "tasting" done as a part of bdev examine is
done asynchronously in background, so we need to sleep
for a while before checking the list of created lvol
bdevs.

Fixes #801

Change-Id: Iab94ac5c66d329cd93955b53262fc03c6b3c9f3a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458228
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-18 03:29:52 +00:00
yidong0635
9c62e1c132 ioat_spec: fix gcc9 warning
Warning: taking address of packed member of ‘ ’ may result in
an unaligned pointer value [-Waddress-of-packed-member], to compatible
with new compiler,
Add "__attribute__((aligned))" for a registered struct.

Change-Id: I82a499f6534389d39bd64ce31e51aaffee292301
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458242
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-17 14:09:03 +00:00
Tomasz Zawadzki
030ec62a00 doc: move SPDKCLI to tools section
SPDKCLI is not considered experimental, so it was moved
to tools section.

Since it was last one in "Experimental tools",
that section is now removed.

Change-Id: I27ada7e2964c756e997abafe1fabfa5b5fd0d9bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-17 10:10:59 +00:00
Xiaodong Liu
10d659a748 test/json: cover get_notification_types RPC
Change-Id: I703febc5999793c7deb6435648b7d857232bff6f
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456775
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-17 10:08:39 +00:00
Karol Latecki
5afa401230 test/iscsi: add iso option to rpc_config test
Make it able to run as standalone.

Change-Id: Ib8d5dcd300911978f3bd9192c00049a3617438f4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:51:52 +00:00
Pawel Kaminski
6520c0e70c test/vhost: Fix. Do not remove results dir if it is needed.
Results dir is used by jenkins job.

Change-Id: I4821eab2578cae68cb9070347a877a1d2fbd0624
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457928
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:51:48 +00:00
Xiaodong Liu
26e5165ecc test/iscsi: cover wait_subsystem_init RPC
Change-Id: Ia1f6eee4b4f74652116d04298e4b0321aba160a1
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456759
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:19:11 +00:00
Karol Latecki
fb9c4ee651 test: replace backticks with dollar-parenthesis syntax
Scripts were using a mix of two approaches, lets unify that so
just dollar-parenthesis syntax is used.

Change-Id: Id093b6c82d1f766ba6af13bed720977eceaa7ffc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457744
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:19:03 +00:00
Karol Latecki
ffc26bc3b6 test/vhost: fix pep8 W605 warnings in python scripts
Latest pep8 versions detect regex syntax as W605 wargnins -
"Invalid escape sequence".

Adding "r" prefix to regex string fixes this.

Change-Id: I82cd9c260e3c5242ed3c9886a0f6c8ac77daf465
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457923
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:18:56 +00:00
Wojciech Malikowski
5089a9c5f9 lib/vhost: Fix unaligned pointer value error
Reported by clang:

rte_vhost_compat.c:114:36: error: taking address
of packed member 'payload' of class or structure
'vhost_user_msg' may result in an unaligned
pointer value.

To fix it, just remove the extra unaligned pointer
and inline all its accesses.

Change-Id: I7e4ab536b87ab02a4ea12c55d55a6e495c3091ca
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457559
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-17 07:18:48 +00:00
GangCao
dc1e8d71f1 ut/iscsi: add the login related UT code
This patch adds the UT regarding below function
iscsi_op_login_session_normal()

It covers the target related, connection related
and session related operations.

Change-Id: I570df82df850a0b359b2d57b4e1694d8cb16de37
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457925
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-17 07:18:42 +00:00
Changpeng Liu
fdd365ed79 UT/blobfs: add asynchronous Write/Read unit tests
Change-Id: I4a64e7b883274f3bcd5d4a7ae151156d61acae01
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-17 06:37:54 +00:00
JinYu
a298d77ccf test/nvme: fix printf param in deallocate_test()
stay the print parameter same with it in the deallocate_complete().

Change-Id: I21734f2b3097fd3a1006e390ab87b0578f0cbc2a
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458029
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-17 05:30:34 +00:00
GangCao
f874737792 iscsi: set the rsph with correct error information
In several error cases, the status_class and status_detail
is not properly set.

Change-Id: Ie9cf8ec13b971d7295862872e8c7e834d08e7f14
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-17 04:56:31 +00:00
Ziye Yang
ba3adc4367 iscsi: return the correct error code in spdk_iscsi_read_pdu
We shoud not always return SPDK_ISCSI_CONNECTION_FATAL
in spdk_iscsi_read_pdu function.

Reason: In iscsi_conn_handle_incoming_pdus, the loop
directly return only rc==SPDK_ISCSI_CONNECTION_FATAL.
But it masks all the necessary information. So we would like
to keep some information of the return value for spdk_iscsi_read_pdu,
and we can use error log to track those information.

Then we can return SPDK_ISCSI_CONNECTION_FATAL as the error
return value for iscsi_conn_handle_incoming_pdus function.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I4c40fcb27052b55cb92e06273701a881def18e12
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457078
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-13 06:04:31 +00:00
yidong0635
858718c31d target/filesystem: Add partprobe after parted
Partprobe is a program that informs the operating system kernel
of partition table changes, by requesting that the operating system
re-read the partition table. For instance, if you create a new
partition on one of your disks using parted, you should run
partprobe afterwards to make the kernel aware of the new partition
configuration.

This is useful for issue #799.And it's compatible with other systems.

Change-Id: Icd4c85193bd9d9e6c2b32b8463e75c7a6ff06f34
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457735
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-13 00:49:52 +00:00
Shuhei Matsumoto
c758dc088a nvmf: Reject bdev with separate metadata to attach to subsystem
NVMe bdev module support separate metadata now but NVMf subsystem
cannot process bdev with separate metadata yet.

Hence reject any bdev with separate metadata to be attached
explicitly by this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I793c6c5f61deb766d7bf427ff67ccc57a48974cf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457167
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-13 00:48:11 +00:00
Changpeng Liu
a47e7de332 bdev: rename _spdk_bdev_io_split to spdk_bdev_io_split
_spdk_bdev_io_split() will be used to requests that have IO buffers.

Change-Id: I5ba0b83e47b84066e6c5032de2e02404155896d3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457549
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-13 00:44:34 +00:00
Darek Stojaczyk
59c8eeddf6 test/vhost: remove virtio-pmd tests
Those tests are absolutely awful. For the last few days
they've been randomly failing way more often than usual
and we have no way of figuring out what the root cause is.
We know it's 'discover_bdevs' that hangs, but since it
doesn't produce any output (all redirected to null), we
can't get any relevant information out of it. The only
way to move forward is to rewrite this entire test from
scratch.

Change-Id: Ie874110008f82f3ca0dd62f733b0dac73160fbb8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457766
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-12 10:17:58 +00:00
Darek Stojaczyk
4cc2d1d814 test/iscsi: don't use set +e
We used `set +e` before bdev deletion RPCs and
`wait $fio_pid`. Disabling errors from those RPCs
doesn't make sense. We expect FIO to fail due to
bdev hotremoval, but the bdev hotremoval itself
must succeed.

Besides, it's easy to forget to undo `set +e` so
don't use it. For `wait` we can use the following
paradigm instead:

```
rc=0
wait $pid || rc=$?
```

Change-Id: I8716bd861eae85d7a8590d4bd26a65cfa10e0dc1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457469
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-12 05:29:25 +00:00
Darek Stojaczyk
1a8ee925b0 rte_vhost: fix compilation against dpdk master
struct ether_addr was renamed to struct rte_ether_addr
in latest DPDK master, but our internal fork of rte_vhost
still used the old name, which can be now a non-defined type.

Together with the struct, the RTE_ETHER_ADDR_LEN define
was renamed as well, so we'll now check if it's defined and
we'll manually define struct ether_addr to keep the old
rte_vhost working.

Change-Id: I78b8104ed3bfe03397881a94f0f8bee14f9efae8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457609
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-12 05:29:02 +00:00
Changpeng Liu
3ec061800f nvmf: add a persist through power loss configuration file when constructing NS
For reservation feature in NVMoF, we can't support the persist through
power loss feature, now we will add the configuration file parameter
with Namespace, after users set the configuration file parameter with
one NS, then the PTPL feature can be enabled.

Change-Id: Id72699093f7e68318b9529f7bacc5c9804f7f86b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-12 00:30:03 +00:00
Shuhei Matsumoto
37bdd0e87f scsi: Add data offset to DIF context separately from start block address
This patch uses the change by the last patch to initialize DIF
context in SCSI layer. Besides this patch changes the name of a
parameter from offset to data_offset to clarify the meaning.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I54bf1168ec5959432aa15dae0360c0640138b033
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Shuhei Matsumoto
73204fe2e5 dif: Add data offset to DIF context separately from start block address
Data offset are intended to correspond to DATAO in NVMe/TCP and
Buffer Offset in iSCSI.

Previously for iSCSI, buffer offset had been merged to start block
address, but passing buffer offset separately from start block address
clarifies the logic more.

On the other hand, for NVMe/TCP, passing DATAO separately from start
block address will be critically important because DATAO will bave any
alignment and will be necessary to use for not only reference tag
but also guard computation.

This patch adds data_offset to struct spdk_dif_ctx and adds it to the
parameters of spdk_dif_ctx_init(). ref_tag_offset is also added to struct
spdk_dif_ctx and it is computed by dividing data_offset by data_block_size
and is used to compute reference tag.

The next patch will use this change when getting DIF context in SCSI.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id0e12ca9b1dc75d0589787520feb0c2ee0f844a5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Shuhei Matsumoto
d9ec66dfb6 dif: Use variables instead of numerics to test DIF insert/strip in UT
DIF UT code had used numerics extensibly for complex DIF insert/strip
UTs. But numerics was error prone and difficult to follow the logic.
Hence this patch changes numerics to variables in DIF insert/strip UTs.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I40cb27167c068855dc5d1eb9e0f36818d39c73fa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00
Shuhei Matsumoto
3e52bdf722 dif: Fix wrong data_offset of spdk_dif_generate_stream in UT
data_offset to spdk_dif_generate_stream() was wrong but the function
worked as expected because the wrong data_offset was in the same
data block as the correct data_offset.

But the subsequent patches will refine spdk_dif_generate_stream()
and this bug will cause UT failure. Hence fix the bug up front.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I45f2302deaf377fda0ec5454c61fb463231a39db
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/457538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-11 18:53:58 +00:00