Commit Graph

10383 Commits

Author SHA1 Message Date
Tomasz Kulasek
8e70c6e320 test/asan: suppress memory leaks in external libfuse3
There are some leaks in libfuse3 that is external to SPDK.
With this patch, any leaks in libfuse3 will be suppressed.


==1944861==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 32 byte(s) in 2 object(s) allocated from:
    #0 0x7fc26a462048 in __interceptor_realloc (/lib64/libasan.so.5+0xf0048)
    #1 0x7fc269bfcfe2 in fuse_opt_add_arg (/lib64/libfuse3.so.3+0x19fe2)

Indirect leak of 10 byte(s) in 2 object(s) allocated from:
    #0 0x7fc26a3ade60 in strdup (/lib64/libasan.so.5+0x3be60)
    #1 0x7fc269bfcfc6 in fuse_opt_add_arg (/lib64/libfuse3.so.3+0x19fc6)

SUMMARY: AddressSanitizer: 42 byte(s) leaked in 4 allocation(s).


Appropriate patch solving this issue was sent to libfuse, but
not merged yet.


Change-Id: I66625e155b78082f2f2c9790bf3f3b48c3c04f33
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475117
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-21 08:01:57 +00:00
Tomasz Kulasek
53184430a5 lib/nvme: fix do not use external_io_msg_qpair after free
Change-Id: I20ef8303c2fae6abf43d15ebb025ea368c0dfd67
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-21 08:01:57 +00:00
Shuhei Matsumoto
e79a29b264 lib/iscsi: Make spdk_del_transfer_task() return success/failure
After recent refinement of LUN hotplug, it is possible that
for large write I/O, primary task is freed doubly as a github issue
is reported.

However we could not notice the case because spdk_del_transfer_task()
had not return success/failure, and to make matters worse,
the second call of TAILQ_REMOVE() to the same header and instance
caused no error if the first call succeeded.

This patch changes spdk_del_transfer_task() to return success/failure.

Besides, the next after patch expects the stub of spdk_del_transfer_task()
returns true in the unit test, and hence do that.

The next after patch will fix the issue of double free of primary task
by using this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibc0b65723050362d5fafa913417b64393feb874e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-21 08:01:29 +00:00
Shuhei Matsumoto
fa2dd2722d lib/scsi: Move scsi_task_free_data() in a file to remove function declaration
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9c98ded51bf88cd199bace7b3703b86ed4a9c25b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-11-21 08:01:29 +00:00
Shuhei Matsumoto
16dbc19508 lib/scsi: Consolidate error paths in bdev_scsi_readwrite()
This simplifies the code a little.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idf53616bda21c82b82f9e9438f4dcb87ffaa59a6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-21 08:01:29 +00:00
Shuhei Matsumoto
ca0972d3b8 lib/scsi: Match parameters of bdev_scsi_readwrite() with bdev_scsi_sync/unmap()
We can pass bdev, lun->bdev_desc, and lun->io_channel to
bdev_scsi_readwrite() as same as bdev_scsi_sync/unmap().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib8b368da4da54b057ae09a91ab60a50388a5c684
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472517
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-11-21 08:01:29 +00:00
Shuhei Matsumoto
84e64cc9d7 lib/scsi: Pass SCSI task to SCSI layer to get DIF context
By the recent refactoring, SCSI task is configured when getting
DIF context from SCSI layer. Passing not CDB and offset separately
but SCSI task to SCSI layer is more concise and do in this patch.

In iscsi_send_datain(), we have to update task->scsi.offset for the
case that data is split into a sequence, but the update is no harm
because task has completed what it must to do.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I153352dfa7aa7325db4452f03d863df11b3e0cfa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-11-21 08:01:29 +00:00
Maciej Wawryk
f924c94b4c test: Shellcheck - correct rule: Use grep -q...
Correct shellcheck rule SC2143: Use grep -q instead
of comparing output with [ -n .. ].

Change-Id: Ibfce9633eda332c64b27914b2fde51c9a26ff7f3
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 10:03:15 +00:00
dongx.yi
94ec4785ef test/unit: Add test_nvme_pcie_hotplug_monitor to nvme_pcie_ut.c.
Add this test to increase code coverage rate.

Signed-off-by: dongx.yi <dongx.yi@intel.com>
Change-Id: I46533dcd1a528ae7385cdaa143aea9c396e2942c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472915
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-20 10:03:00 +00:00
Tomasz Zawadzki
4ea996ce19 nvmf/rdma: dont refer to rtransport when poll_group_create failed
This issue was found by code inspection.
It only occurs when pdk_nvmf_transport_poll_group_create()
itself calls spdk_nvmf_transport_poll_group_destroy().
Other transports at this time do not show this issue.

spdk_nvmf_rdma_poll_group_destroy() depends on
rgroup being assigned a transport, which is only being
done on generic nvmf layer using
spdk_nvmf_transport_poll_group_create()
after successful spdk_nvmf_rdma_poll_group_create().
When failure occurs during create, such assignment was not
performed so any references to rtransport will segfault.

Reported-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id54482d562bd6d7c71371306cf1de93bc05f4e8a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475002
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-11-20 10:01:43 +00:00
Yao Liu
c8ef5793e3 bdev/uring: add JSON config dump
Signed-off-by: Yao Liu <yotta.liu@ucloud.cn>
Change-Id: Ic060a62f20c97dcc0c52ad761f848074efaade3d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 10:01:07 +00:00
Yao Liu
bc7b39a5f0 bdev/uring: dump filename in driver_specific JSON
Signed-off-by: Yao Liu <yotta.liu@ucloud.cn>
Change-Id: I9a3bf72175d8bf9ede7149fb6ea14dc9931ce1f6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-11-20 10:01:07 +00:00
Yao Liu
81b4f2fec2 bdev/uring: add support for bdev_uring_get_spdk_running_config()
Signed-off-by: Yao Liu <yotta.liu@ucloud.cn>
Change-Id: If86ef2dad60260b5261fc05d075625dfa2ed0340
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474178
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 10:01:07 +00:00
Ben Walker
e475586b2a util: Add spdk_iovcpy, an iovec to iovec copy function.
Like memcpy, but works on two iovecs.

Change-Id: Ia1cf462a95690286f0c19325fc10937b9ba6baf3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:35:32 +00:00
Ben Walker
063252fa66 pipe: Add a utility for buffering data from sockets
When dealing with sockets most code in SPDK buffers
data into large chunks to minimize the number of
syscalls made. The pipe utility is designed to make
that easy.

Change-Id: Ie29966712bbfb43fb49457e042903cf45864e6c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465707
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:35:32 +00:00
Jim Harris
aa4856632e bdev: remove _spdk_bdev function prefix
The _ was supposed to indicate an internal function,
but really leaving off the spdk in the function
is the standard way to denote an internal function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I74facafb67b793502838b9c1b5f90aec2c88c69b

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475033
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-11-20 09:34:18 +00:00
Jim Harris
0e58d1549b bdev: remove spdk_ prefix from static functions
We try to reserve the spdk_ prefix for functions that
are part of the public SPDK API.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0a95ba0f9db04b588a2e948427f8a9f53a7b5740

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475032
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-11-20 09:34:18 +00:00
Jim Harris
fdfb4e1270 bdev: rename spdk_bdev_get_io to bdev_channel_get_io
This function is not part of the SPDK public API, so
remove the spdk_ prefix from it to make it more clear
this is internal to the SPDK bdev library.

Also add "channel" to the name since this function is
operating on the channel parameter.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I57897ab4bf601b90551259b7cf6efa63152ed02f

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-11-20 09:34:18 +00:00
Jim Harris
a1d68e9c02 bdev: remove spdk_ prefix from spdk_bdev_io_init
This is not part of the public SPDK API, so remove
the spdk_ prefix to make it clear this is internal
to the bdev library.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibda76e4d8e34dde0c2fe638cb965e5ba2d9e47b5

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475030
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-11-20 09:34:18 +00:00
Jim Harris
6c07e9d1a8 bdev: remove spdk_ prefix from spdk_bdev_io_submit
This function is not part of the SPDK public API, so
remove the spdk_ prefix to make that more clear when
reading the code.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5ea4c04c474a6cf0862f11921daa97c9df728e96

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-11-20 09:34:18 +00:00
Jim Harris
c3aaaa0181 nvme: don't monitor hotplug events in secondary process
NVMe hotplug must be monitored in the primary process -
DPDK doesn't support trying to handle it in the
secondary process.

This issue was somewhat masked previously in secondary
processes, since usually it would just probe(NULL) which
meant probe all attached NVMe controllers.  So in the
secondary process, we would probe just once, and create
the hotplug fd - it would never actually try to monitor
it.

But when explicitly specifying multiple trids in a
secondary process, probe would get called multiple
times.  First time would be fine since it only creates
the hotplug fd.  But second time would segfault since
monitoring for hotplug requires checking the DPDK-allocated
context which doesn't exist in the secondary process.

Fixes issue #1063.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2a9a91e222c206034293d90e30e3f598c8d7baa8

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-20 09:34:18 +00:00
Jim Harris
27e88b8d91 nvme: add g_ prefix to hotplug_fd
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8cc03e1a8b5d2eb28bf945115f3c9b3980b30f1c

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-20 09:34:18 +00:00
Vitaliy Mysak
506a90a90b vhost: wrap some of rte_vhost functions
This patch introduces indirection layer for session management
functions that makes it possible to switch their underlying implementation
if we want to (in unit tests for example).

Change-Id: I563c97bc65d55cc42fecbd1b7eb6679e394784a2
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470459
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:33:43 +00:00
Vitaliy Mysak
0a089121f6 vhost: move memory registration functions to vhost_compat.c
This change will allow us to move other DPDK specific functions
to rte_vhost_compat.c, such as session callbacks which are
the only consumers of these memory management functions.

Change-Id: Ieb7b3f08ddf2e7cf04ecf18e8af4ad04124ccfea
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470458
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:33:43 +00:00
Vitaliy Mysak
3f7e2ba18e vhost: move global definitions to common header
Move FLOOR_2MB() and CEIL_2MB() definitions to spdk_internal/memory.h
because that is where common memory functions are located.

Change-Id: I0d366686f86520e5564be07254d98a579faa3650
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471713
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:33:43 +00:00
Darek Stojaczyk
6231a197f5 vhost: simplify memory registration functions
Make them accept a struct rte_vhost_memory * param
instead of a full spdk session object.

Change-Id: Ibe12f29146d2e971ef58fd5f7366cb059123224d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2019-11-20 09:33:43 +00:00
Vitaliy Mysak
30998f9b1f vhost: return error codes for all session callbacks
Change type of `vhost_stop_device_cb()` and `vhost_destroy_connection_cb()`
to return response code instead of "void".

While DPDK callbacks `stop_device()` and `destroy_connection()`
do not have response code, it does make sense to have them in
our VHOST wrappers because those actions can fail.

Practical benefit we get by adopting this change is that we can
now use high level `vhost_stop_device_cb()` and `vhost_destroy_connection_cb()`
in unittests and check if they succeeded or not.

Change-Id: I2cd1886728b1edce7946e87db7ca0ac435e83a41
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:33:43 +00:00
Vitaliy Mysak
0e96d724d8 vhost: move session callbacks to rte_vhost_compat
This will allow us to write some more interesting unit tests
  because we can now mock high level callbacks instead of
  low level DPDK API and also, in future, we won't have to deal with sem_wait()
  in our mocked implementation since it's a DPDK specific thing.

Change-Id: I9ed5cff216e750685c00e718025ff1802fbe32c8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:33:43 +00:00
Darek Stojaczyk
1dae563373 vhost: move rte_vhost socket creation to rte_vhost_compat.c
rte_vhost_compat.c will now not only handle vhost-user
messages over the unix domain socket, but also setup
that unix domain socket with rte_vhost's APIs.

What was previously called vhost_dev_install_rte_compat_hooks()
is now called vhost_register_unix_socket() and is responsible
for creating the entire unix domain socket.

This enables us to write more advanced unit tests for vhost.
Instead of mocking low-level rte_vhost APIs, we could
now potentially mock vhost_register_unix_socket() and
create vhost devices and sessions without any actual
unix domain sockets involved.

Change-Id: Ifb18b92b37915c3f683b6d4fcdcc9259a3770561
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470455
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-20 09:33:43 +00:00
Darek Stojaczyk
8e68dfd99f rte_vhost: unaffinitize the thread inside rte_vhost API function
We used to call rte_vhost_driver_start() under
spdk_call_unaffinitized() because that function could
spawn a new pthread and we didn't want to to be pinned
to the one single cpu of the current SPDK reactor.

New DPDK versions (>= 19.05) already unaffinitize the
pthread by themselves, so our spdk_call_unaffinitized()
was only required for the legacy, internal rte_vhost fork
in SPDK. To clean up SPDK code, move the un-affinitization
down to the rte_vhost fork.

Change-Id: I53836517e9ec2ff366b509f00e1403845e3c3172
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-20 09:33:43 +00:00
Shuhei Matsumoto
6bf57c7fd1 lib/iscsi: Remove unused spdk_iscsi_task_get_cmdsn()
This is not used anywhere now.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If65321abcd3601af91725c2117cdce10dd0ffc63
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-19 09:10:14 +00:00
Shuhei Matsumoto
c5ef36742a lib/iscsi: Move function declaration to param.h to match function body
Four functions are defined in param.c and so moving their declaration
from iscsi.h to param.h is a little easier to read.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id16eab56d20d7ec99759e69525e791b091a93783
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2019-11-19 09:10:14 +00:00
Shuhei Matsumoto
b7b9855651 lib/iscsi: Factor out non-read task completion from spdk_iscsi_task_cpl()
Read task completion has been factored out into
process_read_task_completion(). Factoring out non-read task completion
into process_non_read_task_completion() makes the code a little
clearer and makes us possible to add unit tests.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4da3cd05fc3668d0db4436301e4bcb1b554de7cd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-19 09:10:14 +00:00
Shuhei Matsumoto
cdb7398dca lib/iscsi: Factor out freeing iSCSI tasks to a helper function
iSCSI target frees iSCSI tasks when exiting connection or removing
LUN.  The difference is only that the passed LUN is NULL or not.
To make the code clearer, this patch factors out freeing iSCSI
tasks from iscsi_conn_free_tasks() and _iscsi_conn_remove_lun()
into _iscsi_conn_free_tasks().

The refactoring has subtle cases and so add UT code together.

The next patch will fix the issue that secondary tasks are left
even after primary tasks are freed when exiting connection or
removing LUN, and this patch clarifies the next patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I18aaed6fe18a1c561ac88a0e5dc1296f9941d0e8
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-19 09:10:14 +00:00
Richael Zhuang
ac417aa9d6 util: use the crc32c in isa-l if isa-l is enabled on aarch64
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I9b72f8a657f22d06e965b6b55697a5ff2b0cc61b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474161
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-11-19 09:09:53 +00:00
Seth Howell
d6034240f1 bdev_nvme: pass nvme_bdev_ctrlr to bdev_nvme_reset
This is the data structure we end up using here anyways, and it will
make this function a little bit more versatile.

Change-Id: I530cb5b1b94f57cad4bb3931fc4b7a6335b6a00e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473742
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-19 09:09:40 +00:00
Seth Howell
5ad0db1cf2 bdev_nvme: allow bio to be null in resets.
Also, add a small callback function to display an error when reset
fails.

Change-Id: I74c9e9f6842cbd3b608eae2178bd3605cd642fbd
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473741
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-19 09:09:40 +00:00
Seth Howell
0cdf3f5189 bdev_nvme: rename _bdev_nvme_reset_done.
The new name, _bdev_nvme_reset_create_qpairs_done will help distinguish
it from a future function _bdev_nvme_reset_complete which will become
the new completion point for a reset.

Change-Id: I4f538a57e3c3de6c21afdd79febcdf01ad079342
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474606
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-19 09:09:40 +00:00
Michael Haeuptle
92df995526 json: increase the json rpc client value limit
The jsonrpc client has a limit of 1024 JSON values per
request which is hardly enough for any meaningful config.
For example, calling getbdevs for 24 NVMe drives require
~2300 JSON values.
I kept the original 1024 limit for the RPC server where
it makes sense to have a smaller limit and introduced
a seperate limit for the client.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: Id0300991b76151e4003e323f5ea29bc5fc0d2d11
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473568
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:16:49 +00:00
GangCao
a31ae23f38 test/bdevperf: add the QoS testing
Add a required check that the reasonable IOPS limit is quarter
of the measured actual IOPS without QoS. As I/O size is set
to 4K, so that bandwidth limit is also able to meet once the
IOPS limit is achieved.

Change-Id: Id9a401ce59f97245c505ebe0d1f4c18049177132
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-18 13:15:45 +00:00
Maciej Wawryk
809a6f82de scripts/vagrant: Allow running multiple VMs in parallel
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7047c3152558911a782be86e5ccf8214f2418c02
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:15:17 +00:00
Maciej Wawryk
ff231506d3 scripts/vagrant: Add more NVMe emulation options
Current scripts/vagrant allow us to create only a single file for
emulating NVMe with single namespace.
This patch allow to create multiple emulated drives with multiple namespaces.
This patch also allows to emulate OCSSD disks which are used in OCF tests.

Change-Id: Ia7a4667bd405ba18235e72bcdf608270002b5242
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:15:17 +00:00
Richael Zhuang
8cad9cc10a spdk: enable isa-l by default aarch64
In addition to x86, enable isa-l by default on aarch64.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I2db3f248bdc857326ec9b6f4b040a95d2a2ca1ca
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:14:41 +00:00
Seth Howell
c4d4800a46 test/nvmf: add a test for bad discovery ctrlr.
Make sure we don't crash or do any other bad memory accesses when we
fail to connect to a discovery controller.

Change-Id: I7c3e25899bc78f4beff9ad017612bacf286d52c7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474153
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:06:49 +00:00
Seth Howell
08d5d8474b test: fix SC2103 errors on older shellcheck.
Shellcheck version 0.6.0 reports all instances where we cd .. or cd - as
instances of SC2103. The version we are using on the test pool must be
newer, because we don't get the same errors, but for people running
check_format locally, it can make it difficult to parse the output.

Change-Id: I09f81a83c6f37480f13c36eb622e500364a1c437
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2019-11-18 13:06:49 +00:00
Ziye Yang
2df770535a test/ceph: fix the dev name in start and stop.sh
I am confused with the dev_backend variable, I think that
we may not need this variable.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I26fa5e8a829e574bb1b97add753b21ad1b0fc23e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 13:01:58 +00:00
Jin Yu
301c5aeec9 pci: fix the hotplug issue
DPDK intr thread is designed that it can't unregister the src
callback in this callback handler. So I think we can't detach
the PCI device in the hotremove callback as it needs to unregister
the VFIO notification callback which will be not successful
but it still can free the device. So at the next req notification
in the handler function, we meet the freed device.

Fix #994

Change-Id: Id4b45a2d0fe6b45b132355d59471bc80240fad70
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473176
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-11-18 12:54:01 +00:00
Maciej Szwed
c330fbe618 bdev/nvme: Correct variable name
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: If7d55636eed5e03d002ca6ea6a24d96b127ae705

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471621
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-11-18 12:53:17 +00:00
Maciej Szwed
c30609c9ea bdev/nvme: Allow multiple bdevs per namespace
Some future NVMe namespace types will be able to
create multiple bdevs per one namespace. This
patch makes it possible.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I47b4c1fc545c59dcc3171ab0960f1835b6aa6d24
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471620
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 12:53:17 +00:00
Maciej Szwed
77dcf4534b bdev/nvme: Use Namespace object for tracking bdevs
In future implementations it will be possible to create
different types of namespaces (standard, OCSSD, ...).
This patch introduces new nvme_namespace type which
will make possible to implement such cases.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I27747d3985915f45c0e0a28dd5f391cca06b13cc
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471273
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-11-18 12:53:17 +00:00