Commit Graph

11243 Commits

Author SHA1 Message Date
Maciej Szwed
941d9e7aa8 nvmf: Add support for compare op command
This patch introduces new spdk_nvmf_bdev_ctrlr_compare_cmd
function which implements support for compare operation.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Iadf402a6441a78ea0e6468f1066c6b0e10e63b9b

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477782
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-27 17:39:52 +00:00
Maciej Szwed
05e7f56c3a nvmf: Add spdk_nvmf_bdev_ctrlr_compare_and_write_cmd function
This patch introduces new function that is a part of
upcoming support for fused commands.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I019c587bee7fd0f745ec17c141baf4cb7bf86645

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476611
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-27 17:39:52 +00:00
Tomasz Kulasek
67c9c1c5d8 lib/nvmf: add fused operations
Change-Id: If3162a5683d1c57011f9a66cbcfe47ba161734bf
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476138
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-27 17:39:52 +00:00
Maciej Szwed
adf90938b1 bdev: Add spdk_bdev_io_get_nvme_fused_status function
Added new function for getting NVMe specific return code
for fused commands. Also changed one of the return codes
in fused commands so that we could distinguish error
cases.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I86417ea4f5b8f3e6496162be3d6c6128076e35d4

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481666
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-27 17:39:52 +00:00
Maciej Szwed
d417768c00 nvme: Fix define name
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ice5b7ca3ed3fef93636db064d7f5fcafa9cf2d3b

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481771
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-27 17:39:52 +00:00
paul luse
87dcedb817 docs: add section for containers
Starting with how to containerize an SPDK app for Docker by example.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ice2cbf0ab54f2c541e38508a133a7ef0f23dd40e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479900
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-24 20:16:58 +00:00
Karol Latecki
6df2fa8c2e test/vhost_perf: refactor test scripts to use disk map
Use provided configuration file via --disk-map option
instead of creating bdevs and VMs in ordered sequence
(e.g. 0 to 10, etc.).

This allows to:
- specify which NVMe device we want to use
  (PCI BDF identifier)
- how to name it in  SPDK nvme bdev configuration
- how many splits or lvol bdevs to create on this device
- which VMs should use created bdevs

With CPU mask configuration file this allows to better
control resources when running the test (especially in
case of NUMA optimization where using sequential for/while
loops is not a good approach).

vm_count and max_disks parameters removed. These are not
needed anymore as they're controlled by config file.

Example of config file contents:
(BDF,Spdk NvmeBdev name,Split count,VM list)
0000:1b:00.0,Nvme1,2,2 3
0000:89:00.0,Nvme3,4,4 5 6 7

Change-Id: I9fc73458825d8072537aa04880765a048e034ce4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464565
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-24 15:13:00 +00:00
zhangjf
2a00a12892 vhost_blk: need init task when resubmit the blk request
Change-Id: I10fca86be6a2834fe3238d8881a4645ac810a201
Signed-off-by: zhangjf <zjfhappy@126.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482346
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-24 08:07:13 +00:00
Karol Latecki
006abfecb8 doc: add 19.10 NVMe-oF RDMA performance report
Change-Id: I9a99af32d7c85c2d3d43469ae7f7786f8e62d8e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482547
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-24 08:06:56 +00:00
Seth Howell
08add32896 test: add test to validate dpdk mem utility
This test essentially confirms that the new RPC properly creates a test
file and that the script can parse that file and run through a basic set
of operations and exit without an error.

Change-Id: Idf0c831020696a3a62fcef13171eedf3fcf63f5b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-24 05:46:43 +00:00
Seth Howell
0e9727db73 scripts: add a parser for the dpdk memory stats.
This script takes an optional dpdk stats file argument and tries to
parse the information from that file into a comprehensible structure. It
then provides a basic UI for printing that memory information at various
levels of granularity.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ibbb09f41122be18b7e640781f3e140618e52c0ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477635
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-24 05:46:43 +00:00
Mike Carlin
076821454d lib/nvme: fix hotplug trtype 256 not available
There was an issue in the hotplug poller where it would fail to
probe the added/removed nvme pcie devices due to an error trying
to find the PCIe transport type. This happened because the
`struct spdk_nvme_transport_id` needs to have its trstring filled in
after a change was made to get transports by name to allow for custom
transport types. This change fills in the trstring so that downstream
checks correctly pass.

Fixes #1159

Change-Id: I35d2834f3ba58a8e6f8e91d290c1f4cb9c158e5a
Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-23 16:40:29 +00:00
Seth Howell
85fcc49fd4 nvmf/rdma: fix call to spdk_nvmf_rdma_listen
This change fixes a merge incompatibility between commits
50cb6a04ac and
708ed4fb6e.

Change-Id: I5bc71a3c214667f01de66857cf61b9eb25f6cf6b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482586
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 16:38:46 +00:00
Seth Howell
ca693eaba8 lib/nvme: fix cm event handling during rdma qpair shutdown.
In the event that we have more than one event outstanding for a qpair
at the time of destruction, we need to ack all of the events, Luckily
the synchronization is already there in the form of the ctrlr lock.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib297598f2e28d9b9bd83e904f950795a61fa883a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-23 15:14:55 +00:00
Seth Howell
50cb6a04ac lib/nvmf: handle RDMA_CM_EVENT_ADDR_CHANGE
This allows features like transparent failover on
target RNICs.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iab494ad3e9e4efea4db9cbb30bc18ea5b584f345
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478879
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <jacek.kalwas@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-23 15:14:55 +00:00
Jim Harris
12ae64f863 Revert "test/iscsi: add application for target fuzz testing."
This reverts commit 4700ef0fa6.

This has merge conflicts with the iSCSI async write patch
series that was merged.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5a27460a369ef5f13bf490a287603e566071be40
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482482
2020-01-23 06:19:00 +00:00
Konrad Sztyber
42226f33ed bdev/ocssd: store nvme_bdev_ns in a separate variable
Change-Id: I5a8cb443929c0749aa50e19da1a57244479d2895
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482411
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 04:05:39 +00:00
Konrad Sztyber
d5ffabca72 bdev/ocssd: verify namespace population in notification cb
Each namespace should be checked if it is populated when handling chunk
notifications.  Otherwise we risk segfaulting if a namespace gets
depopulated before the notification callback is executed.

Change-Id: Ic55104a52087b1ea7090eeaede3e2221682cd331
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482410
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 04:05:39 +00:00
Changpeng Liu
56c7da0e3b bdev/nvme: check the return value for spdk_nvme_probe_poll_async()
When there is an error returned from low level driver, we should unregister
the poller and free the context, or it may get double free when reaching next
poll round.

Fix issue #1156.

Change-Id: I34ca605f11249b885756d761291aebbb7a382d7e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482215
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-23 02:41:41 +00:00
Shuhei Matsumoto
2c8ddd08bd module/crypto: remove need to allocate a buffer for writes
Use new bdev aux buf feature.  Huge performance benefit for writes.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5a27460a369ef5f13bf490a287603e566071be41
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478384
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-23 02:36:51 +00:00
paul luse
82a4c84fec lib/bdev: Add support for new per bdev_io auxiliary buffer.
A new API was added `spdk_bdev_io_get_aux_buf` allowing the caller to request
an auxiliary buffer for its own private use. The API is used in the same manner that
`spdk_bdev_io_get_buf` is used and the length of the buffer is always the same as the
bdev_io primary buffer.  'spdk_bdev_io_put_aux_buf' is called to free the
auxiliary buffer.

The initial use case is crypto, in the next patch in series it is used. No UT were
added as the logic isn't that complicated and it is fully tested with each run
of crypto.

Fixed a comment typo also (not mine for once).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib1939fcbc8e5db36fd909ef26771a725a551e8e6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478383
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-23 02:36:51 +00:00
Hailiang Wang
4700ef0fa6 test/iscsi: add application for target fuzz testing.
Enables us to test randomized data against the iSCSI target interface.

Change-Id: I9ff9a06c11bb16b315686156b27855664f21bd48
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470925
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:32:28 +00:00
Ziye Yang
74ce72edca lib/iscsi: Using async writev for ISCSI_OP_LOGOUT_RSP PDU
Change-Id: I9d6d547645930c5075dca7d1e8c566634cda8e73
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482028
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
16d5a6155a lib/iscsi: Using async writev for ISCSI_OP_LOGIN_RSP PDU
Change-Id: Ia69c996c731dfd89702bbb28468d8798c391034d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481922
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
67067ea4de lib/iscsi: Add a helper function iscsi_conn_params_update
Purpose: To reduce the duplicated code.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I6f6e79af602281ed50fa0fde7651238065c9bd31
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482291
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
b03612bf4d lib/iscsi: Using async writev for ISCSI_OP_TEXT_RSP PDU
To avoid partial write issue of this PDU.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Id9b22da844c75ae53c6881850d192b40ac4098ac
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481948
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
e199f1a5b4 lib/iscsi: adjust the location of spdk_iscsi_param_free in two functions.
Purpose: To prepare for the further patch submission.
Since we do not need to keep this variable too late.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ibaa100925e1ea317253d4fe7e560917e063fcf6b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482290
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
d8d1168c06 lib/iscsi: Add real callback for DATAIN PDU complete
Since only after DATAIN pdu sending out, we can have
free slot to handle queued data in tasks.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I49a52597e8660453ea90c5960d020eb53f81265d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482048
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
098d32273a lib/iscsi: Add two parameters in spdk_iscsi_conn_write_pdu
This is prepared for the further call back usage.

Change-Id: Iccf304c87e67debfb4e7c330acc9cc233cc3ec48
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481917
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
d648dde682 lib/iscsi: Use asychronized writev for sending data on sockets
This patch eliminates the flushing logic and simplies
the writev logic. And this patch can also improve the performance.

We support async write for PDUs other than login response, logout response,
and text response in this patch. We will support async write also for them
later in this patch series.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I243f598f297d594da0bb18466bc47dab918ed3ee
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481686
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
377a016f69 lib/iscsi: add the conn in spdk_iscsi_pdu
Purpose: Prepare for the async writev usage for spdk
iSCSI target application.

Change-Id: Iff0e932159b0ad80be32aed3fc543b67d8fb8f51
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481644
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-23 02:29:58 +00:00
Ziye Yang
be6aa42708 sock/vpp: Add spdk_vpp_sock_writev_async support
Purpose: To support SPDK iSCSI target applications
when it uses spdk_sock_writev_async.

PS: for some duplicated code between posix and vpp,
we should consider it later. Since if we do it early,
if it cannot be abstracted in the common header,
it will invoke addtional work.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I30d8ee81f80ea5e74c53ff726ee44b0612867c71
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481749
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-23 02:29:58 +00:00
Anil Veerabhadrappa
9857ff6015 nvmf/fc: Fix compilation errors in FC ut
Fix includes passing correct argument to spdk_nvmf_get_transport_ops()
and also add the spdk_nvme_trid_populate_transport() function stub.

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: Ic470dbf7c4c674531358df2dec14a09c12a41f4e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482199
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-22 13:54:34 +00:00
Changpeng Liu
5e8a3a77b6 vhost: make SPDK internal vhost library can work compatible with live recovery
We will not enable the live recovery feature for SPDK internal vhost library,
so we unmask the protocol flag for internal vhost library.

For the purpose to make it can be compiled with latest DPDK version, some
mandatory APIs are required, so add them here.

Change-Id: I34fab7ed90c86a0fb612852a47f6cadeb8a072f3
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482069
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-22 13:54:19 +00:00
Changpeng Liu
ca073ef3ac nvme/fio_plugin: also set appmask/apptag when PRACT is enabled
Previously we only initialize the DIF context when PRACT is 0, because
the DIF library can only support that case, but when end-to-end data
protection feature is enabled and PRACT is set to 1, the controller
will help to check the metadata, but we still need to pass
appmask/apptak to controller.  This patch will fix this case.

Change-Id: Ia62d4f8a7adf822b75541f69ce57aeff8f9eb505
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482047
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-22 13:54:06 +00:00
Jacek Kalwas
a715543325 nvmf: change default connection scheduler
CONNECT_SCHED_TRANSPORT_OPTIMAL_GROUP is preferable to use by default
for rdma/tcp and CUSTOM transport types.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I1d0e89e6e48e69bfd3c99247c79193f75a4bf264
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481770
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-22 13:53:45 +00:00
Jan Kryl
2167c68d18 lib/nvmf: nvmf target stops to listen when subsystem is destroyed
There is a spdk_nvmf_tgt_listen() which opens a port for specified
transport (trid) which opens possibility to accept new connections
from initiators. However there is no counterpart of this function
(i.e. spdk_nvmf_tgt_stop_listen()), which would stop listening.
Instead the current code relies on spdk_nvmf_subsystem_destroy()
to stop the listener, which seems to be wrong.

Fixes #1129

Change-Id: I6e73d8c234dc451f0fee8394132eae34cd4f4756
Signed-off-by: Jan Kryl <jan.kryl@mayadata.io>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479873
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-22 13:53:34 +00:00
Or Gerlitz
8e8a5f7c28 nvme/tcp: Use writev_async for sending data on sockets
Amortize the writev syscall cost by using the writev_async socket API.

This allows the socket layer to batch writes into one system call
and also apply further optimizations such as posix's MSG_ZEROCOPY
when they are available. As part of doing so we remove the error
return in the socket layer writev_async implementation for sockets
that don't have a poll group.

Doing so eliminates the send queue processing.

Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Change-Id: I5432ae322afaff7b96c22269fc06b75f9ae60b81
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475420
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-22 13:53:09 +00:00
Or Gerlitz
e61b0904a8 sock/posix: Add flush
Initiator drivers (e.g nvme/tcp) don't use poll groups but rather directly
poll the qpair. In this case we want to allow the polling function (e.g
_qpair_process_completions()) to flush async writes pending on the socket.

Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com>
Change-Id: Ibd8c73691213d58e287b7110d0f5a381a89a64d0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-22 13:53:09 +00:00
Tomasz Zawadzki
1fdee03c3c lib/blob: split loading next md_chain to separate function
Replaying md through _spdk_bs_load_replay_md_cpl() starts with
md page 0 in search of first valid md page starting a chain
for particular blob.
When it is found, next pages read are from the current pages
`next` page - next in chain.
After whole chain is read, it goes back to first page in chain
and starts search for next valid chain from there.

This patch adds separation between reading particular chain,
and moving to the next one.
Moving on to the next one happens in _spdk_bs_load_replay_md_chain_cpl().

Further in the series, extent pages will be added in the metadata.
Those are not within any particular blobs chain of metadata,
but spread out over the md region.

It is not enough to read all md and read extent pages.
In case of power failure, only extent pages known to be valid
are the ones which are pointed to by some valid md chain.

In futher patches, a step will be added after reading particular
valid md chain to go read extent pages pointed by it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6e7cd64af66ce5db0abd2ad5962d604ac2b30994
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481900
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-22 13:52:49 +00:00
Tomasz Zawadzki
bb25821c7e lib/blob: move finishing unload to _spdk_bs_unload_finish()
Moved finishing of unloading to separate function,
which is now called on every failure and success when unloading
the blobstore.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I34539b78c5cc63a6fe5891014cba89b9eb62d4df
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482009
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-22 13:52:49 +00:00
Tomasz Zawadzki
f7bd1e1eb9 lib/blob: check bserrno on each step of bs_load
Before this change it was possible to fail at
writing out some of used md pages.
bserrno output of those was not verified.

This patch adds it at every step.

With that two function don't need (and never needed)
to pass the bserrno:
_spdk_bs_load_write_used_md()
spdk_bs_load_complete()

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1a61763f03665ba1b00e5949ef0cf37eefaaf08f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482008
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-22 13:52:49 +00:00
Tomasz Zawadzki
cf5df9b41d lib/blob: remove seq argument from _spdk_bs_load_ctx_fail()
This is simplification of load path.
seq is save in ctx already, no need to pass it to the function.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ief0ddc1826c461adbad71ba1a3897c510ec2a971
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482007
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:52:49 +00:00
Karol Latecki
917976766d autotest.sh: add hello_blob to BlobFS tests
Hello_blob example was never run before in autotest.

Change-Id: I1f858bed816471efed947f06107cec981dd562a4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481570
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-22 13:52:22 +00:00
jiaqizho
f1687a1bd2 script: add iostat.py to check bdev iostat
this tool likes sysstat/iostat
added to qos test

Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Change-Id: I7ed6538095d34fa8c5835f28b25d618f1b3d81b7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476537
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-22 13:51:41 +00:00
Seth Howell
253fb41b65 autobuild: Confirm we are building all files w/ scanbuild
This gives us an assurance that we know scan-build is being run against
all relevant files in the repository.

Change-Id: Ic0b871e98a9ea7acd2d6b2a99ab81955af29fc66
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479898
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
de30327534 test/blockdev: parameterize the blockdev tests.
Now we can actually know which tests we are running on which bdevs when
we call blockdev.sh. Removing the conditionals from the configuration
setup (or failing if they don't succeed) and then calling run_test from
the top level allows us to reliably track the bdev tests and fail if
they don't run on a desired class of bdev.

Change-Id: I14e6e3d993b4af4995adcbc5f138bac4ae9d63be
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
6ac949c0d1 test/blockdev: check bdev list before running unmap
Some bdevs don't support unmap so they may not be able to run the fio
unmap tests. In case none of the bdevs we are testing support this
operation, the test will be skipped.

Change-Id: I5063297f0378bc87abf50e5311d147251cfe0ad9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478337
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
5883d34300 test/blockdev: parameterize the nbd test.
This allows us to run the NBD tests on any combination of bdevs
based on the flags passed to the script. Previously we ran the tests
on a specific grouping of bdevs and opted to only free one type (passthru)
over RPC after the test ran. Rather than free all bdev types in the new
implementation, I opted to not free any of them via RPC and rather allow
the application to close them at termination time.

Change-Id: I574db4f941bfbb073cceed4610e1f5d43028569f
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-22 13:50:17 +00:00
Seth Howell
b2235c1f97 test/blockdev.sh: parameterize hello_world test
Just to make sure that we always test against a valid bdev.

Change-Id: I7c9786f32b8d5ff5f2bf9c86151fb7f2bdfdf5f3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478245
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-22 13:50:17 +00:00